Having released the ever popular Job Simulator as a launch title for the HTC Vive back in 2016, Owlchemy Labs is one of the most veteran VR game studios that exists today. Over the years the studio has built a strong foundation of VR interaction design which is seen throughout their newest title, Vacation Simulator. Interactions which might seem simple and usable to the player are often much more complex than they appear. Case in point: some surprisingly brilliant paintbrush tech that just feels right in VR. Developers from Owlchemy are here to explain how they built it.
Guest Article by Peter Galbraith & Zi Ye Peter (Implementer of Unityisms) and Zi (Developer, Physics / Math Genius) are both developer/designer dual-wielders at Owlchemy Labs. Their work spans design ideation and prototyping to iteration, programming implementation, and testing.
Both are important contributors to the legacy of absurd and highly VR polished games at Owlchemy Labs, including: award-winning title Job Simulator, Emmy-nominated Rick and Morty: Virtual Rick-ality, and the recently released Vacation Simulator, which is also coming to PSVR and Oculus Quest later this year.
Hey everyone!
Pete and Zi here. We’re both developers at Owlchemy Labs, and we’re excited to talk to you about one of the most highly-iterated features in the entire Vacation Simulator: Painting!
Painting is one of our most colorful activities in the Forest level, a creative space tucked away in a Treehouse where you can unleash your inner artiste. Whether you wield our super-gooey paintbrush to make a masterpiece from scratch or use a photo from the in-game Camera as a starting point, aesthetic greatness is always within reach. However, like all great features, Painting went through several prototypes and iterations before we arrived at our picture-perfect result.
To kick things off, Zi will explain the tech behind the most important part of an artist’s toolkit: the paintbrush!
Simulating the Feel of [PAINTBRUSH]
One of the most challenging aspects of making Painting feel great was the paintbrush tip. As our single, powerful tool to express your creative vision in Painting, we knew it was necessary to address the expectations that come with having a paintbrush in VR, all the way down to its most technical nitty-gritty.
Like all the best features in our games, the squishy brush tip was created using fake, made-up physics! We start with a mathematical model consisting of a straight line that we shoot at the canvas, and then we figure out where the tip would bend along the surface. That bent line is used to manipulate the shape of the brush, like so:
This helped us solve one of the biggest issues we had with painting in VR: lack of feedback. With current VR hardware, we can’t accurately simulate the feedback of a flexible brush pressing against canvas. Without this feedback, we found it was difficult for players to tell if the brush was making contact, causing them to put the brush too far into the canvas and creating jittery or hiccupy motions as the brush collided with the painting easel. This behavior led to a lot of ‘squiggly lines’ and often caused the brush to break out of players’ hands entirely from excessive collisions—not exactly something that made our players feel like art pros!
By providing visual feedback in the form of the squishy tip, fewer players pushed the brush as far into the canvas, making for fewer breakouts and prettier lines. We also paired this with an auto-respawn if the brush did break out of a player’s hand, making it both more likely the player kept hold of their brush and easier to grab again if they didn’t.
But again: it’s all fake! The brush hair doesn’t actually collide in the physics system except at the very base. The squishing action creates the illusion of resistance and tricks the player into thinking the brush is being pushed back.
We also use the same mathematical model to determine the size of the contact area of the brush and control the size of the paint dob applied to the canvas. This specific feature allowed us to ship with a single brush size, just with variable stroke width: lightly touch for a thin precise line or swab on the colors with a firmer pass.
The brush also factors in ‘hair stiffness’, which determines how the brush hair changes direction when it’s dragged on the canvas or when the brush handle is rotated. The stiffness is also used to add a bit of jiggle to the brush hair when it’s not in contact with the canvas. These little details let us get a super ‘gooey’ feel for the brush, and from our very first playtest it was clear that folks really responded to that perceived sensation.
The squishy brush solved many technical and UX issues, and it also played into that initial desire to empower players to create beautiful art. We had several devs that were convinced they “weren’t artists,” yet when put in front of the new painting easel made things like this:
The calligraphic size changes of the brush tip combined with our artist’s hand-picked palette let everyone create images that looked painterly and fun. Voila!
Now, as the first developer to put down code on Painting, Pete’s going to share our design process behind the feature (including its many iterations!):
Design Challenges: Back to the Drawing Board!
Every feature in Vacation Simulator started from the same series of questions:
- What do people expect when they vacation at a given destination?
- How would Bots hilariously misinterpret this activity?
We knew sketching, painting, and artistry were fun aspects of going out into nature, and when we thought about how Bots would misinterpret painting, manipulating photos instantly came to mind. We loved the idea of editing the photos you took around the island and customizing them to express yourself as a player. That simple idea was the seed for a feature that took over a year to design and bring to life.
Part of our early design ideas were influenced by what we’d previously made for Job Simulator (e.g. painting the sign in the Auto Mechanic; the painting software on the Office computer). Both were simple and fun, but not very deep. Painting in Vacation Simulator had to feel different—and bigger—to match the rest of the game. We wanted to give players more than just increased pixel density—you should be able to feel like you’re actually painting. Like real life, only better.
Many works of art require revisions to turn ‘meh’ into a masterpiece, and Painting was no exception. It was one of the earliest features we developed for the Forest and one we continued working on right up until launch! Here are a few examples of Painting experiments and iterations on the feature:
‘Photoshopping’ Only
At the very beginning, Painting only involved photos. Players would place a photo on the easel and then use the paintbrush to ‘paint’ the photo onto the canvas. Players could mix and match, combining bots or scenery from multiple photos, and even (if they got clever) take closeup photos of specific colors to paint with. The more people played, the more we realized that actual painting—complete with swatches of colored paints—drastically increased players’ ability to be creative.
Filters
Filters are a common element in photo manipulation software, so of course we thought the Bots would include them in their misinterpretation. We experimented with buttons to apply filters: pixelation, invert colors, black and white, and sepia. Pressing any of these filter buttons would stack effects, and the next stroke of the brush would apply the filtered image to the canvas. While interesting, filters often lead to frustration as players struggled to anticipate what would actually appear on the canvas as they painted. For example, with inversion, if you dipped your brush in the orange swatch, your brush would paint blue! We wound up removing filters from the easel entirely, instead integrating them into your camera as lenses. All the cool effects with none of the confusion!
Paint ‘Bucket’
Our biggest feedback from early playtests was the ability to start with background colors that weren’t white, and the option to erase paintings entirely. By adding a paint bucket slider that wiped a solid color across the canvas, along with a white paint option, we resolved both of those needs with a single feature. As a surprise benefit, it also enabled the creation of more… Bot-like paintings.
Blending & Transparency
The paint bucket used hard edges, but when looking at those same hard edges on the brush, it just didn’t feel realistic—or look that great. Jagged pixels definitely aren’t part of most real life paintings. For a more realistic effect, we created a transparency gradient representing the amount of paint to apply to the canvas. Used in conjunction with the size and position of the contact area, we could determine how much the paint color blends with whatever is beneath it. The longer the brush is in contact with the canvas, the more color is applied, allowing us to simulate how real brushes transfer pigment to fabric.
Multiple Brushes
We went through several iterations of brushes and brush sizes. Before the paint bucket slider, players wanted a giant brush to make things easier to fill. Then, after we implemented the slider, players wanted a smaller paintbrush for detail work. We had a smaller paintbrush for a long time, but ultimately cut it due to one of the most important gamefeel features of Painting: the squishy tip!
– – — – –
We Hope You [EMOTION] Painting!
Sometimes there’s just no way around the amount of iteration it takes to make something look, feel, and play the way you want. Painting was a momentous exercise in VR design affordances, player expectations, and utilizing constraints for the most creativity.
Thanks for checking out this peek behind the scenes of Painting! We hope you’re inspired to channel your inner Leonardo BotVinci in Vacation Simulator.
Read more guest articles contributed by experts and insiders in AR and VR.