Dr. Llŷr ap Cenydd is the developer of two well known Oculus Rift demos: Ocean Rift and (my personal favorite) Crashland. As an expert in procedural animation, Cenydd has imbued his creatures with beautiful and organic animation that's convincing and unscripted. In a guest article, he explains some of challenges and benefits of procedural animation and explains its importance in virtual reality. Dr. Llŷr ap Cenydd is the developer of Ocean Rift and Crashland. He holds a PhD in Computer Science and is currently a lecturer at the School of Computer Science, Bangor University, Wales, UK. His research interests include real-time computer graphics, procedural animation, artificial life and medical simulation. Cenydd blogs about his virtual reality development work on his blog, Llyr's VR Experiments. Most recently he announced that Ocean Rift will be coming to Samsung's Gear VR headset later this year. You can also follow Cenydd on Twitter @sfaok. The resurgence of VR has sent developers back to the drawing board in many respects; we have to rethink how we model depth and scale, how to interact with and move within a 3D world, how to display and navigate menus, load levels, present tutorials. Even staple graphics techniques like normal mapping and billboard particles are unsuited to VR due to a lack of stereo depth. The prospect of needing to maintain 60, 75, 90 frames per second also puts a much greater emphasis on game engine design and optimisation, especially now that we have an emerging mobile platform spearheaded by Samsung's Gear VR. "...one of the striking things you might find is how dead [virtual characters] appear without the breathing, blinking, eye contact and many other subtle queues we subconsciously perceive in living beings." Animation faces similar issues. If you try some of the more realistic VR human model tech-demos, one of the striking things you might find is how dead they appear without the breathing, blinking, eye contact and many other subtle queues we subconsciously perceive in living beings. While games continue to chase the elusive goal of photorealism, there remains a massive amount of work before characters in VR don’t regularly break the sense of immersion. Even the slightest hitch when an animation loops can be enough to pull you out of the experience. Furthermore, I believe this problem extends to data-driven animation in general. I'm developing an app called Ocean Rift that aims to immerse the user in a vivid underwater world. One of the main draws of the app is the ability to freely swim with large marine animals like dolphins, orcas, humpback whales, and sharks. If I was to animate these creatures using traditional techniques, their movements would largely be constrained by the data. While you can achieve fantastic results using a team of animators and advanced blend trees, there is still an inherent lack of realism when the underlying motion is driven by pre-set animation sequences. I wanted the animals in Ocean Rift to behave like they are real living things, capable of displaying the range of behaviour and dexterity shown by real-life equivalents, and crucially behave differently every time you visit. To get anywhere close to this I needed to animate them procedurally. Procedural Animation A glimpse at Ocean Rift shows the natural motion and convincing behaviors that can be achieve with procedural animation. Procedural animation is a type of computer animation where the motion is generated automatically rather than being driven by predefined data. When applied to autonomous characters, this is sometimes known as Behavioural Animation where the character determines its own actions and this is reflected in the resulting animation. "What you are essentially creating is a puppet that can pull its own strings..." With Procedural Animation, you can take a 3D mesh rigged with an articulated skeleton and bring it to life by connecting its joints up to algorithms. What you are essentially creating is a puppet that can pull its own strings, and with carefully designed and fine-tuned algorithms it can lead to very organic motion that is both unique and entirely contextual. The most famous early example of behavioural animation is Craig Raynolds’ Boids system, which he developed to simulate how creatures flock, herd, and school. The basic idea is that the complex motion seen in these natural phenomena can be simulated by prescribing three simple rules: don’t get too close or too far away from neighbours (separation and cohesion), and always steer towards the average heading of the local flock (alignment). Using just these three simple rules, it is possible to realistically simulate the motion of an entire flock of birds or school of fish. While this type of behavioural animation has been used a lot in film and games to animate crowds (it also animates the fish schools in Ocean Rift), it is still quite rare to find procedural animation systems in modern games. One of the biggest examples is the Euphoria Engine by Natural Motion, which has powered the animation system in many of Rockstar’s modern games. However even these type of systems are largely limited to situations where the sub-conscious brain takes over, like being knocked off balance or falling over. Continue Reading on Page 2... Crashland During my time as a PhD student, I looked at how to calculate motion automatically for limbed animals, concentrating on ones that have a very low centre of gravity like insects and lizards. What I came up with was a technique where the creature’s AI is divided into a series of components that all attempt to regulate a small part of the animal’s pose or movement. For example the legs need to maintain balance relative to their neighbours and centre of gravity, the body needs to maintain a set height above the surface, and the thorax-abdomen needs to pivot around corners. The perceived aggregate motion resulting from all these components interacting in real-time was an ambulating creature—they could even climb walls! An example of a spider driven by Llyr's animation system Of course while this technique was grounded in robotics and biological research, this is only a gross abstraction of how the real things move. However the key thing is your brain doesn't care—if it sees a movement that obeys the laws of physics it will fill in the blanks and attempt to contextualise and even anthropomorphize the perceived motion. For example when the aforementioned spider is finding it difficult to successfully pivot around a corner the results are interpreted by the viewer as a struggling creature rather than a failing algorithm, and it can actually enhance the overall realism as perfect robotic movements are out of place in nature. "...you can generate any kind of death animation you want, from straight flops, twitches and tumbles, to gross spasms, and even drawn out hammed up death throes." I applied some of what I learned in this research while developing my motion controlled FPS VR demo Crashland, shown in the video below. When you frag one of the giant alien insectoids in the game, the animation system transitions from being kinematic (data-driven) to fully procedural. Essentially what happens is the ragdoll (physics driven skeleton) is activated and joint angles are pulsed through the articulated figure. By varying the type of pulses being sent through the virtual muscles you can generate any kind of death animation you want, from straight flops, twitches and tumbles, to gross spasms, and even drawn out hammed up death throes. See Also: The Next Version of Crashland Looks Amazing The great thing about procedural animation is that it can turn even a simple action (falling over, blasting alien spiders) into something that rewards you with a unique animation directly tied to the gameplay. I believe this can make all the difference in VR as it imbues a greater sense of organicity in the creatures you're interacting with. Continue Reading on Page 3... Ocean Rift In the original Ocean Rift demo I essentially had a bunch of models with a looping animation moving around underwater. This was a great proof of concept but once I added the ability for the player to swim the illusion was reduced; the animals were basically just models moving around on rails after all. Early this year I joined Samsung and Oculus VR's Gear VR initiative to develop Ocean Rift for their (incredible) mobile device. The project has been immensely challenging in many ways, and it has led me to develop a generalised animation system for aquatic animals. The primary reason for development was aesthetical; as mentioned, I wanted these creatures to behave as realistically as possible—but arguably procedural animation offers several advantages for mobile devices. While the CPU overhead of animating something procedurally is probably greater than moving through a simple blend tree, it can be very fast if there’s no physical simulation taking place, and there’s no memory overhead in storing animation clips for each animal. The motion and behavioural space is also inherently limitless, as everything is calculated on the fly. The first animation system I developed for Ocean Rift was a sea snake. Previous experiments in creating and animating the giant Hell Worm monster from Crashland (described in this blog post) had led me to produce many prototypical animation systems, including one based on snake ambulation. Applying this to a sea snake model and linking it to a rudimentary AI, that pulsed waves through the virtual muscles, generated some quite convincing animation as you can see below: The key thing here is that the snake’s animation will always appear reasonably plausible because its joints are always moving in a continuous smooth pattern. Provided that you extrapolate a decent movement vector from this animation it can look very convincing as something physically swimming around. Extending this system to a shark required the addition of other scripts that controlled various functions of the body such as the fins, tail, and jaws. In the following video, the parameters that drive the shark’s behaviour are randomised every few seconds. You should notice that sometimes this leads to very erratic motion of the creature; this is essentially due to the animal moving it’s backbone at a high frequency. However, perceptually you see an agitated great white shark exhibiting behaviour very similar to what we find in nature. As the animation system became more advanced, I’ve been able to animate and produce increasingly elaborate and complex behaviours. For example the following footage shows an early prototype Kronosaur (now extinct) swimming around, doing barrel rolls and snapping at invisible targets. Fun fact: Adult Kronosaurs were 10 meters in length, about the same size as a bus. However, currently the most complex animation system developed for Ocean Rift is exhibited by the dolphins. Being the most excitable, acrobatic and friendly aquatic creatures they also represent the biggest animation challenge. At first the dolphins are distant creatures swimming around but eventually they start to investigate and interact with me. You will also see them exhibit a variety of behaviours that are totally emergent like twisting, twirling and diving. The cool thing is this encounter will be completely different every time. The ability to bring virtual creatures to life has always been of great interest to me, and with the advent of VR I can now actually meet and interact with them in the virtual world which is immensely exciting. This industry is in its absolute infancy but it’s going to be really fascinating to see how it develops over the coming years, and just how far procedural animations systems like this can be taken to give the player totally unique and evocative experiences.