Cabling

This is a training and assessment application that lets users operate networking devices in a virtual datacenter. One very interesting feature is that all elements are synchronized in real-time with actual Cisco devices running in the cloud, creating a bridge between the virtual world and real-life.

The most challenging part was getting the cables right. To create each cable, we used a skinned mesh with 22 rigid bodies chained together using joint primitives. Physics-driven objects are always a potential source of bugs which made me spend a great amount of time making sure that cables would not only feel realistic but also would respond well to abuse.

Since there are many interactive elements so close to each other, it’s also very important to fine-tune many other details such as visual cues, haptic feedback, and smooth grab/release transitions. All these features working together can turn a complex interaction into something that feels natural.

Rope & Tire

This interaction showcases part of a training session that requires the user to scan different elements looking for smuggled items.

Rope physics can be tricky to implement, and even more so in VR if you give the user freedom to swing them around. The rope in this video is modeled using approximately 20 bones that can be grabbed individually. The narrower part that goes through the hand is simulated by scaling the bone when it’s grabbed which makes the mesh shrink along that point. For such a simple operation it provides a significant visual impact.

The tire at the end makes it difficult to move the rope around, which increases the perceived weight. To simulate the haptic feedback I use the movement of the tire itself, sending signals that are proportional to the relative speed at which it is moving, an approach that works really well—especially when it swings around.

– – — – –

As you can see, developing interactions in VR is a detailed and challenging task, but one of the most rewarding for both developers and users. Compelling interactions in VR are usually the by-product of many different iterations until the result feels as natural and satisfying as possible.

Thanks for sticking with me. I’ve been inspired by many talented VR developers out there; I’d be honored if some of these examples can inspire other fellow developers as well.
1
2
Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Road to VR
  • Raphael

    Richie’s rich interactive plank drilling experience.

  • DanDei

    This illustrates very well how many parts of game design need to be reinvented for VR. I find it more and more off-putting to be put in very static environments, especially in Tutorials that limit my controls even more while telling me exactly what to do when and how. There is a natural desire to grab and manipulate a VR world in an exploratory way that is just different from monitor games.

    • impurekind

      I agree. . . .

      But one thing I actually find frustrating and often counter-productive in many current-gen VR games is them trying to do overly complicated interactions when the tech just isn’t responsive/precise enough to do them properly, particularly really sophisticated hand stuff, and sometimes I’d rather it were kept simply but reliable like it is in something like Job Simulator for example (which still has one of the most interactive environments in VR to date).

      Precision, reliability and fun is more important than just trying to be more simulationy of everything even when it’s often rather clunky and clumsy.

      • DanDei

        That is why I am very, very much looking forward to Boneworks. They really seem to have nailed the physics and object interactions without being clunky.

        • impurekind

          Well it doesn’t look perfect but it’s definitely some of the best realistic interaction in VR I’ve seen.

          • asshat

            yeah at first they were kinda overly zealous about their videos, like most of thier physical interactions have been done by other before. but yeah they seem to be spending a lot of time making sure they work well. if they dont after all this itll make them look bad as well

      • A really good point. I decided to not add finger usage with the Index Controllers for this very reason. It wasn’t as precise and it just led to having to explain more on how to use the feature. I stuck with a ‘pincher’ like movement that was activated by gripping the center palm shaft. I also allowed a larger sweet spot for grabbing to reduce frustration in picking up smaller rocks and tools. Not as realistic but it seemed to be just as satisfying for the novice who may have never used a VR controller. I also think the Valve Index controllers add much needed transparency to allowing for more realistic hand actions.

        I also what to give a shout out to Valve for providing their instructional demo that has been my inspiration in interactivity and setting an early standard on what VR should be. In saying that, I still feel there are way too many buttons that get in the way of experiences that aren’t always intuitive or consistent. Sadly I ran into this as well and started replacing buttons with actual hand movement or ‘laser’ pointers using just the triggers.

    • PJ

      Agree, this is exactly why Skyrim and Fallout didn’t translate well into VR for me, ok it’s great to be in those worlds, but what’s the point if as soon as you want to intersect with it, you can’t

  • Ratm

    Lol NO. Key is to stimulate the fantasy not put the user to work,this gets boring fast .Beatsaber has 1000 songs and all patterns are different.
    Job simulator was in the wow! Vr era its literally trash not a game.

    • Raphael

      I find beatsaber mindnumbingly boring. Slashing shapes is a very tedious game mechanic. I know it’s popular but that’s mainly because it’s accessible without any brain power to non-gamers.

      • Peyton Lind

        I’ve owned Beat Saber for about 7 weeks on my Quest with custom songs installed the entire time. While I am not yet there in finding it mind-numbingly boring, I am getting a bit bored of it and agree with what you said about one reason it is so popular.

        • Raphael

          It’s a good game in the same way Tetris is a good game but it’s the dietary equivalent of McDonalds. I can’t exist on a diet of McDonalds so games like that are short-burst for me. For developing fast reflexes BS is unbeatable though.

          For me a healthy diet is DCS World, Elite Dangerous, No Man’s Sky and anything else with depth and longevity.

          For exercise in VR I mainly use BoxVR (wearing 1.5kg wrist weights).

          I haven’t tried Quest yet but looks like it has a bright future.

          • Marcus

            It’s the other way round: For most people sitting is extremely unhealthy while moving (like in Beat Saber, Ratchet NX etc.) is so much more healthy. It may only be a metaphor of yours, but I don’t like it.

          • I finally got a chance to try the Quest and was pleasantly surprised. First, I did not know that they created projected black & white stereo paired image back to the viewer, so I thought I was in some office type VR experience, then I realized it was my own space. Once the familiar Oculus landing page came up after identifying my boundaries it was less confusing, (but still the thought it was cool). After that I tried “Robo Recall” to see how much it strayed from the desktop version and delightfully it seemed to be just as engaging. Yes the graphics take a hit, but not as much as some reviews seem to think. I for one found the experience quite convincing, and in fact more liberating since I didn’t have to worry about where I was, or have my position abruptly affect my view due to turning and having my headset lose tracking since I only have the two camera setup.

            The headset is comfortable but noticeably heavier than the Oculus Rift CV. Image quality is better and audio seemed about the same, but not doing a side by side comparison, I cannot say for sure.

            My only complaint has more to do with the business strategy of Facebook. By pushing curated content and (expensive) apps in their store, trying to turn it into virtual social activity and hampering connectivity with the desktop, I believe they will kill the market just like Google did with Daydream. It would have been much better headset if they built in a WiGIg wireless solution to turn it into the Vive killer. From my understanding in the research of the SLAM data, it is processed locally on a custom (Qualcomm) chip which would reduce dramatically the bandwidth needed (MS WMR) since the camera streams don’t need be streamed to the host system. If this is true, which I believe to be due to all the third market solutions using WiFI, this would have negated having to bring two systems to the market. But here I believe like many companies before it, we are seeing what happens when two camps are set up within a company pursuing different business models.

      • That’s the key Raphael – “non-gamers.” But as I found in my “Lunar Excursion VR Experience.” Creating actions that come easy to the novice is what draws them in. But it is also external cues, like wearing “moon boots,” and being assisted by those dressed in white lab coats and coveralls with appropriate patches also helps the patron/novice believe in the experience. But the “assistant” plays a vital role and ensuring the novice makes a smooth transition into the virtual world. I think this is why the Void has been so successful because your experience does not start once you put the headset on.

        Also back to Beat Saber. I think something else they did is make the experience for those not participating in VR engaged while watching the monitor. But the same could be said for a number of console type games which put a lot of effort in making the experience enjoyable to those spectating. This is something I tried to accomplish as well by recreating a black and white camera to simulate the B/W Westinghouse that beamed back images we saw on TV with of course the new “third (player) astronaut” introduced to the scene (there is a small color inset view of what the user is experiencing as well). This helps the spectator relate to what the patron/player is experiencing, since many of them blurted out “Did you see that?” to their friends.

    • Peyton Lind

      All patterns are not different. WTF are you smoking?

      I think stuff shown here looks far more interesting than what I do in Beat Saber.

      • Ratm

        So.. You beat all songs with the same repeating moves?Idd my smoke is better than yours !!

    • Żéñ Źdźbło

      Can’t believe we’re comparing beat saber to this.

  • namekuseijin

    rich interactions is key indeed in VR. but I hope that doesn’t mean 90% rich interaction with a gun in your hands and 10% rich interactions with a few puzzles to pace the longevity of the game…

    all in all, it’ll still take some time to see VR evolving to integrate better physics so rich interactions become more prevalent… more grabbing and handling things with your own hands and less magnetism… inventory management is an area that should improve much from better systems with rich interactions. this is an area in which native VR titles really shine even now compared to flat ports…

    • See my comment and couldn’t agree more. There are so many world experiences that many of us will never experience (or want to) and constantly seeing experiences that simulate killing, maiming or warfare may do more to keep VR out of the reaches of the public than anything else.

      It should be noted that it was Myst, The Sims, and SimCity, Madden Football, MS Flight Simulator that brought computer gaming to the masses. And frankly we should not forget that some of most popular apps in VR have nothing to do with holding a gun. On that note I have to say my guilty pleasure has been Arizona Sunshine but feel their inventory system could be improved, but then again when I wear my camera vest in doing a medai shoot, I always forget which pocket I put something in and always doing the “pocket dance.”

  • I love these articles about UX! I miss the ones by Leap Motion…

  • Sandeep Banerjee

    Exactly right. When I play VR, I want to do stuff that is *impossible* in flat-screen gaming.

  • LiquidKaos

    Thanks for the great article. It was really informative and presented in the perfect way to keep me truly engaged.

  • MatBrady

    VR Studios don’t know what to do with VR, and this article is case in point. Tasks are just not fun for most people. Look at the end scene in Lone Echo as a terrific example. They had a high-quality assets, a terrific locomotion system, great characters, and what was the big finale?? Menial tasks with a time limit. Are you freaking kidding me? Stormland looks like it’s going to get it right, but the kinds of things demonstrated in this article screams lack of imagination. We’re going to look back at this kind of stuff in two years time and groan.

  • Excellent article and couldn’t agree more as I have found in my own testing of “Apollo 11: ‘One Small Step For…’ VAMR Experiences,” “Lunar Excursion VR Experience.” Interaction is what drew patrons to the event on 7/20/2019. Not only did the patron use their hands, but also their chest and feet since they actually donned a (virtual) A7L lunar excursion astronaut suit through the use of three Vive Trackers. This not only allowed them to see their limbs, but also experience what it felt like to be encapsulated. This became apparent in not only in my own testing, but watching others try to look up and realize since the helmet is not connected to their head but to their shoulders, they had to lean back to look up and see the earth or Neil Armstrong climbing up the ladder at the end of their excursion/experience. I also provided a removable chest attached Hasselblad 500EL camera so the user can capture their own images that are time stamped and saved in a folder on their harddrive. Again because it is attached to their chest, they have to bend at the waist, or squat to capture images. Finally, one of the more important things I wanted to convey is how 1/6 the gravity of earth would affect objects, so many of the tools and rocks can be picked up or kicked around, which as you can imagine do not react the same way they would on earth due to the limited gravity and not atmosphere. All of these things led to a very convincing experience that brought some of the older participants to tears, since like this old timer, watched the event on TV in 1969 (I was eight).

    If you are interested in trying it out yourself, it is my goal to release it in September on Steam, Vive Portal and Microsoft Store for FREE after I further optimize the build size and add a version to those limited to an HMD & controllers, with an extra “sandbox” option that allows the user to roam the area around the LM using teleporting and “Kangaroo Hopping” mode for those who don’t mind roller coaster type movement through the measuring of the controllers vector acceleration (pushing down) to control jump force and angle.

    Finally, after its release I will be introducing “Ascent” a simulation of the Apollo 11 LM liftoff from the lunar surface and docking with the SCM (Service Control Module) that uses another technique I am working on and that matches physical controls to their virtual versions using proprioception. In this case several buttons, an analog thrust control and 8-way joystick for RCS (Reaction Control System) to allow you the pilot to lift off, reach orbit and navigate and dock with the SCM. The goal is to create a flexible kosk like system with ‘lego’ like components attached to a perforated surface to match the controls to the VR equivalent for all kinds of vehicle and control system simulations. More to come in the next couple of months.

    Below seen in action with Joshua Young, facilitator of the ‘Reality Design’ Meetup in Portland, OR. Images are from the virtual Hasselblad 500EL attached to my chest 1Kx1K with Lareau plate markings.

    https://youtu.be/uMPBtMTI55g
    https://uploads.disquscdn.com/images/e43eb8e3f32fae1731f176f7b3b534c3a8611bfa12cc87a00f2cb751c87d9e38.gif
    https://uploads.disquscdn.com/images/59d7251a5e7ec0ee6e76629e1ecf5e16e58e8e6e849e1050c897ab2942ec8699.jpg
    https://uploads.disquscdn.com/images/6d80f3a24361b5a11473392cfec8dc3728224900d277b7fb4d99a4d757e93d3c.png https://uploads.disquscdn.com/images/b25d5efa83664b987ab6be88df39d4e7c576a87d372027caf08a3da26e5ffc32.png https://uploads.disquscdn.com/images/6e62ff80ca4c5b757250cc2a73267d3d21ca2534587c87333c485e75aa5cb874.png
    https://uploads.disquscdn.com/images/c6e9926ca7a9ef1cfa85f3056cbcc911a5ba39906b53d60b7c73d9135105c075.png