Vive Tracker Used to Create Pressure Sensitive Photoshop Stylus for VR

Developer Jimmy Gray recently posted a short video (see above), showing the latest version of a DIY motion-tracked VR stylus prototype. Using an HTC Vive Tracker mounted above a stylus-shaped unit, a VR stylus can be tracked and rendered, usable in Photoshop and capable of pressure sensitivity.

Update (7/8/17): Gray has released a new video showing a more polished version of his prototype which he now calls the Versa. The video shows how the device can be held like a traditional controller (with the Tracker facing upward), utilizing a thumbpad for typical interactions.

The Versa is an all-in-one VR stylus and controller powered by the @htcvive. Here’s a prototype being used with @tiltbrush! pic.twitter.com/ZCKFGVAA9R

— Jimmy Gray (@jimmygray_) July 7, 2017

When flipped upside down with the tip pointing outward the Versa is in Stylus Mode, which gives the user a pencil-grip for better control with drawing tasks like sketching in Tilt Brush. Gray calls it Tablet Mode when the device is used against a flat surface, allowing for precise control for 2D interactions.

This latest Versa prototype appears to be fully wireless now compared to the prior version which appeared tethered to a computer. Though bulky today, it’s an interesting concept for a different type of ‘tool’ that might become a staple of VR input in the future. It isn’t clear how far Gray plans to take the project, which seems like it could be on a trajectory to a crowdfunding campaign; we’ve reached out to learn more about Versa’s future.

Original Article (4/12/17): According to Gray’s tweets, the prototype emulates an HID device using ‘Teensy‘, a very compact USB development board. For now, this part of the device appears to connect directly to the PC via a USB cable, but it could potentially be made wireless, like the Vive Tracker itself, in future iterations.

It’s easy to see where the motivation to start this project came from, as Gray is also an artist, with a few examples of his 3D sketch work in Tilt Brush on his YouTube channel. Like many of these motion-tracked accessory projects, the stylus started life using a Vive motion controller to perform the tracking, and this earlier footage shows the flex of a stylus tip being connected to an input on the Vive controller’s trackpad, acting as the pressure sensitivity gauge.

Tracking a stylus in this way is yet another example of the precision of Valve’s Lighthouse technology, something that was highlighted by Valve themselves in a similar way with the marker board in The Lab. However, the system has its limits, and is not going to be as accurate as using a conventional stylus, but there may be applications where a VR stylus is favourable, especially thanks to the real feedback it can provide.