What Would a Real Head Mounted Display That Uses This System Look Like?
They say a picture is worth a thousand words, so I decided to add a few 3D renders to better show how the idea could be applied to real-life HMDs. Please bear in mind that they are not intended to be very accurate, mechanically correct or even have the right proportions.
The HMD would have to be separated in 2 different sections, and these could be connected by a telescopic tube for instance, used to maintain rigidness between both parts and to measure the distance between them (in order to feed the tracking algorithm with the exact 3d model). This same information could be useful for other purposes like feeding binaural audio processing with the approximate size of the head.
Original idea for a full 360 degree positional tracking HMD, on which the cardboard prototype of the pictures and videos is based:
Front and back don’t really need to be equal, they simply need to have the markers on fixed positions. I think if this system was applied to a commercial HMD the back part could be used for connectors or to store a wireless receiver and a battery for instance, or simply to counter the weight of the front part and even the center of gravity of the head mount. Maybe following a similar design to some of the 90’s Virtuality systems. Also if having a separate back part of the HMD was too much of a problem, the idea could still be used with only the front part, limiting the movement to about 100 degrees in each side (200 total) which would still probably be usable in VR if you keep this limitation in mind while playing.
Additional Thoughts
Even though my main priority with this project was seated play, I believe that in combination with an HMD, wireless connection, and with some improvements on distance detection, you could have complete freedom of movement limited to a small area (the FOV of the PlayStation Move camera at a maximum of 3 meters distance approximately). As it is now I think it’s good only for seated play; standing/rotating in place could work but you would have to mount the camera on some sort of stand (at head level) and you wouldn’t have much margin to move around.
As mentioned earlier, it’s clear that for a consumer device it would be ideal not to have an external tracking device, but I believe having only one camera is acceptable, as most people are used to devices like the PlayStation Move or Kinect. In fact it could even be seen as a ‘safety feature’; when you walk out of the FOV of the camera the system could pause and warn you. Finally I would like to point out that sometimes we geeks tend to forget that looks is one of the most important aspects for a consumer device. I know that my little prototype doesn’t look awesome (and looks are very subjective), but I think it has the potential to look pretty cool. Personally I think it has a Cyclops meets Iron Man kind of vibe!
General Limitations:
- The main “problem” of this solution is that the case design of the HMD needs to be adapted to it. I want to make clear that this whole idea is not very practical if you want to add positional tracking to an existing HMD (although everything is possible). It is intended to be part of the shell design for a new HMD built with optical tracking in mind (either DIY or commercial).
- Accuracy degrades with distance, still quite good from 1.5 meters. Could be improved.
- If someone walks between you and the camera while playing you can lose positioning. Again maybe this could actually be turned into a good “feature”: If someone occludes the camera for more than 3 seconds it could be a way to communicate they need your attention. It would be good to have a method to do this as someone already suggested before:
Rift Developer Kit Add-on
Here’s a possible idea for a Rift developer kit add-on, or as I like to call it “Rift-Hugger” (inspired by the face-huggers from Alien!). The back part could be used to store standard batteries that would power the LEDs. My guess is that the front plate of the developer kit can be easily removed with the clips on the side. If not the add-on could be simply attached on top of it. The moving parts on the side that hold the Rift could have some sort of locking mechanism as well.
As you can see, as a separate add-on it wouldn’t be ideal, it would have to be targeted to a specific HMD model and it wouldn’t be easy to DIY, but it could still be done (and very cheaply in fact if mass-produced in a factory).
Possible Future Improvements:
- Build another improved prototype: modify LED distribution, separate circuits by colors, use potentiometers instead of resistors, etc.
- Leave orientation to nrp’s adjacent reality tracker IMU and use optical only for positioning; unfortunately it’s not available yet. Reality tracker should be much more accurate and precise for orientation, play with fusion algorithms.
- Adapt prototype to developer Rift size and shape. As i mentioned before the ideal path would be to design the casing with the optical tracking in mind not the other way around, but after seeing the prototype pictures i think we could do something with it. As it is (without adding a rear part) it would not be possible to have the full 360 deg but i reckon i could get about 240 deg lateral tracking (120 rotation to each side), something still very usable in many scenarios, and a nice step up from TrackIR for instance.
- Increase capture distance by software (adjust color thresholds depending on distance between LEDs).
- Improve blob recognition to allow lower resolution input (120 fps mode).
- Use a cheap graphics card (CUDA) or a FPGA to speed up the image processing?
- Track 2 headsets at the same time 1 with camera?
- Extend system for hand tracking?
- Add FreeTrack support / create a plugin for FreePie / create a new VRPN driver?
I still have to look into these options and decide which one should I try first. It would be really cool if I could test the prototype with an existing game that supports 6 DOF, like Arma 2 for instance.
About the PosiTTron Name:
I have to say that I’m not really planning to patent it or anything, I put a name just for fun. I think it suits the project quite well, and sounded better than the boring alternative thread title “HMD optical marker integration”.
The POSIT part is quite obvious, as it’s a positional tracking system, and I also happen to use the POSIT algorithm to solve the pose estimation problem. On the Tron side, well Tron is like the original movie about VR (although not very realistic) and they also happen to use helmets with bright LEDs surrounding them — not too different to my design really! Oh, and I quite like the English band Ladytron!
To Finish Off (Sales Pitch Stuff)
I know the proposed solution is not the holy grail of tracking, but I believe it’s a realistic and ready-to-go solution that solves some real problems. Let me finish with a list of benefits (box ad style); they are NOT all true with the current prototype version, but could be on a more finalized design:
- Precise (sub 1mm) positional tracking (6 DOF) with 360 degree horizontal and 90 degree vertical freedom of movement. No magnetic interference, no self-occlusion, no drift.
- Working range from 50cm (in front of a computer) to 3 meters (in front of a TV set/play area). Seated or standing play. Wireless!
- 120 FPS ready, low latency, fast processing (not computationally expensive).
- Uses open-source software and of-the-shelf available hardware (cheap components).
- Only one external camera needed which can be placed anywhere in front of the player, no need for calibration.
- It can look pretty cool, I believe!
Oh and of course in marketing terms, if this was to be added on top of a ‘9 DOF’ tracker like the adjacent reality, I guess it could be called a 15 DOF system:
3 DOF accelerometer + 3 DOF gyroscope + 3 DOF magnetometer + 3 DOF optical orientation + 3 DOF optical positioning = 15 DOF!
I have to say that it has been quite a challenge to find the time to implement all this; I had the idea burning in my head since the end of July more or less, but couldn’t start working on it until September Since then, with a full-time job as web developer, I have been able to spend only a few hours a week on this project.
Anyway it seems like Oculus are looking into using PTAM for localization/positioning, which i think is the right move in the long term; I really hope they can manage a properly robust and low-latency implementation. Still I think something closer to my design could be a simpler and safe alternative / plan B, in case the PTAM research doesn’t pan out in time for the Oculus Rift consumer version. I also think this design fits quite well with Palmer’s approach of “doing more with less.”
Personally I believe this solution would be ideal for the developer version that we will get soon, as it would let developers start programming and experimenting with positional tracking. Also it would help the Oculus team design the SDK with positioning as a main feature, and to receive feedback from the community. Unfortunately it’s probably a bit too late for this to happen on the Kickstarter revision… who knows, maybe there’s still time to add a few holes on the Rift shell as part of the fabrication process!
In any case I think this could still be useful to developers and DIYers as an alternative to strapping a PlayStation Move, Razer Hydra, or a mobile phone on their heads. I’ll be more than happy to help if anyone wants to give it a go. Also of course any good ideas are more than welcome, either from the hardware or software side. I’m sure this particular design and tracking software could be greatly improved but I think my little demo is enough to prove that this is an avenue worth exploring.
Ok, finished (for now), and sorry for the long ramblings!
Acknowledgedments:
I would like to thank some members of the MTBS3D forum like Brantlew and Chriky who kindly answered some of my newbie questions when I got started, and specially Edz who gave some good advice when we were discussing all this back in July/August. Last but not least I would like to thank my lovely girlfriend for her craftsmanship input while building the physical prototype and for her constant support in all my crazy endeavors!
Discussion of Batallé’s ‘PosiTTron’ DIY positional tracking system is taking place over at the MTBS3D forums.
