Is This the Future of Augmented Reality Gaming?

One of my favorite YouTube directors, Freddie Wong happened to make a video that is quite relevant to this site. The video is called ‘Future Motion Control Gaming’ and shows a fictional augmented reality gaming setup. But is this really the future of augmented reality and motion control gaming? Will it be possible to achieve what’s been shown off on the video? Step inside to see the video and find the answer to that question.

Without further ado, here’s the video:

[youtube http://www.youtube.com/watch?v=Z-FRb8CZ__I&w=600&h=335]

Definitely a cool video, but the question we’re here to answer is whether or not this is something that could really be done or if it will forever be in the realm of science fiction.

Through the research I’ve done for this blog, I think I can safely say that this video actually represents a great concept video for augmented reality gaming and doesn’t show anything that we won’t be able to achieve in the next 10 or so years. Everything that happens in this video is technically feasible and based on technology that exists and is being developed today.

Let’s start by looking at the technology that would be used to accomplish something like this:

• Motion control peripheral
• Head mounted display or pico projector
• Camera for environmental mapping
• Head tracking hardware (optional, actually)
• Compact PC for processing (or this could be done remotely with a video feed sent wirelessly
• Backpack battery rig

In the video, the motion control peripheral used actually already exists. This is a PlayStation Move, a motion gaming accessory created by Sony. They’ve inserted the move into a third-party add-on to create a more gun-like controller. In its real-world use with the PlayStation 3, the Move requires an external camera to get a fix on exactly where the accessory is in space. However, the implementation seen in the video shouldn’t need an external camera because the particular scenario shown involves the player always wielding the gun in their hands, directly in front of them. Relying on internal sensors (accelerometer, gyroscope, and magnetometer [electronic compass]) would be sufficient to calculate the trajectory of the virtual bullets being fired from the gun relative to the environment. An external camera, or some other solution for locating the end of the gun in real space would only be necessary if the player were to change elevation.

The video also utilizes a pico projector which functions as a display device as well as an environmental scanner. I think more realistically, such augmented reality gaming will make use of a head mounted display rather than a projector. With a projector, the distortion of the objects projected on extreme planes (like on the ground in from of you) would be very hard to compensate for in real time and with multiple on-screen elements. It could be done, but a head mounted display should be easier because it can simply find the angle of the ground plane, then alter the scale and orientation of the on-screen elements within the display itself, rather than having to greatly distort a projected image in order for it to look correct to the viewer. Both pico projectors and head mounted displays exist today and are readily available. The ST1080 HMD from Silicon Micro Display already fits the bill quite well, providing 3D, 1080p resolution, and 10% transparent displays (though the transparency might need to increase to achieve the results seen in the video), and we’re sure to see more devices like the ST1080 which will enhance the experience further.

Before the gaming starts, you can see that the pico projector scans the environment to determine where to place enemies and how they will be able to move around. The ‘scanning’ function was presented unrealistically. I think they wanted to make it look like the pico projector was shooting out beams of light to scan the environment, but more realistically, a camera would be used for environmental mapping. With ever increasing processing power and higher quality compact cameras, it’s become possible to accurately track a complex environment without any sensors other than a camera. Mounting a camera on the head mounted display would make the environmental scanning/mapping, which is required for enemy placement, possible. In reality, the environmental mapping process would take more time than shown in the video; the user would walk around looking at the entire play space to be sure that the computer had a complete map of the environment. With an augmented reality head mounted display, the virtual environment would be projected over top of the real environment to show the user which areas have been mapped, and which places still need mapping.

As I mentioned, head tracking hardware might actually become optional as long as a camera is used in the setup. It has been demonstrated that extremely accurate environmental mapping and tracking is possible with a camera alone; this eliminates the need for head tracking hardware, as long as the camera is mounted to the head of the player. Take a look at this video to see just how accurate real-time video-only environmental mapping and tracking can be:

[youtube http://www.youtube.com/watch?v=Y9HMn6bd-v8&w=600&h=437]

If you wanted such a system to be fully mobile, you’d need a compact PC and a backpack battery rig. Both of these things already exist. In fact, I covered a really neat wearable computer setup, powered by a UMPC and equipped with some external battery packs, several years ago. One quick look at it and you’ll see that it bears surprisngly similarity to the setup used in the video. You can see details on that here: http://www.umpcportal.com/2009/07/awesome-wearable-computer-setup-is-powered-by-sony-vaio-ux-umpc/

So yes, I’d say the video shows something that could be considered a non-fiction concept of future of augmented reality gaming. Someone just needs to bring all of the pieces together. It probably won’t really take of until the big 3 console companies, (Microsoft, Sony, and Nintendo) decide to give augmented reality gaming a try. I, for one, am quite excited. I just hope it happens sooner rather than later!