New recruit Jonathan Tustain talks to Jeri Ellsworth and Rick Johnson about castAR, Valve, Oculus and the Kickstarter roller coaster.
ChatAR
Despite a lack of faith from former employers Valve, Jeri Ellsworth and Rick Johnson recently celebrated raising over a million dollars on Kickstarter to commercialise an augmented reality product called castAR.
The first product to be released by their newly formed company Technical Illusions, castAR is a high resolution, interactive, 3D augmented reality gaming system. Two projectors above each lens project images to retro-reflective surfaces in front of the user and integrated 120Hz real-time tracking means the angle of view is rendered according to where they are positioned and orientated. An additional interactive wand then allows people to interact with the holographic style visuals.
With multiple stretch goals achieved and even Oculus Rift creator Palmer Luckey himself pledging the top $10,000, castAR has attracted a lot of interest, not just from the VR community but also the Kickstarter crowd as the fundraising efforts were regarded as a blueprint to launching a successful campaign.
Now the Kickstarter dust has settled, we caught up with Rick and Jeri to find out what is next on the castAR roadmap.
Road to VR: Why do you think Valve lost faith in the project and do you think they have missed out on a great opportunity?
Rick: It’s a very complex story with Valve. We always had a difficult time even showing the product in the company. I estimate probably less than 5% of the employees ever saw what we were working on. Initially, when we were just taking the AR approach, if you think about the type of games Valve produces, it is not really in line with that. We had a hard time attracting Valve employees to the project. Employees are financially rewarded by being visible on certain projects so as a ‘potentially obscure’ hardware project it was not going to be as rewarded as other projects.
Road to VR: Valve have announced they will reveal their own virtual reality system next month. Did that surprise you?
Rick: No, we were not surprised. They have been researching this for a while.
Road to VR: What has the Kickstarter campaign been like for you emotionally?
Rick: Ever since Valve, it has been a roller coaster of every possible emotion. You get kicked out of the Garden of Eden and you are suddenly working at home putting in 12-14 hour days, 5 or 6 days a week; it’s a long, emotional and stressful effort.
It was probably the single most rewarding moment of my life…
When we were at Maker Faire in California, we had about 12 people come down who helped us run the booth and associated efforts. The evening of the first night, we all went out for dinner, I got up to do a thank you speech to everyone who came down to assist us. It was probably the single most rewarding moment of my life; feeling the connection between everybody there.
For Kickstarter, you get to the first day where you press the launch button and you are waiting for the first pledge. You really don’t know what to expect. You try to put your expectations low so you look at the first four days at perhaps getting 35% and hopefully that will make a good start towards success. We got funding within the initial 53 hours. It’s just been incredible and we are grateful for everybody who believed in the product. Hopefully all of the hard work we have put into it will show first hand and we will deliver something that will truly amaze people.
Road to VR: Do you now feel a great sense of pressure to deliver the goods?
The completion of Kickstarter is just another transition point towards different challenges.
Rick: The completion of Kickstarter is just another transition point towards different challenges. We now have to grow the company, seek out additional investment opportunities, hiring, and the work needed to do to produce the hardware.
Road to VR: How does the virtual reality add-on work?
Rick: We had originally conceived of this idea while we were still at Valve. During the summer we began work on the AR/VR Clip-on. We decided to keep this part of our technology secret as to not alert the competition that we were going to enter their space.
What we are doing is a bit different than other head mounted displays, in that we take the image our micro-projector emits and run it through a series of expansions and other optical paths. The end result is you have a squarer eye box and less distortion around the sides.
In addition, our projectors have a fill factor of 93%, which eliminates most of the screen door effect that you might get from LCD panels used in other devices.
Jeri: The shutters are not used to expand the image, but manage the reflected light. The retro-reflective surface does a very good job of returning the image to the correct eye, but there is still about 10% of the light that returns to the incorrect eye. We use the shutters to eliminate this in projected mode and disable the shutters entirely for use with the Clip-Ons. We drive the shutters with 50% open time which is very different than current 3D TV glasses which are open 2-5% of the time, so there are no flicker issues seen with other systems.
We tried several polarizer schemes and comb filters to control the light, but the best user experience was with the more expensive shutters.
Road to VR: Why not just stick to AR when the VR market is set to become competitive with dedicated VR devices?
Rick: We knew we could do VR and we knew we could do it very well and be a strong competitor in the market. Part of our strategy was that we had always kept the consumer costs in mind. By doing the AR/VR Clip-on, we could be the only high quality device in that price range which could do both AR and VR experiences.
Road to VR: What field of views are you working with for AR and VR and can you expand them?
Rick: The prototype we have shown to the public for projected AR is 45 degrees. The next prototypes we have in the office bring that out to 65 degrees. The AR/VR Clip-ons are giving us around 90 degrees for VR. It all comes down to a series of choices as to how you want to take the pixels and resolution and expand it. We could expand the FoV out further, but it all comes down to choices on where we want to have those pixels spread across the FoV.
Road to VR: Is it an option to mount the two projectors in the same place and to use one for left eye and one for right eye to do a stereoscopic version?
Rick: With our two projector system, each projector is running at 120Hz. Having the system run this fast means a smoother, more stable experience for the user.
We considered using a single projector to cycle between generating the image for the left eye and then right. While this would reduce the costs, it would also mean the refresh rate of the system would be at 60hz—not be ideal for the gamer or professional use case.
Road to VR: Will the META Spaceglasses be a direct competitor to castAR?
Rick: We met the Spaceglasses / Meta people at the Homebrew Computer Club reunion a few weeks ago. We had some good conversations and they are nice guys. While we both are doing things in AR space, they are developing a different set of technologies than what we are. Competition is good for all of us and will help bring a broad spectrum of technologies to the consumer quicker.
Road to VR: How are you going to keep the price low?
Rick: One of Jeri’s strengths is that she has a significant background in manufacturing for the toy industry. One aspect of her experience is looking at technologies which were previously expensive, but now through advances in manufacturing practices and consumer adoption, are low cost.
Many of the components that are instrumental to our design of the glasses, the micro projectors and the tracking cameras, were all originally developed for the cell phone industry. The micro projector was originally conceived of to use with your cell phone to show presentations for a business meeting. The lumens though were never really bright enough for that environment. With our use of this technology in combination with the retro-reflective material, they are now actually too bright and we have to bring that down a bit. The tracking camera is actually a cell phone camera. Since most cell phones come with one or two cameras now, this has significantly driven down the cost.
Road to VR: How have you been experimenting with communications on mobile?
Rick: Our initial experiments were using Bluetooth communication. We found this to be too latent and not deterministic. We are now starting to experiment using the audio jack in a similar means to an old fashion modem. Our custom developed hardware on the glasses does the entire image processing for the tracking system. The end result is that we are extremely efficient at the data which is passed to the computer for tracking. Currently we are around 72 bytes per frame. We can explore non-traditional data transfer mechanism, such as the modem concept, which can fit our bandwidth needs.
We believe having the experience on a mobile device will help bring people together easier for social experiences. It also allows for greater flexibility in doing group collaboration for the business aspects.
We intend on having Android available on the initial day of launch. We would like to have iOS available as well, but haven’t fully committed to that yet.
Road to VR: Could your solution become an alternative to CAVE setups?
Rick: We’ve been talking to some companies about CAVEs in general. The current design of CAVEs use around six projectors and are generally designed for a one person perspective experience. When you have multiple people in a CAVE, everyone sees the perspective of the one person. With our technology, you can make the CAVE out of the retro-reflective material. This allows multiple people to wear the glasses, each projecting their own unique stereoscopic view of the environment.
Road to VR: What is the range of castAR?
Rick: The retro-reflective material is very efficient at returning the light back to you. We have put the material at the end of long hallways and you still get a bright image. We expect this to work well in large rooms. The tracking system also has a pretty far range, though the final range is determined by some manufacturing choices we will make.
Road to VR: What will the integration of a gyroscope mean for CastAR?
Rick: By adding a gyro, this will allow us to have an improved tracking experience that will complement our existing six degrees of freedom system. For example, while our tracking solution requires IR markers in the world, if you should happen to rotate your head in a position where the tracking camera can’t see any markers, then the gyro can take over.
Road to VR: Will VR experiences created for castAR only work with castAR? Will there be any compatibility with other VR systems?
Rick: Having proprietary APIs between the different vendors will only segregate and hurt the consumers. There is an effort to help solve this, but I’m not able to go into details at this time.
Road to VR: Aside from entertainment, what other industries to you expect CastAR to be used in?
Rick: There are a lot of uses, such as data visualization, flight simulators for the aviation industry, displaying geological survey results, etc. castAR—We are at the start of the revolution to see the world in new ways with AR.
We look forward to getting our hands CastAR when it ships late next year. Until then, if you missed the Kickstarter but still want to support castAR, you’ll soon be able to grab glasses and peripherals on Technical Illusions’s website—check back soon.
