FOVE, the Japan based VR headset creator, is using their eye-tracking technology in a new project: ‘Eye Play the Piano‘, a collaborative partnership with the University of Tsukuba’s Special Needs Education School which aims to give physically-disabled students the gift of music creation.
The integration of low latency eye-tracking into head-mounted displays is often hailed as the next great step in improving VR immersion. Ideally the technology could help maintain high performance application with the use of foveated rendering, a technique that would allow a computer’s GPU to focus its graphical power only where the fovea is focused, the region of the eye that picks up high detail. But that isn’t all it can be used for.
Eye-tracking has served the physically-disabled community since the home computer revolution of the 1980s, but with peripherals like the Intel owned Tobii EyeX coming down in price and size, the technology has found its way into a very different use case: ‘Eye Play the Piano’ a crowdfunded project conceived by FOVE and the University of Tsukuba’s Special Needs Education School that plugs the company’s VR headset into the control unit of an automatic piano.
FOVE approached seventeen-year-old Kota Numajiri, a student at the University of Tsukuba’s Special Needs Education School for the Physically Challenged, to help demonstrate the system that upon completion uses a specially developed GUI in part realized from the Kota’s personal feedback.
Selecting single notes or chords on the piano requires only a pointed gaze and a blink of the eye to activate, while titling the head down signals the software to hold the note, just like a piano’s foot pedal—making for a truly hands free operation. After four months of development and practice with the system, Kota was ready to present his new found skills at the school’s Christmas concert, an annual affair presented by his fellow classmates.
The University supported initiative plans to “donate the ‘Eye Play the Piano’ device nationwide to 135 schools for the physically-disabled (334 schools if calculated by departments)” with donations made through the charity crowdfunding website JustGiving.
Donate to ‘Eye Play the Piano’ (Click English Option)
Backer Rewards
- 1,000yen ($8.40) – Thanks from the project team will be sent by email with a limited release movie of the making.
- 3,000yen ($25) – Supporter names will be featured on project homepage.
- 5,000yen ($42) – Piano audio composed from the Eye Play the Piano system will be sent.
- 10,000yen ($84) – Invitation to a networking event with project members scheduled to be held in Tokyo. You can hear the project review and experience operational use of the system.
- 50,000yen ($420) – Invitation to a networking event with project members scheduled to be held in Tokyo. You can hear the project review and experience operational use of the system. Afterwards you can participate in the social gathering of the development members.
- 100,000 yen ($840) – FOVE Prototype – After achieving this project, you will receive a FOVE prototype that are used in ‘Eye Play the Piano’. This refers only to the head-mounted display – electronic piano is not included.
The Road to VR team got to check out FOVE’s latest VR headset prototype at CES 2015 last week, details coming soon.