Back in December of last year, after seeing few compelling consumer augmented reality applications, I wrote a short article about one that I would actually use, an AR piano trainer. I’m thrilled today to see that something very similar has popped up, and can’t wait to get my hands on it!
Though the system in question, falls closer to ‘mixed reality’ than augmented reality, it’s easy to see how it could be extended to the latter — just add water! the right pair of AR glasses.
According to New Scientist, the system, called Projected Instrument Augmentation system (abbreviated: PIANO, somehow…), was developed by pianists Katja Rogers, Amrei Röhlig and others at the University of Ulm in Germany. The system projects visual notes onto the piano top. The notes flow down the projection space and reach the edge as they are intended to be pressed. The length of the note indicates the length it should be held, while the color indicates which finger to use. Anyone who has played Rock Band of similar rhythm games should be familiar with the concept.
One thing that I really like about this solution is that it doesn’t require the ability to read sheet music (which has always felt to me to be very outdated). This means anyone can jump in right away and start learning basic piano phrases, chords, and hand movements. As someone who has always been very visual, I’d absolutely love to learn to play piano with this system.
Three modes allow players to quickly learn a given piece. Listen mode is to simply watch and hear the piece played. Practice mode pauses on each key until the user presses it, allowing them to play at their own pace. Practice mode doesn’t slow down the keys and provides visual feedback of note accuracy after the song is complete.
According to New Scientist, formal research is underway to test the effectiveness of the system.
“We had quite a few novices use it who were very sceptical at first, but then were really impressed by how quickly they could play relatively well,” Florian Schaub, who presented the system last month at the UbiComp conference in Zurich, Switzerland, told New Scientist.
This seems like an opportune time to share this beautiful audio/visual representation of a piano piece (one of my favorites by YouTube user musanim).