A new fun and intuitive way to learn the piano using Augmented Reality
Magic Keys is an Augmented Reality app for the Magic Leap One AR Headset that is supposed to teach you simple piano pieces and techniques within just minutes in a playful manner. Similarly to Guitar Hero, the notes of any piece you would like to play approach to the according keys in real time, making it possible to play along pieces without any prior practice or experience.
I started working on this project about two years ago, intially intended to be a small AR app running on a tablet. After it turned out that it could be of actual use, I ported the application to the HoloLens alongside with some major improvements. I continued to add features and tried to enhance the visualization of the music but I was greatly limited by the small field-of-view of the HoloLens.
In early 2019, I finally got access to a brand-new Magic Leap One, which was a huge step forward and enabled me to add some cool new features, including in-app feedback on note hits or misses (using MIDI input and output). I continue to work on this project since the ultimate goal is to develop a fully-featured Virtual Piano Teacher.