Motivation
What if your beloved musical instruments are with you all the time? What if you could pick up and try new ones right away? What if you could entertain your friends just with your phone? What if...
These are just a few, out of many possibilities that could be achieved using the power of Augmented Reality.
In this small project, I've prototyped an AR Instrument experience, using the Spark AR Studio.
These are just a few, out of many possibilities that could be achieved using the power of Augmented Reality.
In this small project, I've prototyped an AR Instrument experience, using the Spark AR Studio.
Design Process
At first, I wanted to test two different approaches for having the instruments rendered, one in the field of view and relative to the head position and the other one, in the camera space and attached to the device. Although having it attached to the device was much easier/ quicker to achieve, yet the first option provides a better sense of presence and immersion in my findings. I've also decided to not waste any time on 3D modelling before testing the functionality, therefore I picked a harmonica from the Spark AR library ( Sketchfab) And built the first run with it.
I kept experimenting with the different ways to get a smooth head tracking movement. The first iteration was successful in terms of having the harmonica following my head. The second and third iterations were focused on improving the tracking, fine-tuning the optimal head movement range and turning that into an acceptable scale of notes.
As there's no haptic feedback to communicate which hole you are blowing into, I needed a visual feedback system in place.
As there's no haptic feedback to communicate which hole you are blowing into, I needed a visual feedback system in place.
At first, I thought "head rotation" would be the best data to track and use for this purpose. Although after trying it out and also asking others to play with the early prototype, I noticed that it could easily lead to neck tiredness and create an uncomfortable experience, then I changed that to head movement instead.
For the final iteration, I got back to my initial idea of creating an AR pan-flute. I used Gravity Sketch To model the flute in VR, optimised the model in Blender, applied texture using Adobe Dimension and finally built it in Spark Ar Studio.
For the final iteration, I got back to my initial idea of creating an AR pan-flute. I used Gravity Sketch To model the flute in VR, optimised the model in Blender, applied texture using Adobe Dimension and finally built it in Spark Ar Studio.
The next steps would be adding an adjustable metronome, the possibility of blending notes, changing the keys, and providing learning capabilities, such as following specific note orders.
You can find a quick video of the entire process below.
