Musician's Mirror
The Musician’s Mirror is a tool to help musicians correct bad habits in their technique, posture and form. A computer vision based system, it allows the musician to track any part of their body and be given feedback when they fall into bad habits. The musician or their teacher can apply labels for good and bad positions, e.g. hunched shoulders or a slouched back would be labelled as positions to avoid.
The system provides real-time “thinking fast” audio feedback whenever the musician falls into a bad habit. For acoustic instruments this is white noise, while for electric instruments the system can distort the sound of the instrument itself. Non real-time “thinking slow” feedback is provided in the form of a visual heatmap that summarises hours of practising, showing the positions the musician spent the most time in.
A whole practise session summed up in a single image
While applicable to other activities (yoga, sports, desk-work), the system was tailored for musicians: using cameras to be as non-invasive as possible, using markerless body tracking so that musicians do not have to wear anything that would interfere with their playing, and providing feedback through the auditory channel; the only thing that musicians hate more than being in pain is sounding bad.
Fixing these musculoskeletal habits is extremely important for dedicated musicians, a recent survey found that 80% of music academy students have complaints of arm, neck and/or shoulder problems due to playing (reference).
Functional Prototype
A functional prototype was developed using a commercial depth camera and OpenCV, which was tested and developed in collaboration with Professor Peter Buckoke of the Royal College of Music (Alexander Technique and Double Bass) and guitar students from the college.
Testing at the Royal College of Music
Product Mockup
As a consumer product, a camera small enough to fit into a Violin case would work with your smartphone or tablet, and be able to clip onto a music stand, or stand on a flat surface.
Project Status
The project is currently under testing and development, if you would like to know more then get in touch arthur@arthurcarabott.com