esc button
search icon

Play Music

Learning music and other motor skills using VR

Shay Almog

Mentor — Ronel Mor
Design and Technology track

Learning to play music is a challenge not only for beginners. Even an experienced musician has to invest much time and effort in order to imprint their hands, muscles and brain with a sequence of movements. 

Musicians are practicing and repeating that sequence until it is embedded in their body memory.

Shay Almog’s project seeks new and intuitive ways to learn music, focused on the motor aspect. Shay is inspired by video games such as Guitar Hero and Beat Saber, wherein the player is required to perform predefined movement sequences, in sync with a song playing in the background and with the help of visual elements guiding the movement. The project explores the use of such visual aids as an alternative for musical notes. Shay created an application that connects the physical world to the virtual world - the user sits in front of a piano, puts on a VR headset and sees a virtual piano overlayed precisely over the physical one, and virtual hands. Visual aids are then displayed, guiding the fingers. Later, the piano was replaced by a percussion instrument and the ten fingers were replaced by two mallets, after realizing that the piano is too complicated of an instrument for this interaction. Long, complex pieces were replaced with short, simple loops. In this later version the visual aids were replaced by virtual mallets. The mallets move around in space and if the users follow them they succeed in playing along. The name of the project arose from the double meaning of “play” – play music and play a game – and from the insight that one of the most effective forms of learning is playing.

No items found.

Shay Almog

Shay Almog was born in Jerusalem in 1987 and currently lives in Tel- Aviv. Shay holds a bachelor's degree in mechanical engineering from the Technion. Following his studies at Bezalel, Shay began developing virtual reality applications. Shay is a musician, guitarist, pianist and drummer. In addition to music, Shay is interested in math and science and creates software-based content in these areas.

More projects