Air Maestros

Air Maestros

Ryu Nakagawa, Ryo Komatsubara, Taku Ota, Hidefumi Ohmura

Concept/Creative Direction: Ryu Nakagawa
Programming/System Direction: Ryo Komatsubara
Visual Design: Taku Ota
Sound Design: Hidefumi Ohmura


Ordinary music sequencers, such as Pro Tools, use timeline- based sound composition systems in which a user places note objects on the timeline and clicks the play button to hear the music. As the song cursor moves with the music, it overlaps the note objects. In this case, we expand this method into 3D space and into a multi-user system using HoloLens. As a result, users can freely place and move 3D note objects anywhere and shoot a glowing ball at them from any location with a gesture. These shots then act as an alternative to a song cursor. Moreover, when the shot hits a note object, audiovisual effects appear at the objects’ spatial position. The pitch of the sound is related to the object’s height in real space. It also plays a high-pitched sound in the high position and a low-pitched sound in the low position. This demonstration includes five types of 3D note objects with different timbres. When placing a 3D note object, users can change the object’s type by voice input. Finally, we used Unity, MRTK, and some visual effects for this demonstration.



Published Paper

  • Air Maestros: A Multi-User Audiovisual Experience Using MR, Ryu Nakagawa, Ryo Komatsubara, Taku Ota, Hidefumi Ohmura, SUI ’18 Proceedings of the Symposium on Spatial User Interaction, 168 – 168, 10 , Refereed [ > ]

Supported by

The work was partially supported by JSPS KAKENHI Grant Number 18K00210.