Multimodal connectoR

Multimodal connectoR

Ryu Nakagawa, Ryo Komatsubara, Art and Media Experience Lab

Concept/Creative Direction: Ryu Nakagawa
Programming/System Direction: Ryo Komatsubara
Visual Design: Taku Ota
Exhibition Facilitator: Sayuri Ohashi, Ken Sonobe


“Multimodal connectoR” offers a new form of multi-participant audiovisual art experience. Participants can experience the environment with Microsoft HoloLens units. In this experience, participants can place a Note object that produces sound of the audiovisual effect by using the Air tap gesture with their index finger and thumb. When the Note objects collide the belt-like Trigger line object, audiovisual effects are generated. Then, the pitch of the sound is related to the height in real space. Participants’ intentional movement, unintentional movement, and relationships among them or their interaction in the environment creates a diverse audiovisual experience.

This art work was upgraded, and exhibited at METACITY, by Nangok R/Studios, a company specializing in MR content development.





METACITY, Chiba, Japan, Jan, 2019 [ > ]

Exhibited by Katsutoshi Hata (Nangok R/Studios) + Nangok R/Studios (Nangok R/Studios) [ > ]


Published Paper

  • Multimodal ConnectoR : A Multiplayer Audiovisual System using Mixed-Reality, Ryo Komatsubara, Ryu Nakagawa, NICOGRAPH 2018, 158 – 159, 11 , Refereed  [ > ]

Supported by

The work was partially supported by JSPS KAKENHI Grant Number 18K00210.