Drawing Sound in MR (DSMR) Project
This work is a multi-participant piece experienced with internet-connected MR devices and haptic devices. In the real-space “here and now”, participants can interact with remote users’ hand avatars to: 1) create note objects by touching fingertips. 2) trigger audio-visual and haptic feedback by touching note objects. 3) freely generate sequencer units to compose musical fragments. 4) produce warm haptic sensations, sustained tones, arpeggio sounds, and visual effects by touching palms with others, with changes occurring as hands move without parting. 5) blend the shadows of note objects and oneself using the XLS system.
The remote MR experience unfolded in this work is a sort of “real-space based metaverse” using AR markers as shared space anchors. Unlike using VPS, each participant’s “real space” in the mixed reality differs, making the MR experience “asymmetric” among participants. This “imperfect shared experience” may create some communication conflicts and disturbances in each real-space. However, it allows participants to engage in an alternative reality, interacting virtually in a familiar real space using their real bodies. By connecting multiple isolated spaces (closed systems) through musical acts and generating disruptions (“interpenetration”), this work aims to explore the potential of communication through XR technology in a playful manner.
【Detail of the work】
Meta Musicking