Encounters Project

Encounters / Encounters 2.0

Ryu Nakagawa + Ken Sonobe
(2019 – 2020)

Concept/Creative Direction: Ryu Nakagawa
Programming/System Direction: Ken Sonobe


“What if we could make sound with physical objects by virtual objects?”
If we achieved this, then we could move a step closer to realizing a cross reality (XR) world in the near future: “How can we experience the XR world?” “How does such an experience change a form of art expression or a relationship between physical objects and human beings?” We would like to explore such concepts. Therefore, we aim to realize the interactions between physical objects; and virtual objects, and participants and physical objects, as well as between participants, using XR technologies. In general, the acronym “XR” is mainly used as the abbreviation for “extended reality.” However, as we mentioned, we use XR to mean “cross reality” rather than “extended reality” in this project. Furthermore, “cross reality” has been defined by Paradiso et al. as the union between ubiquitous sensor/actuator networks and shard online virtual worlds [1].

“Encounters” is a multi-participant audiovisual art experience with extended reality technologies. In this experience, participants can fire virtual bullets or beams at physical objects to create a physical sound along with a virtual visual effect. In addition, in version 2.0 of the Encounters, participants can also interact with light’s physical and immaterial aspects as if light has a ghost. This art experience provides an XR experience that participants can cross over the borders between physical and virtual spaces.

We believe this XR experience gives participants a sense of having a sort of supernatural power. On the other hand, participants would become more aware of physical objects in their presence (things, their own body and environment).

[1] J. A. Paradiso and J. A. Landay. 2009. Guest Editors’ Introduction: Cross-Reality Environments, in IEEE Pervasive Computing, vol. 8, no. 3, pp. 14-15, July-Sept. https://doi: 10.1109/MPRV.2009.47

Kuroko units

Light unit

Installation set






Nagoya City University, Aichi, Japan, Feb 21~23, 2020 [>]


Aichi University of the Arts, Aichi, Japan, Jan 15, 2020 [ > ]


SIGGRAPH Asia 2019, Brisbane, QLD, Australia, Nov 18~20, 2019  [ > ]


VR CREATIVE AWARD 2019, Tokyo, Japan, Aug 25, 2019  [ > ]


Published Paper

  • Encounters: A Multiparticipant Audiovisual Art Experience with XR, Ryu Nakagawa, Ken Sonobe, Proceedings of SA ’19 SIGGRAPH Asia 2019 XR, 6 – 8, 11 , Refereed   [ > ]


Supported by

  • The work was partially supported by JSPS KAKENHI Grant Number 18K00210.