Immersive virtual acoustic spaces

Creating immersive sonic experiences by situating participants in the acoustics of any building or physical structure in augmented and mixed reality.

Bringing the Room to the Music 

After Stanford Music Adjunct Professor Jonathan Abel ’84 M.S. ’89 Ph.D. and Eoin Callery ’16 DMA co-invented a technology that can recreate the acoustics of any physical space—including Istanbul’s Hagia Sophia—they knew there were potential commercial applications for the system, which they developed with input from Stanford Music Professor Jonathan Berger  ’83 DMA. But they realized how much they would benefit from the expertise of the HIT Fund team in exploring those applications.

To help, the HIT Fund is setting up meetings with high-level executives, including in film and gaming, and an MBA student will be shaping the business model.

“Nitin and Laura asked questions that just really focused us on the kind of things we could be doing,” Abel says. “It takes a certain skillset to understand how to get the buy-in to make it happen.”

Callery agrees.

“You can have all the ideas you want, but you need an outside voice to rein everything in,” he says. “If you have 150 applications, that’s great, but what’s the one that should be tackled first?”

About the Technology

CAVIAR (Chamber for Augmented Virtual and Interactive Audio Realities) alters the acoustic conditions of any space, transporting sounds and listeners in the space to new and even changing architectures. Using speakers and microphones about the space, CAVIAR allows participants to freely interact with each other, hearing sounds as if they were in a completely different room.

The fully immersive acoustic environments created with our system can take on and morph among the characteristics of any number of specific or idealized rooms, concert halls, caves, etc. In addition, the system can be used to create truly surreal acoustic conditions, including those of spaces that can not be experienced through real-world physical architecture. CAVIAR is also being developed for integration with networked communication technologies such as gaming, video conferencing, and sports broadcasting, so that users can interact in shared acoustics remotely, increasing the possibility of meaningful virtual engagement.

Team Members

Video

Press

Listen: The Sound Of The Hagia Sophia, More Than 500 Years Ago (NPR, Feb. 2020)

Publications

• Jonathan S. Abel, Eoin F. Callery, Elliot K. Canfield-Dafilou. A Feedback Canceling Reverberator. Proceedings of the Digital Audio Effects Conference (September 2018).
• Canfield-Dafilou, Elliot K. and Callery, Eoin F. and Abel, Jonathan S. and Berger, Jonathan. A Method for studying interactions between music performance and rooms with real-time virtual acoustics. Audio Engineering Society 146 (2019).
• Eoin F. Callery and Elliot K. Canfield-Dafilou. Methods for performing with feedback in virtual acoustics. Proceedings of the 17th Sound and Music Computing Conference (2020).

Related papers from an earlier project that prompted the development of CAVIAR:

• Jonathan S. Abel and Kurt J. Werner, K.J. Live Auralization of Cappella Romana at the Bing Concert Hall, Stanford University. Aural Architecture in Byzantium, edited by Bissera V. Pentcheva (Abingdon: Routledge, 2017), pp. 198-223.
• Bissera V. Pentcheva and Jonathan S. Abel. Icons of sound: auralizing the lost voice of Hagia Sophia. Speculum, 92, no. S1 (October 2017): pp.S336-S360.