Virtual Sounds

Are Virtual Sounds Real?

Acoustics Today; https://acousticstoday.org/are-virtual-sounds-real-michael-vorlander/

Author: Michael Vorländer

In this episode, we interview Michael Vorländer from the RWTH Aachen University about the reality of virtual sounds.

Read more from Acoustics Today.

Learn more about Acoustical Society of America Publications.


Music Credit: Min 2019 by minwbu from Pixabay. https://pixabay.com/?utm_source=link-attribution&utm_medium=referral&utm_campaign=music&utm_content=1022 

Transcript

Ambri Phillips

00:06

Welcome to Across Acoustics, the official podcast of the Acoustical Society of America’s Publications office. On this podcast, we will highlight research from our four publications, The Journal of the Acoustical Society of America, also known as JASA, JASA Express Letters, Proceedings of Meetings on Acoustics, also known as POMA, and Acoustics Today. I'm your host, Ambri Phillips, Business Administrator for the ASA.

AP

00:37

Joining me today is Michael Vorländer of our RWTH Aachen University. We'll be discussing his article, “Are virtual sounds real?”, which appeared in the spring 2020 issue of Acoustics Today. Thanks for taking the time to speak with us today, Michael, how are you?

Michael Vorländer 

00:51

I'm fine, Ambri. Thank you very much.

AP

00:54

Of course. Tell us a bit about your background.

MV

00:58

Well, I'm physicist by education and have been a professor of Acoustics in Electrical Engineering in Aachen in Germany since 1996. I have always been interested in the field where physics, computer simulation, acoustic measurements, and 3D audio technology meet. Virtual acoustics combines these fields and offers fascinating possibilities for many applications. 

AP

01:21

Okay, great. What is a virtual sound?

MV

01:24

Virtual sound is real. It is audible, just as a real sound is audible. The difference is how it was created, not by an event or activity in the real world, but in the computer using software algorithms for sound synthesis.

AP

01:38

Great. Can you provide us with the history of virtual reality and virtual sound?

MV

01:45

Yeah, virtual reality technology was developed in the 1960s, where the focus on immersive 3D Video Systems for kind of 3D film experience. Image and video processing or computer vision was in fact the driving technology for virtual reality. So called CAVE systems, surrounding video screens were developed in the 1990s. And around that time, 1990, the first auralization tools were also introduced that could simulate and reproduce music and speech in concert halls and auditoriums. In this context, however, auralization is the sister of visualization in computer vision, but it's not identical to acoustic virtual reality. While in the 1990s, the technology for 3D image processing was already at a very high level in terms of real time performance, virtual sound rendering required several hours or days of computation time. This is a drastic violation of what be called real time processing, an important requirement for virtual reality.

AP

02:48

Okay, great. How do you create an immersive experience via virtual reality? How do you use virtual sounds when creating a virtual reality experience?

MV

03:00

The three pillars of virtual reality are real time performance, interaction, and multimodality in 3D. The better they are fulfilled, the more immersion we get. The user should have the freedom to interact with a virtual environment, the users must receive immediate real-time feedback on the actions typically within 50 milliseconds. And all of this must finally be provided with auditory-visual 3D display technology. Immersion is a complex, psychological construct and involves aspects of a believable environment. Which means spatial and source perception, freedom to interact with the environment in the sense of being part of it, and causality. It is important that visual and auditory perception match in both spatial localization and temporal synchrony. When everything is perfect, we can create the illusion of presence in the virtual space.

AP

03:55

What are some of the benefits of using virtual reality?

MV

04:00

That's a good question. There can be several answers to this. In the gaming industry, one would answer that this is a lot of fun. And it is. In the kind of academic research and development we can push the boundaries by creating highly realistic experiments, where people are exposed to virtual realities. This is called ecological validity, so that the human response to sound is evaluated in more complex rather than drastically reduced laboratory situations. Another example, in industry, we can save cost and time by developing products with digital twins of virtual prototypes, instead of constructing real prototypes, there are also examples in architecture in production and automotive industry.

AP

04:48

What are some of the challenges associated with creating virtual sounds?

MV

04:55

The challenge is to achieve a balance between the complexity of the virtual environment, the number of sources in that environment, everything associated with quality and realism on the one hand, and real time performance on the other. Of course, with physics based simulations, there are also many challenges in defining the appropriate input data for sources and materials.

AP

05:17

Okay, can you describe the steps to create sound in virtual environments? How do you make it 3D?

MV

05:25

Yeah, we can start with sound sources, either from recordings or from physical models and synthesis. These primary sound signals propagate in the environment as they will travel as a sound wave in the real environment, where sound as reflected, refracted, attenuated, and so on. Computer models of sound propagation are used for this purpose, usually geometric methods such as ray tracing and image models. Then the simulated sound reaches a receiver where it's collected with information about the sound incidence as a function of time, and frequency and direction. And finally, it is processed and reproduced in 3D audio formats using appropriate loudspeakers or headphones. This finally depends on the audio technology chosen. Today, headphones and head mounted displays are most popular because they are good and cheap.

AP

06:18

Okay, what is the best way for a listener to experience virtual acoustics? How can you tell that you've created an immersive experience?

MV

06:28

Yeah, the quality of the auditory visual experience depends on the available computing power, but also on the quality of the head mounted display and the headphones. How do we know if the immersion is sufficient for the user? Of course, we can ask the users but more reliable empirical data about user behavior in VR. If human responses to auditory visual stimuli are similar to those in a real world reference situation. If they are attentive and focused on the activity in the VR environment, we can assume sufficient immersion.

AP

07:01

Okay. Can you describe some of the applications of virtual acoustics environments in research?

MV

07:09

In research, auditory cognition experiments in psychology can benefit from virtual acoustic environments with a higher ecological validity, as mentioned earlier. There is a global mega trend towards such use of VR in laboratory experiments. In architecture design tools provide interfaces for acoustic simulation and 3D perception of spaces which are not yet built. Noise perception is another important application. Soundscape research can benefit from virtual reality, as studies can be extended from real on site investigations to sound works in VR. To include social cultural factors when the experiment is conducted with larger and more diverse populations of subjects, and not just in one place, but in international projects.

AP

07:57

What is the importance of studying virtual reality?

MV

08:00

Well, for all the students in the world, there's an excellent job opportunity today and in the near future, we can expect even more VR in academia and industry. It is also important because it combines knowledge from acoustics, audio engineering, computer science and psychology. And a sound multidisciplinary education is always a good idea. 

AP

08:23

That’s great! Okay, well, thank you, Michael, for taking the time to speak with us today. It was great learning about virtual sounds and virtual reality with you.

08:31

So you should start studying it!

08:37

Thank you for tuning in to Across Acoustics. If you'd like to hear more interviews from authors about their research, hit subscribe and find us on your preferred podcast platform.