Themes of Research

Our research topic is the highly impressiveness and improvement of the information presentation capabilities, especially MR world using audio-visual senses. (This is a joint research with Acoustic & Signal Processing Laboratory. in Department of Media Technology.)
In previous studies of MR system using audio-visual senses, we have realized these kinds of functions.
* Mixing the both real and virtual world sound, considering the reflection and insulation
* Using sound event which emitted intentionally in the real world as an input for MR world
* Utilize the characteristics and combining both headphone and speaker
* Appending the Doppler-effect to the sound of virtual world in the MR world

Addition to these functions, we aim to realize MR system using audio-visual senses by introducing acoustic planetarium to construct more advanced MR space.

We research about the completely new sound providing method "acoustic planetarium. "This method is by placing enclosure which is installed several parametric loudspeakers in the middle of the room, these speakers emit the sound like a planetarium projector and we can experiment auditory MR on anywhere.
Parametric loudspeaker which we use have highly directional. This enables us making 3D sound field by emitting sound anywhere and reflecting or spreading it with wall or reflecting object. By this way, several people can share the sound field without mounting headphone, unlike the traditional system.
We define the MR system with acoustic planetarium and the MR world as "X-Media Galaxy," and aim to realize the 3D space with high presence which people have never experienced so far.

We define the MR system with acoustic planetarium and the MR world as "X-Media Galaxy," and are aiming at that multiplayer can listen the sound synthesized real sound and artificial sound at the same time. X-Media Galaxy show CG objects in the position which was located a sound by the acoustic planetarium. In this way, player feel like that virtual object make a sound.
In addition, as the next step of X-Media Galaxy, we tackle the Implementation of “X-Media Galaxy/Dome Type” (X-Dome) which is expanded expressiveness and presence in terms of both visual and hearing by the dome easily control the acoustic. In X-dome, we aim to give a high presence by wrapping player with pictures and sound. We achieve this by projecting the background onto the inner wall by both HMD and projectors.

With Acoustic planetarium, we can localize a sound image in any position by changing the direction of the speaker when a sound image is moved. However, there are several problems that drive sound is generated by physical control and .we cannot change the direction of the speaker in real time. In this study, we suggest the realization method of the moving sound source without changing the direction of the speaker. That is to interpolate the sound between multiple sound images on reflecting points which located with parametric loudspeakers.

Sound is invisible. In our "Sound Visualization" research, we express the sound by using CG in the MR world. We aim to determining the position of sound source, increasing the expressiveness, supporting the system constructing by visualizing the sound. Recently, we have researched “Visualization of sound emitted from parametric loudspeaker” and “Suggestion of sound visualization techniques and creating the tool to support sound visualization.”

"Mixed Reality system using audio-visual senses"

"Implementation example of "X-Media Galaxy""

“Implementation example of "X-Media Galaxy/Dome Type"”

“The example of sound visualization tool”