Transcript
Immersive SOUND for VR
오현오 (Henney Oh) GAUDI AUDIO LAB
SOUND is half the experience
Listen
SOUND is half the experience
- George Lucas
Immersion into is a perception of being physically present in a non-physical world.
technology refers to technology that blurs the line between the physical world and digital or simulated world,
thereby creating a sense of immersion.
VR
Immersive
by wikipedia
Virtual Reality
PS 게임_ “하쿠나 마타타”
for
Immersive & Interactive
in Mobile
Head-Mounted Display
VR like HMD
HMD is hot
Facebook, Oculus VR $2bil에인수(Mar. 25, 2014)
Samsung, Oculus와협력발표(May 31, 2014)
Sony, Project Morpheus 발표(Mar. 18, 2014)
Google, Cardboard VR(June 25, 2014)
Samsung Galaxy VR (Oculus + Note 4) 발표(Sep. 3, 2014)
Crescent Bay (Headphones)(Sep. 20, 2014)
Microsoft HoloLens(Jan. 21, 2015)
HTC – Valve, Revive (SteamVR)(Mar. 1, 2015)
LG – plastic cardboard(Feb. 10, 2015)
Apple acqr. Metaio(May 28, 2015)
Google invst. Magic Leap(Oct. 21, 2014)
but, no one seriously thought SOUND
Immersive & Interactive 3D SOUND
Sound and Room
Sound is a Wave
Sound = Longitudinal waveSound is a mechanical wave resulted from the longitudinal motion of the particles of the medium through which the sound wave is moving
Anechoic and Reverberant Room
Room Impulse Response
Direct Sound
sound object
microphone
0 Time
Freq
A
A
Room Impulse Response
DS + 1 reflection (echo)
0 Time
Freq
Room Impulse Response
DS + Early Reflections (Multiple Echoes)
0 Time
B-2案 _ IN SEARCH OF THE ORIGIN 음향 시스템의 기원인 축음기의 트랙으로부터 영감을 얻어 끊이지 않고 플레이되는 GAUDI 의 음향을 표현
장식적이고 복고스러운 개성이 강하여 아이덴티티로서 베스트안은 아님
Room Impulse Response
DS + ER + Late Reverberations
0 Time
Late Reverberation
Direct Sound Early Reflections
B-2案 _ IN SEARCH OF THE ORIGIN 음향 시스템의 기원인 축음기의 트랙으로부터 영감을 얻어 끊이지 않고 플레이되는 GAUDI 의 음향을 표현
장식적이고 복고스러운 개성이 강하여 아이덴티티로서 베스트안은 아님
Sound and Binaural
Binaural
Widely used to mean “headphone virtualization”
Having or relating to two ears
Binaural Hearing
High Frequency (above 4kHz)
• Inter-aural Level (intensity) Difference
• ILD
Low Frequency (Below 1kHz)
• Inter-aural Time (phase) Difference
• ITD L1 L2
Cone of Confusion
How resolve front-back-above-below ?
?
Head and Outer Ear (pinna)
We can since we are trained !!
Spatial Cues
Azimuth cueCaused by head shape and pinna
ILD, ITD
Elevation cueCaused by shape of pinna
Freq. peak and notch
Range cueLoudness, motion parallax, excess ILD, ratio of direct to reverb
HRTF
HRTF: Head Related Transfer FunctionHRIR: Head Related Impulse Response
HRTF Measurement
Binaural Room Impulse Response
BRIR = RIR + HRIR
B-2案 _ IN SEARCH OF THE ORIGIN 음향 시스템의 기원인 축음기의 트랙으로부터 영감을 얻어 끊이지 않고 플레이되는 GAUDI 의 음향을 표현
장식적이고 복고스러운 개성이 강하여 아이덴티티로서 베스트안은 아님
Binaural Recording v. Binaural Rendering
Binaural
Binaural Recording
Sound Source(Wave) RIR HRTF Human
Perception
Binaural Rendering
Binaural Recording
Manikin measures binaural sound for “general” human
Binaural Recording
Capture immersive sound there
Binaural Rendering
Synthesize immersive sound here
*hL(t)
hR(t) *HRTF Database
mono
Recording v. Rendering
Binaural Recording Binaural Rendering
OOOOO Complexity O
OOOO Immersive OOO
OO Interactive OOOOO
O Personalization OOOOO
Vs.
Challenges in Implementation
Technical Challenges
1. Interactive in Binaural Recording
2. Expensive to filter in real time
3. Hard to keep both timbre and localization
4. Hard to cover full 3D space with measured HRTF
5. Hard to match to visual content
6. Hard to get ‘one-size-fits-all’ HRTF
B-1案 _ A HOMAGE TO GAUDI 가우디 건축물의 도형으로부터 모티브를 얻어 기하학적인 형태로 문자를 표현
Interactive in Binaural Recording
Issues on 360 binaural recording
B-1案 _ A HOMAGE TO GAUDI 가우디 건축물의 도형으로부터 모티브를 얻어 기하학적인 형태로 문자를 표현
Match to visual content
B-1案 _ A HOMAGE TO GAUDI 가우디 건축물의 도형으로부터 모티브를 얻어 기하학적인 형태로 문자를 표현
Implementation Example
A-1案
Summary
Immersive sound is NOT conventional 3D soundFor HMD, Immersive and Interactive are essentialBinaural means Headphone VirtualizationBinaural Rendering vs. Binaural RecordingStill open issues
Demo is OPEN in booth #5
Q&A
Thank You
top related