Spatial Audio.pdf - Washington

2/15/18

1

Spa$al Audio for VR: An Overview

What is spa$al audio?

•  alsoreferredtoas3Daudioor360audio•  sonicexperiencewhere

•  theaudiochangeswiththemovementoftheviewer’shead

• producedbystereospeakers,surround-soundspeakers,speaker-arrays,orheadphones.

•  Spa6almusicismusiccomposedtointenConallyexploitsoundlocalizaCon

•  inusesinceprehistoricCmesintheformofanCphon•  inusesincearound1928as‘Raumusik’or“spacemusic”fromGermany

2/15/18

2

Spa$aliza$on:

•  theprojecConandlocalizaConofsoundsourceinaspace,•  Physical•  simulated

•  anditsspaCalmovementinspace.

•  technicallyknownasspaCaldomainconvoluConofsoundwavesusinghead-relatedtransferfunc6ons(HRTF).

Crea$ng posi$onal sound

• Amplitude•  (ormore)

•  SynchronisaCon•  Audiodelays

•  Frequency•  Head-RelatedTransferFuncCon(HRTF)

2/15/18

3

Amplitude• GenerateaudiofromposiConsources• Calculateamplitudefromdistance•  Includedampingfactors

•  AircondiCons•  Snow•  DirecConaleffectoftheears

Synchronisa$on

•  Earsareverypreciseinstruments• Verygoodathearingwhensomethinghappensa]ersomethingelse

•  Soundtravelsslowly(c340m/secinair):differentdistancetoeachear

• UsethistohelpdefinedirecCon•  DifferenceinamplitudegivesonlyveryapproximatedirecConinformaCon

2/15/18

4

Speed effect

•  30cenCmetres=0.0008seconds• Humancanhear≤ 700µS

What is 3D sound?

• AbletoposiConsoundsallaroundalistener.•  Soundscreatedbyloudspeakers/headphones:perceivedascomingfromarbitrarypointsinspace.

• ConvenConalstereosystemsgenerallycannotposiConsoundstoside,rear,above,below

•  Somecommercialproductsclaim3Dcapability-e.gstereomulCmediasystemsmarketedashaving“3Dtechnology”.Butusuallyuntrue.

2/15/18

5

3D posi$onal sound

• Humanshavestereoears• Twosoundpulseimpacts

• Onedifferenceinamplitude• OnedifferenceinCmeofarrival

• Howisitthatahumancanresolvesoundin3D?• Shouldonlybepossiblein2D?

Frequency

•  FrequencyresponsesoftheearschangeindifferentdirecCons•  Roleofpinnae•  Youhearadifferentfrequencyfilteringineachear•  Usethatdatatoworkout3DposiConinformaCon

2/15/18

6

Head-Related Transfer Func$on

•  UnconscioususeofCmedelay,amplitudedifference,andtonalinformaConateacheartodeterminethelocaConofthesound.

•  Knownassoundlocalisa+oncues.•  SoundlocalisaConbyhumanlistenershasbeenstudiedextensively.

•  TransformaConofsoundfromapointinspacetotheearcanalcanbemeasuredaccurately

•  Head-RelatedTransferFuncCons(HRTFs).•  MeasurementsareusuallymadebyinserCngminiaturemicrophonesintoearcanalsofahumansubjectoramanikin.

HRTFs

• HRTFsare3D•  Dependonearshape(Pinnae)andresonantqualiCesofthehead!•  AllowsposiConalsoundtobe3D

•  ComputaConallydifficult•  Originallydoneinspecialhardware(Convolvotron)•  Cannowbedoneinreal-CmeusingDSP

2/15/18

7

HRTFs

•  FirstseriesofHRTFmeasurementexperimentsin1994byBillGardnerandKeithMarCn,MachineListeningGroupatMITMediaLab.

•  Datafromtheseexperimentsmadeavailableforfreeontheweb.

•  PictureshowsGardnerandMarCnwithdummyusedforexperiment-calledaKEMARdummy.

•  A measurement signal is played by a loudspeaker and recorded by the microphones in the dummy head.

HRTFs

• Recordedsignalsprocessedbycomputer,derivestwoHRTFs(le]andrightears)correspondingtosoundsourcelocaCon.

•  HRTFtypicallyconsistsofseveralhundrednumbers•  describesCmedelay,amplitude,andtonaltransformaConforparCcularsoundsourcelocaContole]andrightearsofthesubject.

• MeasurementprocedurerepeatedformanylocaConsofsoundsourcerelaCvetohead

•  databaseofhundredsofHRTFsdescribingsoundtransformaConcharacterisCcsofaparCcularhead.

2/15/18

8

HRTFs

•  Mimicprocessofnaturalhearing•  reproducingsoundlocalisaConcuesattheearsoflistener.

•  UsepairofmeasuredHRTFsasspecificaConforapairofdigitalaudiofilters.

•  Soundsignalprocessedbydigitalfiltersandlistenedtooverheadphones•  ReproducessoundlocalisaConcuesforeachear•  listenershouldperceivesoundatthelocaConspecifiedbytheHRTFs.

•  Thisprocessiscalledbinauralsynthesis(binauralsignalsaredefinedasthesignalsattheearsofalistener).

HRTFs

•  The process involved in generating true 3D audio using HRTFs:

2/15/18

9

Fourmajordivisionsofauditorysystem–Func6on

I. Outer ear: (1) Pinna (Binaural cue to sound source loca$on)

t Left

t Right

*Differentdistancesfromsourcetoeachear=>differentarrival6mes(Interaural6me-difference)anddifferentsoundlevel(interaurallevel-difference)

Rightear

LeNear

Sound

2/15/18

10

I. Outer ear: (1) Pinna (Spectral cue to sound source loca$on)

Out

er e

ar g

ain

-15° 0°

+15°

ThespectralfeatureofsoundischangeddependingonthesoundelevaCon=>HeadRelatedTransferFunc6on(HRTF)

Basics of Object Based Audio WhatisanObjectBasedAudio?

Audiowhichisgeneratedby

•  aStaConaryormovingobjectOR

•  aclassofobjectsthatareclubbedtogetherasacollecCvesourceofsound

SomeExamplesofAudioObjectsare:

•  AStagearCste

•  Chorus

•  CheeringCrowdatCricketground

•  Oceanwaves,windblowing

•  Birdschirping

•  AnAirplaneorAHelicopter

•  AmovingTrain

•  Abulletfired

•  Amonologueoradialogue

2/15/18

11

Examples of Object-Based Audio AudioSceneorSoundFieldgeneratedbymixing(notjustadding)audiosignalsfrommulCpleObjects.Followingareafewexamples.

•  Watchingafootballmatchinastadiumwithhomecrowd.(SportsTVChannel)3objects:HomeCrowdasaringobject,Commentatorasapointobject,Players-Umpire conversaConsasanotherobject.

•  ParCcipaCngasaplayerinafieldgame.(ComputerGames)4+objects:HomeCrowdasaringobjectaroundyou,Commentatorasapointobject, Player’sownvoiceresponsesasapointobject, OtherPlayers-UmpireasmulCplemovingobjects.

•  BeingapartofScuba-diverteamsearchinganunderwatertreasure.(VR)

•  ListeningtoaconversaConbetweendifferentactors&backgroundsinamoviescene.(cinema)

•  ArendingamusicconcertorasimpleJazzperformance(concert)

ChannelBasedImmersiveAudio ObjectBasedAudio

ContentCreaCon

•  Eachsignaltrackisassociatedwithaspecificspeakerfeed&setupatlistenerend.

•  ContentiscreatedforaspecificListenerEnvironmentorsetup.(mobile,home,ortheater)

•  AudioObjectbasedsignaltracksareindependentofspeaker-setup.

•  =>ContentcreatedisindependentListenerEnvironmentorsetup.(mobile,home,ortheater)

PlaybackatListenerEnd

•  AtListenerend,thecontents(channels)aremappedontouserspeakersetup

•  NeedtousePredefinedchannel-mappingtoheadphones,stereospeaker,2.1,5.1,11.1etc.

•  AtListenerend,theobjectsaremappedontouserspeakersetup

•  ObjectsbasedonposiConsandmovementsaremappedontheflytothespeaker-setup.

Channel Based Audio vs Object Based Audio

2/15/18

12



ContentCreaCon

•  Withinputsastherecordedcontentsortracks,eachChanneltrackiscarefullydesignedandcreatedattherecordingstudios.ORatthegamingdeveloperstudiosforcreaCnggoodimmersiveeffects.

•  AudioObjectscanbesimplyidenCfiedencodedasseparatetracks.

•  Associatedmeta-datashouldbecarefullydesignedtocaptureshape,movement,appearance/disappearanceoftheobjectsassumingthelisteneratthecenter.


•  Ifthecontent-targetspeaker==userspeakersetup,thensimple-mappingandplayback.

•  Elseusesomegoodpre-definedmapsanddelaysforrearspeakerstocreatethecontent.

•  Objectsaredecodedtocreateaudiosignals.•  Frame-by-Frame,posiConsof“acCveobjects”

aremappedontouserspeakersinformofgainsanddelaysfortheseobjects.Mixandplayback.


ContentCreaCon

•  CreaConisacomplexcarefulprocess.•  Encodingstepsandprocedureiscomplexand

henceisdonebyskilledwelltrainedsounddesigners.

•  CreaConandencodingobject-audioisarelaCvelysimplerprocessandcanbedonewithoutmuchpre-thinkingofuser-setups&environment.

•  Audioobjectmeta-dataneedstobecarefullyassociatedwithit.


•  DecodersareRenderersarefairlysimple.

•  Decodersaresimple(assimpleaschannelbasedAudio).

•  HowevertheRenderersaremuchmorecomplex.

•  RenderersneedtomaptheseobjectswithitsposiConstospeakersonaframe-by-framebasis.


2/15/18

13

A summary – basic of Object-Based Audio

Enc-Audio-ObjFrame1

Enc-Audio-ObjFrame2

Enc-Audio-ObjFrameN

MetaData-ObjFrame1

MetaData-ObjFrame2

MetaData-ObjFrameN

ObjectAudioTrack

TypicalObject-AudioEncodedStreamcontains8to16encodedaudioobject-tracks.Andeachaudio-objecttrackhastwoparts

Meta-Data:

Stream-level:•  MaxNumberofAudioObjectspresentinthescene.

Frame-level:•  Shapeoftheobject,DatarelatedtoposiCon,speedof

theobject,Appearance/Disappearanceofobject

CompressedPCMAudio-Data:

StandardDDorAACencodedaudiosignalsassociatedwithaspecificobject.

Object-Based Audio Stream Decoding & Rendering

•  Decoding:Thebasicaudiofromobjectisencodedusingstandardlegacyencoders.Therefore,decodingusesstandardmp3,aac,dolby-digitaldecodingtoprovidebasicaudioPCMfortheobject.

•  Renderers:ChallengesareinRenderingthedecodedobject-basedaudioPCMcontents&useobject’sshape/moConmeta-datatocreate–

•  Animmersiveaudioexperienceonheadphones.(VR,Gaming,smartphones,andtablets)

•  Animmersiveaudioexperienceonourmul6-speakerlayoutsathomesortheaters

2/15/18

14

Object-Based Audio Renderer on Headphones

Diagram of spherical coordinate system(WightmanandKistler, Univ Winsconsin) – 1989 [1]

Earimpulse&frequencyresponsefororientaConshowninpictureonle](90ºazimuth,0º-elevaCon)fromWKSDOset.[2,3](Note:5to9msecduraConimpulseresponsewidth@16kHzsamplerate)

HRTFModel(HeadRelatedTransferFuncCon)

Object-Based Audio Rendering on Headphones

DecodedaudioobjectPCMdata

HRTF_L/R–Filterpoolfordiffvaluesof

φi & θi

DecodedaudioobjectMetadata

(dist,azi,ele)

Gain/delayModule

Distance->Gaindelaymapping

φ & θ

Interpolate&ComputeHRTF_Le]&

HRTF_Rightfromfilter-pool

HRTF_Le]

HRTF_Right

FIRFilterPair

R

MIXER

2/15/18

15

Object-Based Audio Rendering on Headphones Signal

Processing

Challenges

PresetFIRFilterpool

DecodedaudioobjectPCMdata

DecodedaudioobjectMetadata(dist,azi,ele)

GaindelayModule

Distance->Gain,delaymapping

φ & θ

ComputeHRTF_Le]&HRTF_Right

HRTF_Le]

HRTF_Right

FIRFilterPair

R

MIXER

•  TheparametersR, φ & θchangeeveryframe(20-30msec)⇒ filtercoeffs,gainchangeeveryframe⇒ Thismaycauseglitches,distorConsintheoutputs.⇒ NeedfortechniquestoadapCvely&smoothlychangethosecoefficients

•  TherearemulCpleobjects&someappearanddisappeara]erafewframes.⇒ Needforontheflyobjectpcm+associatedmeta-datamemory

allocaCon,updateanddestrucCon⇒ Needforfade-in/fade-out/muteofoutputPCMsamples⇒ Needforawell-designedmulC-portPCMmixingmodule

•  SomeObjectsmoveveryrapidly⇒ “R”changesw.r.t.Cme=>thespeedoftheobjectissubstanCalcausing

Dopplereffectonaudiosignal(e.g.afast-trainpassingby)⇒ Needforpitchshi]ing(variable-delay)moduletobeintroducedontopof

gainapplicaConModule.Oversampling&InterpolaConwouldberequired.

•  VR/ComputerGamesrelated:Head/JoysCckmovementschanges“R,φ & θ”⇒ AnaddiConalHead-TrackingorJoysCckmovementsmodulewhichfeeds

userorientaConparameters“Ru,φu & θu”⇒ AddiConalmoduletoPerform3-DgeometrycomputaConstoderivefinal

objectposiConparametersfromtheabove2sets“R,φ & θ”

Object-Based Audio Rendering on Immersive Speaker-Layouts

DTS/DD+7.1SpeakerLayoutDolbyATMOS11.1SpeakerLayouts

ExamplesofImmersiveSpeakerLayouts

2/15/18

16

Object-Based Audio Renderer on Immersive Speaker-Layouts

DTS-X7.2.4SpeakerLayout DTSNeo:X11.1SpeakerLayout



Ambisonics13.1SpeakerLayoutAuro-3D11.1SpeakerLayout


2/15/18

17


VBAP(VectorBasedAmplitudePanning):•  Alargearrayof“Virtual”SpeakerPosiConsareassumedtosurroundthelistener.Audio-

ObjectsandtheirmoCons/posiConsw.r.t.thelisteneraremappedonalargersetof“Virtual”SpeakerPosiCons.

•  AudiosignalsforeachobjectismappedonthisvirtualspeakerposiConsusingVBAP

method

•  Theaudioassociatedwithvirtualspeakersisthenmappedtostandarduserspeakerlayoutsusingpre-defineddown-mixingmatrices&setofdelays.

TwoMainTechniques.

•  VBAP–VectorbasedamplitudePanning:MappingobjectaudiotoVirtualSpeakerArray

•  HOA–HigherOrderAmbisonics:CreaCngdesired“Sound-Field”atlisteners’si}ngposiCon

HigherOrderAmbisonics [Gerzon1970]

•  Createsasoundfieldgeneratedbyaudio-object(s)whenitgetscapturedbydirec6onalmicrophoneslocatedatthelistener’sposiCon

HOA based object rendering on Immersive Speaker-Layouts

FirstOrderAmbisonicsfields

SecondOrderAmbisonicsfields

AnAmbisonicMicrophone

•  HOAchannelsareencodedandthesechannelsaredecodedandthenmappedontoanystandard“userspeakerlayouts”from5.1,or7.2.4or13.1”.Thesemappingsareeasyandlesscomplex.

•  HOAtechniquemakesiteasytomodifythesound-fieldfordifferentuser(listener)orientaCons(requiredmainlyinVR&ComputerGaming)

2/15/18

18

Ambisonics 3D (1st order)

Reproduction occurs over an array of 8-24 loudspeakers, through an Ambisonics decoder

Original Room

Sound Source

SoundField Microphone

B - format 4 - channels signal

(WXYZ)

Ambisonics decoder

Speaker array in the reproduction room

3D extension of the pressure-velocity measurements •  The Soundfield microphone allows for simultaneous measurements of the omnidirectional

pressure and of the three cartesian components of particle velocity (figure-of-8 patterns)

2/15/18

19

A-format microphone arrays •  Today several alternatives to Soundfield microphones do exists. All of them are providing “raw” signals

from the 4 capsules, and the conversion from these signals (A-format) to the standard Ambisonic signals (B-format) is performed digitally by means of software running on the computer

Ambisonics decoding

•  Each speaker fed is just obtained as a weighted sum of the 4 B-format signals (WXYZ)

•  The weighting factors only depend on the position of each loudspeaker

•  It is possible to add a small FIR filter for matching perfectly the gain and phase of all loudspeakers

r

y

z

x

α

β

γ

2/15/18

20

Ambisonics decoding advantages: n  Tridimensional n  Good later sound perception n  Good bass response n  Wide “sweet spot”, no colouring outside

it

disadvantages: n  Not isotropic n  Advanced decoding required (Y gets

more weight than X and Z, and this must be compensated for)

Bi-square Ambisonics array

Bi-square Ambisonics array

8 Turbosound Impact 50 loudspeakers: n  Light, easily fixed and oriented n  Good frequency response n  Very little distortion

Front-Right

Down-Right

Up-Right

2/15/18

21

What is Ambisonics? • Ambisonicsisamethodforrecording,mixingandplayingbackthree-dimensional360-degreeaudio.Itwasinventedinthe1970sbutwasnevercommerciallyadoptedunClrecentlywiththedevelopmentoftheVRindustrywhichrequires360°audiosoluCons.

•  ThebasicapproachofAmbisonicsistotreatanaudiosceneasafull360-degreesphereofsoundcomingfromdifferentdirecConsaroundacenterpoint.

•  Thecenterpointiswherethemicrophoneisplacedwhilerecording,orwherethelistener’s‘sweetspot’islocatedwhileplayingback.

Ambisonics B-format

•  ThemostpopularAmbisonicsformattoday,widelyusedinVRand360video,isa4-channelformatcalledAmbisonicsB-format

• usesasfewasfourchannelstoreproduceacompletesphereofsound.

2/15/18

22

Ambisonics vs. Surround

•  TradiConalsurroundtechnologiesaremoreimmersivethansimpletwo-channelstereo,buttheprinciplebehindthemisthesame:

•  Theyallcreateanaudioimagebysendingaudiotoaspecific,pre-determinedarrayofspeakers.

•  Stereosendsaudiototwospeakers;5.1surroundtosix;7.1toeight;andsoon.

•  Bycontrast,AmbisonicsdoesnotsendaudiosignalstoanyparCcularnumberofspeakers;itis“speaker-agnosCc.”Instead,Ambisonicscanbedecodedtoanyspeakerarray.Ambisonicaudiorepresentsafull,uninterruptedsphereofsound,withoutbeingrestrictedbythelimitaConsofanyspecificplaybacksystem.

Ambisonics as standard in 360 video and VR: •  TradiConalsurroundformatscanprovidegoodimagingwhenstaCc;butasthesoundfieldrotates,thesoundtendsto‘jump’fromonespeakertoanother.

•  Ambisonicscancreateasmooth,stableandconCnuoussphereofsound,evenwhentheaudioscenerotates(as,forexample,whenagamerwearingaVRheadsetmovesherheadaround).ThisisbecauseAmbisonicsisnotpre-limitedtoanyparCcularspeakerarray,

•  TradiConalsurroundspeakersystemsareusually‘front-biased’:informaConfromthesideorrearspeakersisnotasfocusedasthesoundfromthefront.Bycontrast,Ambisonicsisdesignedtospreadthesoundevenlythroughoutthethree-dimensionalsphere.

•  Finally,whereastradiConalsurroundsystemshavevariousdifficulCesrepresenCngsoundbeyondthehorizontaldimension,Ambisonicsisdesignedtodeliverafullspherecompletewitheleva'on,wheresoundsareeasilyrepresentedascomingfromaboveandbelowaswellasinfrontorbehindtheuser.

2/15/18

23

First-order Ambisonics B-format

•  Thefourchannelsinfirst-orderB-formatarecalledW,X,YandZ.OnesimplifiedandnotenCrelyaccuratewaytodescribethesefourchannelsistosaythateachrepresentsadifferentdirecConalityinthe360-degreesphere:center,le]-right,front-back,andup-down.

• AmoreaccurateexplanaConisthateachofthesefourchannelsrepresents,inmathemaCcallanguage,adifferentsphericalharmoniccomponent–or,inlanguagemorefamiliartoaudioengineers,adifferentmicrophonepolarpacernpoin6nginaspecificdirec6on,withthefourbeingcoincident(thatis,conjoinedatthecenterpointofthesphere).

First-order Ambisonics B-format

• Wisanomni-direcConalpolarparern,containingallsoundsinthesphere,comingfromalldirecConsatequalgainandphase.

• Xisafigure-8bi-direcConalpolarparernpoinCngforward.• Yisafigure-8bi-direcConalpolarparernpoinCngtothele].•  Zisafigure-8bi-direcConalpolarparernpoinCngup.

2/15/18

24

X, Y, and Z channels:

•  Afigure-8microphonehasaposiCvesideandanegaCve(inversephase)side.WhiletheXchannel’sfigure-8polarparernpointsforwards,itsnegaCvesidepointsbackwards.TheresulCngaudiosignalontheXchannelcontainsallthesoundthatisinthefrontofthespherewithposiCvephase,andallthesoundsfromthebackofthespherewithnegaCvephase.

•  ThesamegoesfortheYandZchannels:TheYchannelspickupthele]sideofthespherewithposiCvephaseandtherightsidewithnegaCvephase.TheZchannelpicksupthetopsideofthespherewithposiCvephaseandtheboromwithnegaCvephase.Thisway,bymeansofdifferenCalgainandphaserelaCons,thefourchannelscombinedrepresenttheenCrethree-dimensional,360-degreesphereofsound.

AmbiX vs. FuMa

•  twoconvenConswithintheAmbisonicsB-formatstandard:AmbiXandFuMa.Theyarequitesimilar,butnotinterchangeable:theydifferbythesequenceinwhichthefourchannelsarearranged,withAmbiX,forexample,arrangedWYZXinsteadofWXYZ.

2/15/18

25

First-order to sixth-order Ambisonics •  The4-channelformatisonlyasimple,first-orderformofB-format,whichiswhatmostAmbisonicsmicrophonesandplaybackpla�ormssupporttoday.

• Higher-orderB-formataudiocanprovideevenhigherspaCalresoluCons,withmorechannelsprovidingmoredifferentpolarparerns.

•  Second-orderAmbisonicsuses9channels,•  Third-orderAmbisonicsuses16channels,•  Sixth-orderAmbisonicsuses49channels.

Playing back Ambisonics (DxARTS, Rai` Hall)

Room117uses24full-rangespeakersand4subwoofersforfullheightspaCalsoundreproducCon.ThereisarouCngdecodingsystemtocareofpatching,ambisonicdecoding,speakerbalancingandroomcorrecCon,andthecrossoversfordistribuCngsoundtothesubs.YoucansendaB-formatsignaltovariousdecodersoraddresseachspeakerindividually,dependingonse}ngs.

2/15/18

26

Decoding Ambisonics

•  YoucanplaybackAmbisonicsonalmostanyspeakerarray,recreaCngthesphericalsoundfieldatthelisteningspot.Buttodothat,youneedtodecodethefourB-formatchannelsforthespecificspeakerarray.

• AllfourB-formatchannelsaresummedtoeachspeakerfeed.Eachofthefourchannelsissummedwithdifferentgainandphase,dependingonthedirecConofthespeaker.

•  Someofthesourcesinthemixaresummedin-phasewhileothersaresummedout-of-phaseateachspecificspeaker.

•  TheresultisthatsourcesalignedwiththedirecConofthespeakerarelouder,whilethosenotalignedinthedirecConofthespeakerarelowerorcancelout.

2/15/18

27

Ambisonics on headphones

•  SpaCalsoundonheadphoneismadepossiblebybinauralaudiotechnologies.Inessence,abinauralprocessorreceivesanaudioinputandadirecConinwhichtoposiConit.theprocessoraddsauditorycuestothesignal,sothatwhenplayedbackonheadphonesitisexperiencedatthesetvirtualposiCon.

•  ThemostcommonwaytoprocessAmbisonicsforbinauralspaCalplaybackonheadphonesistodecodetheAmbisonicschannelsforacertainspeakerarray–andsendthefeedstoabinauralprocessorwhichvirtuallyposiConsthematthedirecConthattheactualspeakerwouldhavebeen.

•  Theresultisthattheimmersivesphericalsoundfieldisexperiencedbythelistenerwhenmonitoringonheadphones.

Ambisonics on headphones

2/15/18

28

BinAural Rendering : Immersive Speakers -> Headphones. Dependinguponthefixedφ & θ anglesofFront,Rear&OverheadSpeakersw.r.t.Le]andRightearofthelistener

HRTF_Le]andHRTF_rightareapplied&

mixedasbelow.

HRTF_Le]Ear_Lf

HRTF_RightEar_Lf

Le]FrontSpeakerSignalLf

HRTF_Le]Ear_Rf

HRTF_RightEar_Rf

RightFrontSpeakerSignalRf

HRTF_Le]Ear_Lsr

HRTF_RightEar_Lsr

Le]RearSpeakerSignalLsr

HRTF_Le]Ear_Rsr

HRTF_RightEar_Rsr

RightRearSpeakerSignalRsr

+

HRTF_Le]Ear_Oh

HRTF_RightEar_Oh

OverheadSpeakerSignalOh

+

Content plaborms that support spa$al audio

•  YouTube•  Facebook360SpaCalWorkstaCon• GoogleVRResonanceAudio•  SamsungVR• Others

2/15/18

29

Facebook 360 Spa$al Worksta$on

•  TheFacebook360SpaCalWorkstaConisaso]waresuitefordesigningspaCalaudiofor360videoandcinemaCcVR.ItincludespluginsforpopularaudioworkstaCons,aCmesynchronized360videoplayeranduCliCestohelpdesignandpublishspaCalaudioinavarietyofformats.AudioproducedwiththetoolscanbeexperiencedonFacebookNewsFeedonAndroidandiOSdevices,ChromefordesktopandtheSamsungGearVRheadsetthroughheadphones.

Google VR Resonance Audio ResonanceAudiogoesbeyondbasic3DspaCalizaCon,providingpowerfultoolsforaccuratelymodelingcomplexsoundenvironments.•  TheSDKenables:

•  SoundsourcedirecCvitycustomizaCon•  Near-fieldeffects•  Soundsourcespread•  Geometry-basedreverb•  Occlusions•  RecordingofAmbisonicaudiofiles

•  3rdorderAmbisonicsformatssupported

2/15/18

30

The end…

Spatial Audio.pdf - Washington

Documents