SMART SPACES TESTBED
Marty Herman, ChiefVince Stanford, Computer Specialist
INFORMATION ACCESS AND USER INTERFACES DIVISION
NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY
Smart Space
• Embedded with computers, information appliances, multi-modal sensors
• Allows people to work efficiently through unprecedented access to information and help from computers
• Work can be done collaboratively or individually
Characteristics of Smart Spaces
Situation AwarenessIdentify and perceive users and their actions and goalsUnderstand and anticipate user needsUses speech and natural language understanding, computer vision, multi-modal fusion
Information AccessFacilitate interaction with information-rich sourcesAllow rich, natural forms of interaction between humans and computersProvide multi-media (e.g., text, voice, images, video) information
extraction
Information presentationProvide extensive capabilities for presenting information in an optimal and integrated mannerProvide immersive presentation capabilities
Characteristics of Smart Spaces(cont.)
CollaborationProvide for collaborative working environmentIncrease the rate of information interchange among usersPermit mobile workers to interact and participate in smart space activities
Mobile computingProvide for use of mobile devices and receivers, allowing ubiquitous “anywhere, anytime” accessProvide for discovery and integration of mobile devices with smart space infrastructure
Record keepingProvide improved memory of activities and deliberations for later useRecord and summarize dialogue, activities, and events
Create Smart-Space Integration Testbed at NIST
• Showcase future of Smart Space technologies• Integrate component technologies for demos• Develop and apply measurement and testing approaches• Develop and test prototype standards (e.g., interface
standards)• Develop test collections to support evaluations• Infrastructure for industry and academia to work hand-in-hand
with NIST• Facilitate adoption by industry
NIST Smart Spaces Testbed Scenario:Smart Meeting Room
• Room understands and guides meeting participants• Senses who is talking to whom and where they are located• Realizes when information is requested and outputs information that
seems to be useful• Engages in dialogue with participants to get more information• Room connected to Internet and digital libraries/databases• Personal information from palm/pocket computers are integrated
into room’s information infrastructure• Information displayed on information appliances and wall displays• Room aids in collaboration both within the room and with field
personnel
•Situation awareness•Speech, natural language input•Computer vision input•Integration of handheld computers into room•Retrieval & visualization of information•Distributed collaboration
Smart Meeting Room
Applications:
• Business meeting room• Medical consultation meeting room• Training and education facility• Military command center• Crisis management command center
Role of NIST
Smart Spaces is an emerging research area. Many of its component technologies require research to advance the state of the art. NIST can provide the following:
• measurement methods• testing and evaluation approaches• benchmark tests• reference materials (test data and test protocols)
Role of NIST(cont.)
Many technology suppliers will be required to make the vision of Smart Spaces a commercial reality.
To encourage commercial innovation, NIST can work with industry to provide open interface standards for interoperability.
Near Term Smart Space
• Experimental Collaborative interface using:– Microphone array, with beam fan, for acoustic signal
acquisition– Acoustic source location and tracking– Continuous speech recognition (CSR) using single
channel reduced from array– Speaker identification combined with CSR for
collaborative groups– Shared high resolution visual interface with information
visualization
Near Term Smart Space(cont.)
• Experimental Collaborative interface using:– Video camera array to identify, locate and track
individuals in the smart space– Video teleconferencing– Initial test case will use several functions:
• Broadcast news transcription• Real time spoken query processing• Information retrieval using BN transcripts
Current Status...• Lab space allocated; construction begins:
– Initial smart space November 23– Acoustically conditioned space 1Q99
• Computing, display, sensing infrastructure:– Dual Processor SGI Octane
• 640Mb Ram• thirty-two channels A/D• Dual camera inputs
– 1280x1024 large screen projector– Parallel Linux cluster:
• 112Gb disk array• Seven 400Mhz P-II compute nodes
– Gigabit Ethernet switch in room– Microphones:
• Desktop• Wall mounted array(s)
– Cameras:• Three computer controlled• One digital fixed mount
– 384Kb video teleconferencing– Wireless connection/Palm Pilot
• External cooperative research efforts:– Microphone array/acoustic source locator
from Rutgers CAIP; Jim Flanagan P.I.– BBN and IBM offering research and real
time speech recognition– Investigating possible use of MIT speaker
identification– Investigating possible use of U of Md visual
localization and gesture recognition• Internal cooperative efforts:
– Information retrieval (PRISE)– Spoken document retrieval with spoken
query (SPIDERS)– VRML smart space representation– Information visualization– Face recognition technology
Desired Long Term Capabilities/Technologies
• Accurate, robust speech recognition for transcription and command• Anticipatory Web and database query based on speech• Speech understanding:
– Spoken dialog abstraction– Selective recording of meeting minutes
• Data stream segmentation/annotation• Face/expression recognition• Persistence of meeting memory, automatic links to related meetings• Sensor fusion (acoustic/visual/other)• Smart white boards• Image understanding and person recognition• Participant sensing, task identification, and adaptive response
Metrics and Metrics Research Needed
• End-to-end metrics needed for multi-stage processes, e.g.: Full SDR– Sensing– Recognition– Understanding– Response
• Training and test reference data sets needed for:– Component recognition tasks, e.g.: CSR, speaker ID, gesture
recognition – Data reduction, e.g.: array processing and source location– Information extraction and summary
Metrics and Metrics Research Needed (cont.)
• Initial Smart Space is needed as test bed to bootstrap future improvements
• Command/Crisis Management may offer measurable, reproducible tasks
• Need well defined tasks with solution quality, time, and labor level measurements
Interoperability Issues are Severe• Rapid handheld appliance discovery and connection to IP
infrastructure, e.g.: IP masquerading or NAT • Dense wireless device deployment:
– IR
– RF• Multi-resolution devices and displays• Device protocol discovery, automatic translation among
heterogeneous protocols used by multiple devices• Multimedia output into the space and acoustic input• Security and accessibility to and by infrastructure