Situational Awareness for Fire Fighters (SAFIRE)

SAFIRE: Situational Awareness for Firefighters

Situational Awareness for Fire Fighters (SAFIRE)

Sharad Mehrotra

Professor of Computer Science


SAFIRE Team

Faculty Sharad Mehrotra Nalini

Venkatasubramanian Researchers,

Engineers, Programmers Naveen Ashish Chris Davison Dmitri Kalashnikov Jay Lickfett

Imagecat Inc. Paul Amyx Charlie Huyck Ron Eguchi

Graduate Students Subin Anthony Stefano Bonetti Chiara Chiappini Jeffrey Hu Dani Massaguer Ronen Vaisenberg Bo Xing


Outline

Center for Emergency Response Technologies

SAFIRE Project Goals, concept, deliverables, core research areas

Progress & Timeline SAFIRE system Research Testing & validation


Bren School Center for Emergency Response Technologies

Established in 2007 by the School of Information and Computer Sciences


CERT Mission

lead research, technology development & coordination of ongoing projects on role of IT to improving emergency response.

Provide a forum for collaboration between academia, industry, and government agencies.

5


CERT Research

Robust computing, communication, and storage infrastructure Graceful performance degradation under extreme

loads, catastrophic failures, surge demands

Real-time Situational Awareness from multimodal inputs to support decision making Diversity of information sources, modalities, and

needs, uncertainty in data, challenge of scale


CERT: Current Projects

NSF funded Responsphere Infrastructure Funded by NSF through research infrastructure program

(1.8M, 2004-2010)

NSF funded ITR-RESCUE Funded by NSF through its large ITR program (12.5M,

2003-2009)

DHS funded SAFIRE project Funded by DHS through FEMA fire fighter safety program

(1M, 2008-10).

7

SAFIRE: Situational Awareness for Firefighters 8

1/3 of UCI

sensing, communicating, storing, and computing infrastructure with ~ 300 sensors of ~ 10 types, several comms nets:~200 indoor net cameras (with mics), ~30 MicaZ motes (with light, acoustic, acceleration, magnetic, etc sensors), ~6 people counters, ~6 mobile cameras , RFID readers, outdoor cameras,gas sensor, mesh routers, 802.11 (Wi-Fi), Ethernet, power-line comms, 802.15.4 (Zigbee), mobile sensing platforms, storing server, computing servers, PCs, laptops, cellphones, PDAs, etc.

1/3 of UCI

sensing, communicating, storing, and computing infrastructure with ~ 300 sensors of ~ 10 types, several comms nets:~200 indoor net cameras (with mics), ~30 MicaZ motes (with light, acoustic, acceleration, magnetic, etc sensors), ~6 people counters, ~6 mobile cameras , RFID readers, outdoor cameras,gas sensor, mesh routers, 802.11 (Wi-Fi), Ethernet, power-line comms, 802.15.4 (Zigbee), mobile sensing platforms, storing server, computing servers, PCs, laptops, cellphones, PDAs, etc.

CERT: Infrastructure


CERT Industrial Partners

5G WirelessBroad-ranged IEEE 802.11

networking

AMDCompute Servers

Apani NetworksData security at layer 2

Asvaco1st responder (LAPD), and threat

analysis software

BoeingCommunity Advisory Board

Member

CanonVisualization equipment SDK

ConveraSoftware partnership

Cox CommunicationsBroadcast video delivery

D-LinkCamera Equipment and SDK

Ether2Next-generation ethernet

IBMSmart Surveillance Software (S3)

and 22 e330 xSeries servers

ImageCat, Inc.GIS loss estimation in emergency

response

MicrosoftSoftware

PrintronixRFID Technology

The School Broadcasting Company

School based dissemination

Vital Data TechnologySoftware partnership

Walker WirelessPeople-counting technology


CERT Government Partners

California Governor’s Office of Emergency

Services

California Governor’sOffice of Homeland

Security

City of Champaign City of Dana Point

City of Irvine City of Los Angeles

City of OntarioFire Department

City of San Diego

Department of Health and Human Services – Centers

for Disease Control

Lawrence LivermoreNational Laboratory

Los Angeles CountyNational Science

Foundation

Orange CountyOrange County Fire

Authority

U.S. Department ofHomeland Security


Outline


SAFIRE Project Goals, concept, deliverables, core research

areas Progress & Timeline

SAFIRE system Research Testing & validation


SITUATIONAL AWARENESS FOR FIRE FIGHTERS (SAFIRE)

Improve the safety of firefighters by providing decision makers with

greatly improved situational awareness during response activities


SAFIRE

Varying Situational Awareness Needs

1st responders @ crisis site dangers, hazmat in vicinity, ingress/egress routes, status of equipment,

victim location Technology Needs: information to enable immediate action,

knowledge of surroundings, role-specific and fine granularity data

IC @ crisis site Accurate assessment of the ongoing situation, response monitoring, and

on-site planning Technology Needs: mechanisms to monitor onsite, site-level

granularity, information assimilation, decision-support

EOC longer-term planning & consequence management

Traffic planning, shelter planning, structural health, public health, news/media Technology Needs: inter-organization data sharing, simulations to

help predict future, coarse level monitoring, decision support, ability to understand the “big picture”.

& Technological Requirements


SAFIRE Concept


How Situational Awareness can Help?

Example Scenario:Structure Fire

Abnormal event such as flashover detected by sensors.

Alert on FICB causes IC to conduct roll call earlier than normal.

RIC or closest firefighters can be directed to last known location.


SAFIRE Deliverables

Core technologies for creating SA at IC level

A prototype end-to-end SA system for IC Collect, manage, and transform diverse sensor

data streams from incident site into actionable information.

Connect to existing info systems (e.g., CAD) “Intelligent” interactive display system

Real-time situational data, alerting & analysis capability

Testing & validation of the SA system


SAFIRE System

FICB Visualization

External DataSources

Acousticdata

SAFIRE Core Technology AreasGIS

hazmat

occupancy

Multimodal Sensing Robust Network Infrastructure Visualization and User Interfaces Sensor stream processing Integration of external data sources

Video data

Environmentalsensors

Sensor databas

e FF physio.& location.


Sensing Technologies

SAFIRE has explored variety of sensor technologies: Personnel accountability / health

monitoring, Incident site / environment / hazard information, Resource tracking

Location sensing Despite numerous efforts,

effective indoor localization has remained a challenge! WIFI, UWB, IR, Acoustic,

ultrasound, video-based Acoustic sensing – e.g., first

responder communication Benefit of human cognition

Polar Heart Rate Module

Mote

Polar T31 Heart rate strap transmitter

Environmental sensorsMobile Optical/Acoustic sensors (D-Link TPZ cameras) Fixed Mote Infrastructure (dB, Temperature, Light, Humidity)

Personal sensors Pulse Rate (heartbeat), Acceleration, Temperature, Humidity, location, , Orientation, Barometer

http://images.google.com/imgres?imgurl=http://www.electronicsshowplace.com/images/isimages/DHDCS6620G.jpg&imgrefurl=http://www.electronicsshowplace.com/d-link-dcs-6620-10100tx-ptz-dual-codec-0-05-lux-optical-zoom-internet-camera-155066.html&h=500&w=437&sz=21&hl=en&start=2&um=1&tbnid=V8n6_R-M9iBFmM:&tbnh=130&tbnw=114&prev=/images?q=d-link+ptz+cameras&um=1&hl=en&sa=N

http://images.google.com/imgres?imgurl=http://www.electronicsshowplace.com/images/isimages/DHDCS6620G.jpg&imgrefurl=http://www.electronicsshowplace.com/d-link-dcs-6620-10100tx-ptz-dual-codec-0-05-lux-optical-zoom-internet-camera-155066.html&h=500&w=437&sz=21&hl=en&start=2&um=1&tbnid=V8n6_R-M9iBFmM:&tbnh=130&tbnw=114&prev=/images?q=d-link+ptz+cameras&um=1&hl=en&sa=N


Localization Framework An integrative technology-agnostic

localization framework Modeling location components Modeling location queries

Location Queries Diverse Types:

Simple queries: “where am I” Boolean queries: “Am I in Bren Hall” Nearest neighbor queries Proximity queries: “Are Alice & Bob within 100 feet”

Quality requirement Granularity – room level, building level, ... Certainty

Solve queries using the best combination of location components Best tradeoff accuracy-precision/cost

1 5 9 130

0.2

0.4

0.6

0.8

1

pmf_1

pmf_2

pmf_aggregate

0

0.2

0.4

0.6

0.8

1

pmf_1

pmf_2

pmf_aggregate

Search Space of Possible localization Plans

Based on Lazaridis & Mehrotra, SIGMOD 2006

Merging results of different components

SAFIRE: Situational Awareness for Firefighters23

Acoustic Sensing for Enhancing SA

• Type of Acoustic Analysis:• Human Speech: Who spoke to whom about what from where and when.• Ambient Sounds: explosions, loud sounds, screaming, etc. • Physiological Events: cough, gag, excited state of speaker, slurring, ..• Other features: too loud, too quiet for too long, …

23

Speech

Voice

Amb. Noise

Processing

AlertsConversation

monitoring & playback

Image/video tagging

Firefighters

Acoustic Capture Acoustic Analysis

Spatial Messaging

Localization

SA Application


Robust Networking Infrastructure

Multitude of technologies WSN, UWB, mesh networks,

DTN But ….

Rapidly deployable, self-configuring networks are still elusive!

SAFIRE-Net: a robust networking infrastructure for crisis site Exploit multiple networks that

together provide connectivity, Exploit mobility when disconnected

The Problem

Network Infrastructure that can deliver contextual data sensed by firefighters to the incident commander

FICB


Fire Incident Command Board (FICB)

Receive and display alert messages.

Available GIS layers

Firefighter StatusDashboard

Mapping andLocalization


Integrating External Sources through EBox EBox -- one-stop information source for pre-compiled and

evolving situational information GIS, floor plans, hazmat locations, hazmat descriptions, occupancy, real-time

building sensors and instrumentation, other external data sources

User can select from list of available GIS layers, documents, or sensor streams

Data automatically “ingested” by SAFIRE to provide an integrated view


Outline


SAFIRE Project Goals, concept, deliverables, core research areas

Progress & Timeline


Formation of the SAFIRE Advisory Board

County of Los Angeles F. D.Ron Cabrera, Battalion Chief

Newport Beach F. D.Paul Matheis, Division ChiefChip Duncan, Battalion Chief

City of Ontario F.D.Rob Elwell, Battalion ChiefDave Schuler, Battalion Chief

Orange County Fire AuthorityArt Nevarez, Battalion ChiefMichael Boyle, Battalion ChiefBill Lockhart, CaptainRich Toro, Comm. SupervisorJonathan Wilby, Safety Officer

•Advisory Board formed within 1 month of the grant

•Advisory board members involved in all aspects of SAFIRE

•Research prioritization•System development•Testing & validation•Technology transfer opportunities

•Four CERT Fire Fighter Forum in the past 1 year.

UC Irvine Police Department

UC Irvine Environmental Health & Safety

Deltin Corp.

Tactical Survey Group

Mesh Dynamics

Fire Agencies

Other Agencies and Companies


Progress: Component Level Research

Sensing - incorporation of multiple new sensors CO sensors, localization, speech

Networking & Data collection Infrastructure New antenna array for increased coverage, multi-network & store-

and-forward architecture Stream management

ability to incorporate variety of sensors , multimodal sensor archival and retrieval functionality

Visualization New functionalities in FICB – simplified UI, annotations, ebox

integration, etc. Ebox

Prototype development, ontologies for resource selection, integration of static and dynamic data such as sensing infrastructure of buildings

30

Talk by Prof. Venkatasubramanian

Talk by Dr. Kalashnikov

Talk by Prof. Mehrotra

Illustrated through demo/video


Fire Incident Command Boardreal time visualization of one incident

Alert Generatordefine alert rules, generate alerts

SAFIRE Servercoordinates all aspects of real time inicident monitoring

communication with external systems (CAD, EBox)configuration of stream processor, incident archiver

maintain representation of current statemessaging between clients and data stream sources

Incident Network & Sensors

Incident Replaypost-incident visualization, analysis

EBox

Mock CAD

Stream Processorschedule data collection

receive and manage data streamsstream level data processing

Incident Archiverstore incident and sensor stream

data sufficient to meaningfully recreate an incident at later date

SensorData Streams

Incident -Event DB

Progress: SAFIRE End-to-End SA System Sept. 2008 -- 1st Prototype May 2009 -- Significant redesign to add Robustness &

more dynamic configuration of components


Progress: Testing & Validation

Technology Testing Exercise: 16 SEP 08 Bren Hall Evacuation w/Campus

Police Department & UCI Zone Crew 3 Networking & sensing technologies, 1st

gen. FICB.

Live Burn Exercise: 23 FEB 09 Live Burn with OCFA,LA Fire and

Anaheim Fire Testing Sensing (human bio-sensing)

data collection & 2nd generation FICB32


Progress: Testing & Validation

Technology Testing Exercise: 12 May 09 Bren Hall Hazmat, multiple casualty

incident Integrated technology exercise, 3rd gen.

FICB.

Table Top Exercise: 15 May 09 Part of CERT Fire Forum Participants: LA County Fire, UCI

EH&S, Newport Beach Fire Department, Orange County Fire Authority, ImageCat Inc., Raytheon

33

Talk by Dr. Davison

Situational Awareness for Fire Fighters (SAFIRE)

Documents