Images and Sounds: Audio and Video for Education Joe Wise and Michael Hamilton.

Images and Sounds: Audio and Video for Education

Joe Wise and Michael Hamilton

Images and Sounds:Applications to Education

Instrumentation Platform

Image data Acoustic dataSpatial and Temporal

Coverage

Mobile NodesNetworked

Infomechanical Systems (NIMS)

•Real-time and archived data viewing of time series •Image classification and object recognition (supervised and unsupervised)

•None at this time

Both Sensing and Sampling in 3-D using programmed location or virtual control

Connected Fixed Nodes

Tower CamsMini-rhizotron

Nest BoxesAcoustic Tower

•Real-time and archived data viewing •image classification and object recognition (supervised and unsupervised)

•Real-time listening and recording•(towers and arrays)•Sound classification (supervised and unsupervised signal processing)

•Programmed field of view with pan/tilt/zoom (towers) and X-Y coordinates (AMR)•Fixed field of view (nest boxes, moss cam)•Sound source localization with actuated imaging

Untethered Fixed Nodes

CyclopsAcoustic Array

•Near real-time and archived data viewing•image classification and object recognition (supervised and unsupervised)

•Sound classification (supervised and unsupervised signal processing) •Sound source localization

Fixed field of view

Networked Infomechanical Systems (NIMS) Mobile Nodes

Tower Cameras & Moss camConnected Fixed Nodes

•Pan-tilt-zoom cameras, some with near IR.

• Microclimate, CO2, and soil water potential sensors.

• Remote and pre-programmed automatic operation for time-lapse recording

• Internet access to real-time (5-minute collection interval) and archived data and images.

Root growth at the James Reserve

Automated Minirhizotron and and Rhizosphere Sensor System (AMARSS)

Connected Fixed Nodes

Correlating microclimate variability with reproductive success in cavity-nesting birds

Nest boxes with cameras and microclimateConnected Fixed Nodes

Nest boxes with cameras and microclimateConnected Fixed Nodes

Wireless Networked Acoustic NodeConnected Fixed Node

Click twice to launch Quicktime VRClick twice to launch Quicktime VR

Wireless Networked Acoustic Node Connected Fixed Node

Automated Image Capture and panorama stitcherAutomated Image Capture and panorama stitcher

Wireless Networked Video and Acoustic Node Connected Fixed Node

Real-time Viewer and Camera Control

Streaming Audio Server from all-weather Stereo microphone

Real-time Viewer and Camera Control

Streaming Audio Server from all-weather Stereo microphone

iDCT: Interactive Data Classification Tooladding value to data

Classification questions determined by the scientists

Participants sign in and receive random audio/video/image files. They answer guiding questions that are designed to attach searchable data to the file. The more times a file gets reviewed, the more reliable the data associated with it.

Image, audio, and video files present difficulties for computer classification.Image, audio, and video files present difficulties for computer classification.

recognize patterns recognize anomalies recognize identification methods understand content recognize variables recognize correlations do science

iDCT: Interactive Data Classification ToolEducational Possibilities

Students using curricular tools designed around these data sets are able to:Students using curricular tools designed around these data sets are able to: