Telesupervised Adaptive Ocean Sensor Fleet Year 1 End-of-Year Review

Telesupervised Adaptive Ocean Sensor Fleet Year 1 End-of-Year

ReviewJuly 10, 2007

Carnegie Mellon UniversityNASA Goddard Space Flight Facility

NASA Wallops Flight FacilityJet Propulsion Laboratory

2

Outline

• Project and system overview (slides 2-4)

• Technical status (slides 5-35)

• Schedule, milestones, and work planned (slides 36-40)

• Critical issues (slide 41)

• Financial status (slides 42-45)

• Educational outreach and presentations (slides 46-48)

• Acronyms/glossary (slide 49)

3

Telesupervised Adaptive Ocean Sensor Fleet (TAOSF)

Objective

Key Milestones

TRLin = 4

• Improved in-situ study of Harmful Algal Blooms (HAB), coastal pollutants, oil spills, and hurricane factors

• Expanded data-gathering effectiveness and science return of existing NOAA OASIS (Ocean Atmosphere Sensor Integration System) surface vehicles

• Establishment of sensor web capability combining ocean-deployed and space sensors

• Manageable demands on scientists for tasking, control, and monitoringApproach• Telesupervision of a networked fleet of NOAA surface autonomous vehicles (OASIS)• Adaptive repositioning of sensor assets based on environmental sensor inputs (e.g., concentration gradients)• Integration of complementary established and emergent technologies (System Supervision Architecture (SSA), Inference Grids, Adaptive Sensor Fleet (ASF), Instrument Remote Control (IRC), and OASIS)• Thorough, realistic, step-by-step testing in relevant environments•Gregg Podnar / CMU•Jeffrey Hosler, John Moisan, Tiffany Moisan / GSFC

•Alberto Elfes / JPL

PI: John Dolan, CMU

Co-I’s/Partners

Artist's conception of telesupervised sensor fleet investigating a Harmful Algal Bloom.

•Interface Definition Document Feb 2007•Test components on one platform in water

May 2007•Autonomous multi-platform mapping of dye

Jul 2007•Science requirements for Inference Grid Feb 2008•Multi-platform concentration searchsimulation May 2008

•HAB search in estuary for high concentrationJul 2008

•Moving water test plan & identify locationFeb 2009

•Simulate test using in-situ and MODIS dataMay 2009

•Use MODIS data to target and reassign fleetJul 2009

4

TAOSF Program Synergy

ESTO Office

Inputs

6 PhD, MS, BS, and HS students

ESTO Office

Collaborative Partner

Telesupervised Adaptive Ocean

Sensor Fleet Project

AIST Value Added Outputs

Tools and Technology Users

GSFC

OASIS Platforms

Adaptive Sensor Fleet

Multi-Robot Telesupervision

Architecture

Planetary Exploration

HAB Detection

InferenceGrids

5

TAOSF System Overview• System Components

• System Supervision Arch. (SSA)• Adaptive Sensor Fleet (ASF)• Instrument Remote Control (IRC)• Inference Grids (IG)• Marine platforms (OASIS)

High-level planning and monitoring

High-bandwidth, single-platformtelepresence

Low-bandwidth, multi-platform telemetry

6

Technical Status

• Initial TAOSF architecture design and software integration complete (slides 6-13)

• OASIS platform development and testing (slide 14)

• Sensor validation system design and integration complete; full system test planned before end of year 1 (slides 15-25)

• Continuing Harmful Algal Bloom (HAB) dataset acquisition and analysis (slide 26-29)

• Initial end-to-end system tests (SSA-ASF-IRC-OASIS) performed using rhodamine dye as HAB surrogate in the Chesapeake Bay (slides 30-35)

7

Software Integration• Nov 2006: API for Adaptive Sensor Fleet (ASF)-OASIS

communications developed

• Dec 2006: Conducted dry test of ASF commands sent to and engineering telemetry received from OASIS

• Feb 2007: Initial integration of System Supervision Architecture (SSA) with ASF simulator and existing U.S. Navy OCU (MOCU1) preparatory to SSA-ASF-OASIS end-to-end software test

• 14 May 2007: First end-to-end SSA-ASF-OASIS dry test with SSA sending waypoint paths OASIS and receiving engineering telemetry from OASIS through ASF

• 14 Jun 2007: Second end-to-end SSA-ASF-OASIS dry test, with engineering and science telemetry received by SSA

• 15 Jun 2007: Single-platform moving-water wet test; ASF sent area-coverage paths to OASIS

• 27 Jun 2007: Single-platform moving-water wet test with rhodamine dye; ASF & SSA sent area-coverage paths to OASIS

1MOCU ( Multi-Robot Operator Control Unit) is developed by SPAWAR Systems Center San Diego (SSC-SD)

8

Software Subsystems

OASIS ASV

System (EST)

Platform Communicator

(GSFC)

Multi-Platform Simulation

Environment(GSFC)

Adaptive Sensor

Fleet(GSFC)

SystemSupervision Architecture

(CMU)

OASIS ASV

System (EST)

OASIS ASV

System (EST/WFF)

CMU: Carnegie Mellon UniversityGSFC: Goddard Space Flight CenterWFF: Wallops Flight FacilityEST: Emergent Space TechnologiesJPL: Jet Propulsion Laboratory

Adaptive Sampling /

Inference Grids (JPL)

9

Software Accomplishments

• Platform control and telemetry

– Integration of SSC-SD OCU with CMU SSA

– Design and implementation of messaging interface between CMU SSA and GSFC AFS (*)

• Data archiving and remote display

– Design and implementation of short-term and long-term archival of OASIS telemetry. (*)

– Design and implementation of web-based remote-retrieval interfaces for live and archived telemetry. (*)

(*) Ongoing

10

Software Accomplishments (cont.)

• Map creation and display; Adaptive search

– Integration and configuration of UMN Mapserver for display of map data (satellite imagery, OASIS telemetry, etc.) within OCU and web interface.

– Design and implementation of system to create maps based on telemetry.

– Design and implementation of system to use science data maps to automatically create search patterns. (*)

(*) Ongoing

11

MOCU Screenshot from Latest Test

Vehicletelemetry

OASIS platformfollowing area-coverage

trajectory assigned by ASF.

12

Map Display on OCU

Boattelemetry

MODIS chlorophyll

data

Navigationchart

MODIS sea surface temperature

data

13

Map Display on OCU

Same images available through any web browser

14

Prototype Web-Based Display

15

• 15 Nov 06: First open-ocean deployment of OASIS-2• Feb 07: OASIS-2 has barometer, fluorometer, and temperature,

humidity, and salinity sensors• Mar 07: Forward-looking camera added with web access• Apr 07: Communications – cellular modem and relatively low-cost 24/7

Iridium satellite • May 07: OASIS-1 upgraded to OASIS-2 level• Jun 07: OASIS-3 in production, expected delivery in Aug 07

OASIS at seaOASIS about to launch

OASIS Platform Development

16

Sensor Validation System

To confirm data from OASIS platforms: • Aerial camera with sensors: latitude, longitude, altitude & heading • Image the bloom and the boats

Use existing JPL software to geolocate boats and bloom.

17

Sensor Validation System Tests

• Nov 2006: Initial test at JPL with recording GPS and digital camcorder lifted on tethered weather balloon; GPS and camcorder data compared to Google Earth (GE) aerial image of the test site

• Feb 2006: Second test at JPL with extended avionics package (barometric altimeter, magnetic compass, serial data link, wide-angle monochrome camera, video transmitter); position accuracy ~3m, heading accuracy ~+/-35 degrees

• 10 May 2007: Test of rhodamine dye visibility and persistence in Panther Hollow Lake at Carnegie Mellon using avionics package

• 8 June 2007: Third test at JPL with finned aerostat and improved avionics; position accuracy <3m, heading accuracy <15 degrees

18

May07 Sensor Validation System Test

• 10 May 2007 rhodamine dye visibility and persistence test at CMU

• Panther Hollow Lake: 9900 m2 surface area, ~1 m average depth

• Two dye patches shown in figure on left: upper-right is newly sprayed with 5ppm concentration over 30 m2; lower-right was sprayed 20 min. ago with similar extent and is now less intense and covers ~65 m2

• Samples taken at intervals and different locations and analyzed at WFF using OASIS-platform fluorometers to gain visibility-concentration correlation

Panther Hollow Lake with two rhodamine dye patches and overlay metric

Rhodamine samples for analysis

Patch B

Patch A

19

Automated Dye Patch Mapping

• Aerial image data • Mapped into an RGB cube• A Markov Random Field probability map is generated• By setting a probability threshold, an extent map isolates the dye patch• A sequence of images results in a sequence of dye patch maps

20

Jun07 Sensor Validation System Test

• 8 Jun 2007 aerostat and avionics test at JPL

• Finned 20’-length x 6’-diameter aerostat for greater stability

• Instrument/avionics package: GPS, altimeter, compass, serial link, color camera, video transmitter

21

Jun07 Sensor Validation System TestV

iew

1

Vie

w

3

Vie

w

2V

iew

4

22


View 1

23


View 2

24


View 3

25


View 4

26


Mosaic combining views 1-4

27

Current HAB monitoring in Chesapeake

• Maryland DNR collects water and habitat quality data in the Chesapeake Bay and Coastal Bays at both monthly and continuous stations. A regional study uses a hydrodynamic model and satellite data to predict the abundance of Karlodinium veneficum.

28

Other HAB Monitoring

• Alexandrium fundyense has been closely studied in the Gulf of Maine, and cyst distributions have been mapped for several years by the Woods Hole Oceanographic Institute

• Karenia Brevis has been monitored extensively along the coast of Florida, and an interactive tracking system has been developed for the species by Yang Cai based on satellite imagery

29

HAB Dataset Acquisition/Analysis

• Based on ROMS1 model of the Chesapeake Bay, investigated adaptive sampling approach to optimally characterize the distribution of salinity from known temperature data

Adaptive sampling approach comparison

1ROMS (Regional Ocean Modeling System)

30

HAB Dataset Acquisition/Analysis

• Obtained historical cell count information of three HAB species in the Chesapeake and Coastal Bays from the Maryland Dept. of Natural Resources (DNR) and predicted cell counts from water quality features

• Obtained historical cell count information on Karenia Brevis from the Florida Fish and Wildlife Research Institute

Prediction algorithms comparison

31

End-of-Year1 System Test ConceptOCU

Path planning

Dye spray system

Sensor validation

OASIS platforms

32

27 Jun 2007 Single-Platform Test

John H. on shore operations

• Single OASIS platform operating in Chesapeake Bay, Pocomoke Sound Area

• Rhodamine dye placed in water by nearby chase boat

• No sensor validation system due to ~15-knot winds

• SSA running at CMU, ASF running at GSFC

• Remote tasking and telemetry acquisition and storageEllie monitoring from CMU

33


SSA/MOCU display of OASIS trajectory

34

LegendBlue: 0.0 voltsGreen: 0.1 voltsRed: 0.2 voltsWhite: 0.3 volts

27 Jun 2007 Single-Platform TestFluorometer measurements of rhodamine dye

Elapsed time: 20:04

Dye patch Area of higher readings Start

Finish

Trajectory is that depicted on the previous slide

442 m

173 m

Lat: 37.93, Long: 75.75

35


• ASF & SSA coverage planning and display• Demonstrated different coverage patterns

Spiral pattern created by ASF Raster pattern created by SSA

36

Issues

• Fluorometer measurements of rhodamine were in the 0.0-1.0V range, whereas full range is 0.0-5.6V

• Dye patch moved quickly, on order of maximum platform speed

• Connection problems between OASIS ground station and ASF prevented uninterrupted completion of pattern through the dye patch

• Winds above 10 knots prevented safe use of the sensor validation system; checking into fixed-wing flyover as backup or parallel system


37

Year 1 Schedule

Yr. 1 start date: Sept. 5, 2006 Yr. 1 end date: Sept. 4, 2007

38

Year 1 Milestones

• Conduct initial ground-truthing tests (at JPL) Nov 2006

• Complete/test ASF-OASIS interface Dec 2006

• Conduct interim ground-truthing tests (at JPL) Feb 2007

• Complete Interface Definition Document Feb 2007

• Test fully integrated (SSA-ASF-OASIS) software Apr 2007

• Test components on one platform in water May 2007

• Autonomous single-platform mapping of dye Jun 2007

• Autonomous multi-platform mapping of dye Jul 2007

39

Three Year Schedule

Yr. 1 start date: Sept. 5, 2006

Yr. 3 end date: Sept. 4, 2009

40

Key Project Milestones

• Interface Definition Document Feb 2007

• Autonomous multi-platform mapping of dye Jul 2007

• Multi-platform HAB search in estuary Jul 2008

• Use MODIS data to target and reassign fleet Jul 2009

41

Work Planned

• Refine system in additional single-platform wet tests by addressing issues mentioned, including use of sensor validation system

• Perform multi-platform wet test(s) before end of year (4 Sep 07)

• Write final report for year 1

• Continue to follow up contacts from Feb 2007 San Diego and Jun 2007 NSTC meetings– We have coordinated with Dan Mandl (GSFC) and now have the ability to

task EO-1 imaging in conjunction with our wet tests

– Steve Kolitz’ (Draper Labs) project has created an OASIS replanning component in his Earth Phenomena Observing System (EPOS) project; barring possible ITAR restrictions he is checking into, we will try to test this software as a module in the TAOSF system

– Explore possibility of inserting Bob Morris’ (NASA Ames) planning work into TAOSF

42

Critical Issues

• Resolution of issues identified in the 27 Jun 2007 wet test: fluorometer resolution, dye concentration, dye patch speed vs. platform speed, ASF-OASIS ground station connection drop-outs, safe use of the sensor validation system.

• The upgraded OASIS-1 platform needs to complete certification in order to be available for a multi-platform test during the summer. A summer multi-platform test will probably involve only two platforms, since OASIS-3 will not be ready before late August.

• We continue to have difficulty obtaining HAB or HAB-related datasets that would allow algorithm development and off-line testing of adaptive sampling.

• GSFC is already $5K over its Year 1 budget due to a misunderstanding, yet we need their efforts to complete Year 1 testing

43

Project Financial Status

Notes: 1. Figures reflect all actual spending through Jun 2007 except for WFF, which has not

provided a report. However, Mar-Apr 2007 include WFF estimated spending based on reported hours for those months. JPL awaits approval of $8.9K for an aerostat not reflected in the chart.

2. WFF received authorization to spend in Feb 2007 and has a planned $10K/month spending rate over Year 1.

3. GSFC had $32.1K in overhead withdrawn at Year 1’s start, but we did not understand this until after the annual review. GSFC is accordingly $5K overspent with respect to itsYear 1 total of $98K and will deduct this $5K from its Year 2 budget.

Year 1 Cost Status, All Organizations

0

100

200

300

400

500

Co

st $

K

Cum Cost Plan 35 73 111 148 186 224 261 299 337 368 399 431

Cum Cost Actual 49 66 84 107 122 151 217 255 285 305

Variance 14 -7 -26 -41 -64 -73 -45 -44 -52 -63

Sep-06 Oct-06 Nov-06 Dec-06 J an-07 Feb-07 Mar-07 Apr-07 May-07 J un-07 J ul-07 Aug-07

44


Year 1 Cost Status, GSFC

0

20

40

60

80

100

120

Cost

$K

Cum Cost Plan 8 16 25 33 41 49 57 66 74 82 90 98

Cum Cost Actual 32 32 32 38 41 48 69 77 92 103

Variance 24 16 8 5 0 -1 12 12 19 21


Year 1 Cost Status, CMU

0

50

100

150

200

Co

st $

K

Cum Cost Plan 17 34 50 67 84 101 118 134 151 162 172 182

Cum Cost Actual 17 34 51 68 77 100 114 129 144

Variance 0 1 1 1 -7 -1 -3 -6 -8


45


Year 1 Cost Status, JPL

0

10

20

30

Cost

$K

Cum Cost Plan 0 3 5 8 10 13 16 18 21 23 26 29

Cum Cost Actual 0 0 1 1 3 3 3 4 4 5

Variance 0 -3 -4 -7 -7 -10 -13 -15 -17 -18

Sep-06 Oct-06 Nov-06 Dec-06 Jan-07 Feb-07 Mar-07 Apr-07 May-07 Jun-07 Jul-07 Aug-07

Year 1 Cost Status, WFF

0

20

40

60

80

100

120

140

Co

st $

K

Cum Cost Plan 10 20 30 40 51 61 71 81 91 101 112 122

Cum Cost Actual 0 0 0 0 0 0 30 45

Variance -10 -20 -30 -40 -51 -61 -41 -36


46


Full Project Cost Status, All Organizations

0

200

400

600

800

1,000

1,200

1,400

1,600

Cos

t $

K

Cum Cost Plan 111 224 337 431 546 661 777 892 1009 1125 1241 1357

Cum Cost Actual 84 151 285

Variance -27 -73 -52

Nov-06 Feb-07 May-07 Aug-07 Nov-07 Feb-08 May-08 Aug-08 Nov-08 Feb-09 May-09 Aug-09

47

Educational Outreach

• Steve Stancliff– Ph.D. student,

Robotics– Carnegie

Mellon University

• Ellie Lin– Ph.D. student,

Robotics– Carnegie Mellon

University

• Jeff Baker– B.S. student,

Computer Science– Duquesne University

• Sandra Mau– M.S. student, Robotics– Carnegie Mellon

University– Graduated May 2007

48

Educational Outreach

• David Schlesinger– Student at Mt.

Lebanon High School

– 2006, 2007 Summer Junior Programmer

• Matt Felser– Student at Mt.

Lebanon High School

– 2007 Summer Junior Programmer

49

Presentations Paper entitled “Harmful Algal Bloom Characterization Via the Telesupervised Adaptive Ocean

Sensor Fleet” presented at the 19-21 June 2007 NASA Science & Technology Conference in Adelphi, MD

Abstract entitled “The Telesupervised Adaptive Ocean Sensor Fleet” accepted for presentation in the “Atmospheric and Environmental Remote Sensing Data Processing and Utilization: Perspective On Preparing For GEOSS” segment of the 26-30 August 2007 SPIE Optics and Photonics Conference in San Diego, CA

Abstract entitled “Human Telesupervision of Very Heterogeneous Planetary Robot Teams” accepted for presentation in the “Human and Robotic Exploration” segment of the 18-20 September 2007 AIAA Space 2007 Conference in Long Beach, CA

Paper entitled “Scheduling for Humans in Multirobot Supervisory Control” accepted for presentation at the 29 October-2 November 2007 IEEE International Conference on Intelligent Robots and Systems in San Diego, CA

Abstract submitted to the “Integrated Sensing, Modeling, and Analysis Using Sensor Webs” session of the 1-8 March 2008 IEEE Aerospace Conference in Big Sky, Montana

50

Acronyms/Glossary

• API – Application Program Interface• ASF – Adaptive Sensor Fleet• CMU – Carnegie Mellon University• Delmarva – Delaware/Maryland/Virginia• EST – Emergent Space Technologies• GSFC – Goddard Space Flight Center• HAB – Harmful Algal Bloom• IG – Inference Grids• IRC – Instrument Remote Control• JPL – Jet Propulsion Laboratory• MOCU – Multi-Robot Operator Control Unit• MODIS – Moderate-Resolution Imaging Spectrometer • MySQL – My Structured Query Language, a popular database management system• NOAA – National Oceanic and Atmospheric Administration• OASIS – Ocean Atmosphere Sensor Integration System• OCU – Operator Control Unit• Rhodamine WT – A non-toxic liquid red dye commonly used in water-tracing studies• ROMS – Regional Ocean Modeling System• SPAWAR – Space and Naval Warfare Systems• SSA – System Supervision Architecture• SSC-SD – SPAWAR Systems Center – San Diego• TAOSF – Telesupervised Adaptive Ocean Sensor Fleet• UMN – University of Minnesota• WFF – Wallops Flight Facility

Telesupervised Adaptive Ocean Sensor Fleet Year 1 End-of-Year Review

Documents

system overview slides

environmental sensor

planned slides

oasis platform development

technical status slides

financial status slides

chesapeake bay slides

test components