Telesupervised Adaptive Ocean Sensor Fleet Year 1 End-of-Year
ReviewJuly 10, 2007
Carnegie Mellon UniversityNASA Goddard Space Flight Facility
NASA Wallops Flight FacilityJet Propulsion Laboratory
2
Outline
• Project and system overview (slides 2-4)
• Technical status (slides 5-35)
• Schedule, milestones, and work planned (slides 36-40)
• Critical issues (slide 41)
• Financial status (slides 42-45)
• Educational outreach and presentations (slides 46-48)
• Acronyms/glossary (slide 49)
3
Telesupervised Adaptive Ocean Sensor Fleet (TAOSF)
Objective
Key Milestones
TRLin = 4
• Improved in-situ study of Harmful Algal Blooms (HAB), coastal pollutants, oil spills, and hurricane factors
• Expanded data-gathering effectiveness and science return of existing NOAA OASIS (Ocean Atmosphere Sensor Integration System) surface vehicles
• Establishment of sensor web capability combining ocean-deployed and space sensors
• Manageable demands on scientists for tasking, control, and monitoringApproach• Telesupervision of a networked fleet of NOAA surface autonomous vehicles (OASIS)• Adaptive repositioning of sensor assets based on environmental sensor inputs (e.g., concentration gradients)• Integration of complementary established and emergent technologies (System Supervision Architecture (SSA), Inference Grids, Adaptive Sensor Fleet (ASF), Instrument Remote Control (IRC), and OASIS)• Thorough, realistic, step-by-step testing in relevant environments•Gregg Podnar / CMU•Jeffrey Hosler, John Moisan, Tiffany Moisan / GSFC
•Alberto Elfes / JPL
PI: John Dolan, CMU
Co-I’s/Partners
Artist's conception of telesupervised sensor fleet investigating a Harmful Algal Bloom.
•Interface Definition Document Feb 2007•Test components on one platform in water
May 2007•Autonomous multi-platform mapping of dye
Jul 2007•Science requirements for Inference Grid Feb 2008•Multi-platform concentration searchsimulation May 2008
•HAB search in estuary for high concentrationJul 2008
•Moving water test plan & identify locationFeb 2009
•Simulate test using in-situ and MODIS dataMay 2009
•Use MODIS data to target and reassign fleetJul 2009
4
TAOSF Program Synergy
ESTO Office
Inputs
6 PhD, MS, BS, and HS students
ESTO Office
Collaborative Partner
Telesupervised Adaptive Ocean
Sensor Fleet Project
AIST Value Added Outputs
Tools and Technology Users
GSFC
OASIS Platforms
Adaptive Sensor Fleet
Multi-Robot Telesupervision
Architecture
Planetary Exploration
HAB Detection
InferenceGrids
5
TAOSF System Overview• System Components
• System Supervision Arch. (SSA)• Adaptive Sensor Fleet (ASF)• Instrument Remote Control (IRC)• Inference Grids (IG)• Marine platforms (OASIS)
High-level planning and monitoring
High-bandwidth, single-platformtelepresence
Low-bandwidth, multi-platform telemetry
6
Technical Status
• Initial TAOSF architecture design and software integration complete (slides 6-13)
• OASIS platform development and testing (slide 14)
• Sensor validation system design and integration complete; full system test planned before end of year 1 (slides 15-25)
• Continuing Harmful Algal Bloom (HAB) dataset acquisition and analysis (slide 26-29)
• Initial end-to-end system tests (SSA-ASF-IRC-OASIS) performed using rhodamine dye as HAB surrogate in the Chesapeake Bay (slides 30-35)
7
Software Integration• Nov 2006: API for Adaptive Sensor Fleet (ASF)-OASIS
communications developed
• Dec 2006: Conducted dry test of ASF commands sent to and engineering telemetry received from OASIS
• Feb 2007: Initial integration of System Supervision Architecture (SSA) with ASF simulator and existing U.S. Navy OCU (MOCU1) preparatory to SSA-ASF-OASIS end-to-end software test
• 14 May 2007: First end-to-end SSA-ASF-OASIS dry test with SSA sending waypoint paths OASIS and receiving engineering telemetry from OASIS through ASF
• 14 Jun 2007: Second end-to-end SSA-ASF-OASIS dry test, with engineering and science telemetry received by SSA
• 15 Jun 2007: Single-platform moving-water wet test; ASF sent area-coverage paths to OASIS
• 27 Jun 2007: Single-platform moving-water wet test with rhodamine dye; ASF & SSA sent area-coverage paths to OASIS
1MOCU ( Multi-Robot Operator Control Unit) is developed by SPAWAR Systems Center San Diego (SSC-SD)
8
Software Subsystems
OASIS ASV
System (EST)
Platform Communicator
(GSFC)
Multi-Platform Simulation
Environment(GSFC)
Adaptive Sensor
Fleet(GSFC)
SystemSupervision Architecture
(CMU)
OASIS ASV
System (EST)
OASIS ASV
System (EST/WFF)
CMU: Carnegie Mellon UniversityGSFC: Goddard Space Flight CenterWFF: Wallops Flight FacilityEST: Emergent Space TechnologiesJPL: Jet Propulsion Laboratory
Adaptive Sampling /
Inference Grids (JPL)
9
Software Accomplishments
• Platform control and telemetry
– Integration of SSC-SD OCU with CMU SSA
– Design and implementation of messaging interface between CMU SSA and GSFC AFS (*)
• Data archiving and remote display
– Design and implementation of short-term and long-term archival of OASIS telemetry. (*)
– Design and implementation of web-based remote-retrieval interfaces for live and archived telemetry. (*)
(*) Ongoing
10
Software Accomplishments (cont.)
• Map creation and display; Adaptive search
– Integration and configuration of UMN Mapserver for display of map data (satellite imagery, OASIS telemetry, etc.) within OCU and web interface.
– Design and implementation of system to create maps based on telemetry.
– Design and implementation of system to use science data maps to automatically create search patterns. (*)
(*) Ongoing
11
MOCU Screenshot from Latest Test
Vehicletelemetry
OASIS platformfollowing area-coverage
trajectory assigned by ASF.
12
Map Display on OCU
Boattelemetry
MODIS chlorophyll
data
Navigationchart
MODIS sea surface temperature
data
13
Map Display on OCU
Same images available through any web browser
14
Prototype Web-Based Display
15
• 15 Nov 06: First open-ocean deployment of OASIS-2• Feb 07: OASIS-2 has barometer, fluorometer, and temperature,
humidity, and salinity sensors• Mar 07: Forward-looking camera added with web access• Apr 07: Communications – cellular modem and relatively low-cost 24/7
Iridium satellite • May 07: OASIS-1 upgraded to OASIS-2 level• Jun 07: OASIS-3 in production, expected delivery in Aug 07
OASIS at seaOASIS about to launch
OASIS Platform Development
16
Sensor Validation System
To confirm data from OASIS platforms: • Aerial camera with sensors: latitude, longitude, altitude & heading • Image the bloom and the boats
Use existing JPL software to geolocate boats and bloom.
17
Sensor Validation System Tests
• Nov 2006: Initial test at JPL with recording GPS and digital camcorder lifted on tethered weather balloon; GPS and camcorder data compared to Google Earth (GE) aerial image of the test site
• Feb 2006: Second test at JPL with extended avionics package (barometric altimeter, magnetic compass, serial data link, wide-angle monochrome camera, video transmitter); position accuracy ~3m, heading accuracy ~+/-35 degrees
• 10 May 2007: Test of rhodamine dye visibility and persistence in Panther Hollow Lake at Carnegie Mellon using avionics package
• 8 June 2007: Third test at JPL with finned aerostat and improved avionics; position accuracy <3m, heading accuracy <15 degrees
18
May07 Sensor Validation System Test
• 10 May 2007 rhodamine dye visibility and persistence test at CMU
• Panther Hollow Lake: 9900 m2 surface area, ~1 m average depth
• Two dye patches shown in figure on left: upper-right is newly sprayed with 5ppm concentration over 30 m2; lower-right was sprayed 20 min. ago with similar extent and is now less intense and covers ~65 m2
• Samples taken at intervals and different locations and analyzed at WFF using OASIS-platform fluorometers to gain visibility-concentration correlation
Panther Hollow Lake with two rhodamine dye patches and overlay metric
Rhodamine samples for analysis
Patch B
Patch A
19
Automated Dye Patch Mapping
• Aerial image data • Mapped into an RGB cube• A Markov Random Field probability map is generated• By setting a probability threshold, an extent map isolates the dye patch• A sequence of images results in a sequence of dye patch maps
20
Jun07 Sensor Validation System Test
• 8 Jun 2007 aerostat and avionics test at JPL
• Finned 20’-length x 6’-diameter aerostat for greater stability
• Instrument/avionics package: GPS, altimeter, compass, serial link, color camera, video transmitter
21
Jun07 Sensor Validation System TestV
iew
1
Vie
w
3
Vie
w
2V
iew
4
22
Jun07 Sensor Validation System Test
View 1
23
Jun07 Sensor Validation System Test
View 2
24
Jun07 Sensor Validation System Test
View 3
25
Jun07 Sensor Validation System Test
View 4
26
Jun07 Sensor Validation System Test
Mosaic combining views 1-4
27
Current HAB monitoring in Chesapeake
• Maryland DNR collects water and habitat quality data in the Chesapeake Bay and Coastal Bays at both monthly and continuous stations. A regional study uses a hydrodynamic model and satellite data to predict the abundance of Karlodinium veneficum.
28
Other HAB Monitoring
• Alexandrium fundyense has been closely studied in the Gulf of Maine, and cyst distributions have been mapped for several years by the Woods Hole Oceanographic Institute
• Karenia Brevis has been monitored extensively along the coast of Florida, and an interactive tracking system has been developed for the species by Yang Cai based on satellite imagery
29
HAB Dataset Acquisition/Analysis
• Based on ROMS1 model of the Chesapeake Bay, investigated adaptive sampling approach to optimally characterize the distribution of salinity from known temperature data
Adaptive sampling approach comparison
1ROMS (Regional Ocean Modeling System)
30
HAB Dataset Acquisition/Analysis
• Obtained historical cell count information of three HAB species in the Chesapeake and Coastal Bays from the Maryland Dept. of Natural Resources (DNR) and predicted cell counts from water quality features
• Obtained historical cell count information on Karenia Brevis from the Florida Fish and Wildlife Research Institute
Prediction algorithms comparison
31
End-of-Year1 System Test ConceptOCU
Path planning
Dye spray system
Sensor validation
OASIS platforms
32
27 Jun 2007 Single-Platform Test
John H. on shore operations
• Single OASIS platform operating in Chesapeake Bay, Pocomoke Sound Area
• Rhodamine dye placed in water by nearby chase boat
• No sensor validation system due to ~15-knot winds
• SSA running at CMU, ASF running at GSFC
• Remote tasking and telemetry acquisition and storageEllie monitoring from CMU
33
27 Jun 2007 Single-Platform Test
SSA/MOCU display of OASIS trajectory
34
LegendBlue: 0.0 voltsGreen: 0.1 voltsRed: 0.2 voltsWhite: 0.3 volts
27 Jun 2007 Single-Platform TestFluorometer measurements of rhodamine dye
Elapsed time: 20:04
Dye patch Area of higher readings Start
Finish
Trajectory is that depicted on the previous slide
442 m
173 m
Lat: 37.93, Long: 75.75
35
27 Jun 2007 Single-Platform Test
• ASF & SSA coverage planning and display• Demonstrated different coverage patterns
Spiral pattern created by ASF Raster pattern created by SSA
36
Issues
• Fluorometer measurements of rhodamine were in the 0.0-1.0V range, whereas full range is 0.0-5.6V
• Dye patch moved quickly, on order of maximum platform speed
• Connection problems between OASIS ground station and ASF prevented uninterrupted completion of pattern through the dye patch
• Winds above 10 knots prevented safe use of the sensor validation system; checking into fixed-wing flyover as backup or parallel system
27 Jun 2007 Single-Platform Test
37
Year 1 Schedule
Yr. 1 start date: Sept. 5, 2006 Yr. 1 end date: Sept. 4, 2007
38
Year 1 Milestones
• Conduct initial ground-truthing tests (at JPL) Nov 2006
• Complete/test ASF-OASIS interface Dec 2006
• Conduct interim ground-truthing tests (at JPL) Feb 2007
• Complete Interface Definition Document Feb 2007
• Test fully integrated (SSA-ASF-OASIS) software Apr 2007
• Test components on one platform in water May 2007
• Autonomous single-platform mapping of dye Jun 2007
• Autonomous multi-platform mapping of dye Jul 2007
39
Three Year Schedule
Yr. 1 start date: Sept. 5, 2006
Yr. 3 end date: Sept. 4, 2009
40
Key Project Milestones
• Interface Definition Document Feb 2007
• Autonomous multi-platform mapping of dye Jul 2007
• Multi-platform HAB search in estuary Jul 2008
• Use MODIS data to target and reassign fleet Jul 2009
41
Work Planned
• Refine system in additional single-platform wet tests by addressing issues mentioned, including use of sensor validation system
• Perform multi-platform wet test(s) before end of year (4 Sep 07)
• Write final report for year 1
• Continue to follow up contacts from Feb 2007 San Diego and Jun 2007 NSTC meetings– We have coordinated with Dan Mandl (GSFC) and now have the ability to
task EO-1 imaging in conjunction with our wet tests
– Steve Kolitz’ (Draper Labs) project has created an OASIS replanning component in his Earth Phenomena Observing System (EPOS) project; barring possible ITAR restrictions he is checking into, we will try to test this software as a module in the TAOSF system
– Explore possibility of inserting Bob Morris’ (NASA Ames) planning work into TAOSF
42
Critical Issues
• Resolution of issues identified in the 27 Jun 2007 wet test: fluorometer resolution, dye concentration, dye patch speed vs. platform speed, ASF-OASIS ground station connection drop-outs, safe use of the sensor validation system.
• The upgraded OASIS-1 platform needs to complete certification in order to be available for a multi-platform test during the summer. A summer multi-platform test will probably involve only two platforms, since OASIS-3 will not be ready before late August.
• We continue to have difficulty obtaining HAB or HAB-related datasets that would allow algorithm development and off-line testing of adaptive sampling.
• GSFC is already $5K over its Year 1 budget due to a misunderstanding, yet we need their efforts to complete Year 1 testing
43
Project Financial Status
Notes: 1. Figures reflect all actual spending through Jun 2007 except for WFF, which has not
provided a report. However, Mar-Apr 2007 include WFF estimated spending based on reported hours for those months. JPL awaits approval of $8.9K for an aerostat not reflected in the chart.
2. WFF received authorization to spend in Feb 2007 and has a planned $10K/month spending rate over Year 1.
3. GSFC had $32.1K in overhead withdrawn at Year 1’s start, but we did not understand this until after the annual review. GSFC is accordingly $5K overspent with respect to itsYear 1 total of $98K and will deduct this $5K from its Year 2 budget.
Year 1 Cost Status, All Organizations
0
100
200
300
400
500
Co
st $
K
Cum Cost Plan 35 73 111 148 186 224 261 299 337 368 399 431
Cum Cost Actual 49 66 84 107 122 151 217 255 285 305
Variance 14 -7 -26 -41 -64 -73 -45 -44 -52 -63
Sep-06 Oct-06 Nov-06 Dec-06 J an-07 Feb-07 Mar-07 Apr-07 May-07 J un-07 J ul-07 Aug-07
44
Project Financial Status
Year 1 Cost Status, GSFC
0
20
40
60
80
100
120
Cost
$K
Cum Cost Plan 8 16 25 33 41 49 57 66 74 82 90 98
Cum Cost Actual 32 32 32 38 41 48 69 77 92 103
Variance 24 16 8 5 0 -1 12 12 19 21
Sep-06 Oct-06 Nov-06 Dec-06 J an-07 Feb-07 Mar-07 Apr-07 May-07 J un-07 J ul-07 Aug-07
Year 1 Cost Status, CMU
0
50
100
150
200
Co
st $
K
Cum Cost Plan 17 34 50 67 84 101 118 134 151 162 172 182
Cum Cost Actual 17 34 51 68 77 100 114 129 144
Variance 0 1 1 1 -7 -1 -3 -6 -8
Sep-06 Oct-06 Nov-06 Dec-06 J an-07 Feb-07 Mar-07 Apr-07 May-07 J un-07 J ul-07 Aug-07
45
Project Financial Status
Year 1 Cost Status, JPL
0
10
20
30
Cost
$K
Cum Cost Plan 0 3 5 8 10 13 16 18 21 23 26 29
Cum Cost Actual 0 0 1 1 3 3 3 4 4 5
Variance 0 -3 -4 -7 -7 -10 -13 -15 -17 -18
Sep-06 Oct-06 Nov-06 Dec-06 Jan-07 Feb-07 Mar-07 Apr-07 May-07 Jun-07 Jul-07 Aug-07
Year 1 Cost Status, WFF
0
20
40
60
80
100
120
140
Co
st $
K
Cum Cost Plan 10 20 30 40 51 61 71 81 91 101 112 122
Cum Cost Actual 0 0 0 0 0 0 30 45
Variance -10 -20 -30 -40 -51 -61 -41 -36
Sep-06 Oct-06 Nov-06 Dec-06 J an-07 Feb-07 Mar-07 Apr-07 May-07 J un-07 J ul-07 Aug-07
46
Project Financial Status
Full Project Cost Status, All Organizations
0
200
400
600
800
1,000
1,200
1,400
1,600
Cos
t $
K
Cum Cost Plan 111 224 337 431 546 661 777 892 1009 1125 1241 1357
Cum Cost Actual 84 151 285
Variance -27 -73 -52
Nov-06 Feb-07 May-07 Aug-07 Nov-07 Feb-08 May-08 Aug-08 Nov-08 Feb-09 May-09 Aug-09
47
Educational Outreach
• Steve Stancliff– Ph.D. student,
Robotics– Carnegie
Mellon University
• Ellie Lin– Ph.D. student,
Robotics– Carnegie Mellon
University
• Jeff Baker– B.S. student,
Computer Science– Duquesne University
• Sandra Mau– M.S. student, Robotics– Carnegie Mellon
University– Graduated May 2007
48
Educational Outreach
• David Schlesinger– Student at Mt.
Lebanon High School
– 2006, 2007 Summer Junior Programmer
• Matt Felser– Student at Mt.
Lebanon High School
– 2007 Summer Junior Programmer
49
Presentations Paper entitled “Harmful Algal Bloom Characterization Via the Telesupervised Adaptive Ocean
Sensor Fleet” presented at the 19-21 June 2007 NASA Science & Technology Conference in Adelphi, MD
Abstract entitled “The Telesupervised Adaptive Ocean Sensor Fleet” accepted for presentation in the “Atmospheric and Environmental Remote Sensing Data Processing and Utilization: Perspective On Preparing For GEOSS” segment of the 26-30 August 2007 SPIE Optics and Photonics Conference in San Diego, CA
Abstract entitled “Human Telesupervision of Very Heterogeneous Planetary Robot Teams” accepted for presentation in the “Human and Robotic Exploration” segment of the 18-20 September 2007 AIAA Space 2007 Conference in Long Beach, CA
Paper entitled “Scheduling for Humans in Multirobot Supervisory Control” accepted for presentation at the 29 October-2 November 2007 IEEE International Conference on Intelligent Robots and Systems in San Diego, CA
Abstract submitted to the “Integrated Sensing, Modeling, and Analysis Using Sensor Webs” session of the 1-8 March 2008 IEEE Aerospace Conference in Big Sky, Montana
50
Acronyms/Glossary
• API – Application Program Interface• ASF – Adaptive Sensor Fleet• CMU – Carnegie Mellon University• Delmarva – Delaware/Maryland/Virginia• EST – Emergent Space Technologies• GSFC – Goddard Space Flight Center• HAB – Harmful Algal Bloom• IG – Inference Grids• IRC – Instrument Remote Control• JPL – Jet Propulsion Laboratory• MOCU – Multi-Robot Operator Control Unit• MODIS – Moderate-Resolution Imaging Spectrometer • MySQL – My Structured Query Language, a popular database management system• NOAA – National Oceanic and Atmospheric Administration• OASIS – Ocean Atmosphere Sensor Integration System• OCU – Operator Control Unit• Rhodamine WT – A non-toxic liquid red dye commonly used in water-tracing studies• ROMS – Regional Ocean Modeling System• SPAWAR – Space and Naval Warfare Systems• SSA – System Supervision Architecture• SSC-SD – SPAWAR Systems Center – San Diego• TAOSF – Telesupervised Adaptive Ocean Sensor Fleet• UMN – University of Minnesota• WFF – Wallops Flight Facility