NIDAS NCAR In-Situ Data Acquisition Software Gordon Maclean Software Engineer NCAR/EOL
Dec 29, 2015
NIDASNCAR In-Situ Data Acquisition Software
Gordon MacleanSoftware EngineerNCAR/EOL
Background
Developed at the NCAR Earth Observing Laboratory (EOL), starting in 2004.
Targeted as the primary data system for the NSF HIAPER, NCAR's Gulfstream V research aircraft.
Developers included software engineers with experience at EOL's Research Aviation Facility (RAF) and In-Situ Sensing Facility (ISF)
Design was expanded to include the acquisition of ground-based meteorological measurements.
Built on the past experience of EOL engineers:
Aircraft Data System: ( ADS, ADS-2), 1970s – 2009
Surface systems: Portable Automated Mesonet (PAM I,II,III), 1973-2004 and the Atmosphere-Surface Turbulence Exchange Facility (ASTER), 1988-2002
Targeted ResearchDiverse topics, primarily NSF funded
Aircraft
Boundary layer to 50K ft
Cloud physics, convection, severe weather, chemistry and pollution, remote sensing, sensor validation
20 to 200 flight hours per research project
Surface
PBL surface boundary conditions, energy balance, turbulence
Fluxes of momentum, heat, moisture and trace gases
Horizontal and vertical (0-50 meter) gradients
24x7 operations, often unattended
Short term, typically 1 to 3 months per project
Navigation GPS, inertial reference systems, radar altimeter
State parameters Pressure, temperature, humidity, dew point (various)
Winds Pitot tubesLaser Air Motion Sensor: LAMS (under development)
Laser absorption hygrometers Vertical Cavity Surface Emitting Laser Hygrometer: VCSELNCAR TDL Water Vapor
Cloud particles, aerosols 2D particle probes: PMS 2D-C, 2D-P, Fast2DCCloud Droplet probeForward Scattering Spectrometer Probes: FSSP 100,200,300Ultra-high Sensitivity Aerosol Spectrometer: UHSASCondensation Nuclei countersPMS King Hot Wire Liquid Water ProbeCloud Particle Imager: 3V-CPI
Ice detectors Rosemount Icing DetectorSmall Ice Detector: SID-II
Inlets Counter-Flow Virtual Impactor: CVIHIML
Chemistry Chemiluminescense: NCAR CO, O3
Gas Chromatographs: Panther, UCATS: H2O, O
3, N
2O,CH
4
Soot photometer: DMT SP2Picarro: CO
2,Methane,O18,H
2O,Deuterium
Radiation Pyranometers, pyrgeometers, UV, IR surface temp
Temperature profiles Microwave Temperature Profiler: MTP*
Engineering Accelerometers, CMIGITSAir Pressure Scanner: PSI-9116
Airborne Instrumentation
Aircraft Deployments
Surface InstrumentationState parameters Barometer, T, RH: (various)Time, location GPS with pulse-per-second2d anemometers RMYoung Propvane, 85004, Gill WindSonic, WindObserver,
MetOne, Vaisala WMT3d anemometers Campbell CSAT3
Applied Technologies: ATIKWater Vapor, CO2Other trace gases
Campbell Krypton HygrometerLI-COR 7000, 7500, Campbell EC150
Radiation Pyranometers, Pyrgeometers: K&Z, Eppley, Hukseflux NR01Net: Rebs Q7, Hukseflux NR02, KZ NRLite2Solar and Diffuse: SPN1, LI-COR, ApogeeSurface Temp: EverestPhotosynthetic Active Radiation: LI-COR 190SA
Soil Heat flux, temp, moisture, thermal prop: REBS, Decagon, Hukesflux
Mesh sensor networks Digi Zigbee, EOL WisardWeather stations, loggers Vaisala WXT, CR10
Precipitation ETI, MRI rain gauges, Decagon Leaf WetnessFast pressure Paroscientific 216BHot film anemometers Dantek 1D/3DEngineering Power monitor, battery and charging system status
Surface Deployments
Required Data System Interfaces
Type ImplementationRS 232/422/485 serial Diamond Systems 8 port (with digio)
WinSystems PCM-COM8
Analog to Digital, 16 bit NCAR 8 channel, differential, sigma-deltaDiamond MM-16, MM-32: 16 channel diff, multiplexed, SAR
USB 1,2 Processor motherboardEthernet Processor motherboardPulse counters, frequency Mesa Anything I/O FPGA
Diamond GPIO-MMDigital I/O, analog out, relays Arcom Viper CPU
Diamond MM, IR104ARINC 429 navigation Condor Engineering CEI-420AIRIG-B timing protocol Brandywine PC104-SGBluetooth RFCOMM serial USB/Bluetooth adaptersMesh Networks, sensor arrays Serial interface to EOL Wisard base
motes
Initial Design Choices Operating System: Linux
Open source, near real-time, wide choice of system vendor Same OS on embedded systems and processing and display servers Extensive network services Multithreaded, dynamic (run-time) object loading
Data Sampling Modules: ARM Processors and PC104 expansion bus Low power (~2 Watt) low heat generation Extensive vendor support for PC104 interface cards
Languages: C++: object oriented, compiled C: low level interface driver modules
Configuration: XML: flexible, tree structured, widely used, field editable (by NIDAS geek)
Software Tools: Version control: subversion Builds: scons Packaging: RPM, Debian (embedded)
Server Systems
Data Sampling Modules, DSMs
Networks
Low level driver modulesncar_a2d serial network
Processors
StatisticsNetCDF
networkfiles
Processing Pipeline
Sorter
SensorObjects
Sorter
NIDAS
Calibrationdata
Time-tagged, raw Time-tagged, converted
SensorHandlerSensor Objects
InstrumentationData System Hardware Interfaces PC104 A2D card Serial Port Ethernet
NIDAS Sample Flow
Sensor Object Hierarchy
Highlights Supports diverse instrumentation
Wide range of sampling rates: ~0 to ~1000 samples/sec
Highly configurable, distributed processing
Runnable on low power (~2 Watt) embedded processors
Networked, supporting remote login, serial sensor interaction, power control
Open source, including all low level driver code
Acquired data is archived in raw sensor format
Archived time tags have microsecond precision
System clocks have ~20 microsecond accuracy
Many deployments
Revision controlled, automated tests
GUI configuration editor for aircraft deployments
Future Plans, More Info• Kernel upgrades on DSMs, using newer real-time capabilities of Linux.
• Reduce serial sampling latency (line discipline kernel module)
• Improve network service discovery mechanism
• Enhancements to real-time displays, diagnostics
• Framework for instrumentation control
• Calibration database
• Other server platforms: MacOSX
• Other embedded hardware platforms: Eurotech Titan, gumstix, new ARM/Intel processors, pc104+
• Support and collaboration with outside users.
http://www.eol.ucar.edu/data/software/nidas (under development)
Questions/Answers