Engineering Node Report MC Face-to-Face Washington, DC March 25-26, 2010
Feb 24, 2016
Engineering Node Report
MC Face-to-FaceWashington, DC
March 25-26, 2010
Engineering Node Organization
2
System Engineering
Steve Hughes
Operations
Emily Law
Development
Sean Hardman
Architecture Development and Maintenance
Technology Research and Demonstrations
Standards Development and Maintenance
Information and System Modeling
PDS System Engineering
Software systems engineering
Database development and engineering
Tools development and sustaining
Infrastructure development and sustaining
Web Applications Development
Configuration management
Operations Deliveries
EN Management
Dan Crichton
PDS Management
Operations Engineering
Quality Assurance (Data and Software)
Database Operations
System Operations
Data Engineering Support and Guidance
Software and Data Releases
CCB, Issue Tracking, etc
Website Maintenance
Metrics collection and reporting
Operations coordination across the PDS
PDS Help Desk
FTE Support: 6.25
Staffing and Resource Changes for PDS 2010
• Standards, Development and Operations are being transitioned over to PDS 2010 efforts• Standard development primarily focused on PDS4• Software development primary focused on design
and implementation for PDS 2010• Operations transitioning over; new servers now in
place
3
Data Engineering Mission Assignments
4
Missions in Operations EN Data Engineer
Mars & Lunar Missions (ODY, M3, LRO, MER1, MER2, MEX, MRO, PHX)
Betty Sword
Others (CAS, Dawn, EPOXI, Hayabusa, MESSENGER, New Horizons, Stardust-Next, VEX)
Richard Chen
Missions in Operations EN Data Engineer
GRAIL, LADEE, LCROSS, MSL, MAVEN Betty Sword
Juno, Kepler Richard Chen
Development Staff Projects
• PDS3 Development• Tool Maintenance (M. Cayanan)
• Integrating Validation Tool with the Product Tools Library.• Web Site Maintenance (J. Padams)• Delivery Manifest Tool (J. Padams)
• PDS 2010 Development• Security Service (H. Lee)
• Deployment and integration of the selected open source package.
• Registry Service (P. Ramirez)• Design and development of the service.
5
Tool PlansFunction Current Effort (PDS3) PDS 2010 PlanDesign (Metadata)
Released LTDTool in Sep 2008.
Investigate Open Source and COTS tools.
Generation (Metadata)
No current effort. Investigate current capabilities and develop a generic solution.
Validation (Metadata and Data)
Continued maintenance of VTool.
Redesign and implement for XML.
Ingestion (Metadata)
Integrate CITool with latest Product Tools Library.
The Harvest and Ingest services will replace this functionality.
Transformation (Metadata and Data)
No current effort. Design and implement an extensible solution.
Visualization(Data)
Continued maintenance of NASAView.
Design and implement an extensible solution.
6
7
Recent Tool Activities
• Product Tools Library• Released version 3.0.2 in March 2010.• Supports VTool and the Volume Validator.
• Validation Tool• Beta tested with PDS community input.• Released version 2.1.0 in March 2010.• Integrated with Product Tools Library 3.0.2.
• Catalog Ingest Tool• Released version 1.0.0 in January 2010.
• NASAView• Released version 3.6.0 in January 2010.
PDS3 Tool Architecture
8
PDS 2010 Development Efforts
• Current Efforts• Development of the Registry Service REST-based
interface.• Deployment of the Security Service and integration with
the Registry Service REST-based interface.• Planned Efforts
• Evaluation, selection and deployment of a COTS package that satisfies the Report Service requirements.
• Implement the Harvest Tool and integrate with the Registry Service.
• Initial tool development focusing on validation.• More to come…
9
10
PDS3 Standards Maintenance• Betty Sword and Ron Joyner are now managing the SCR
queue and standards process.
• Metrics• 2 New; 2 In Draft; 3 in Review
11
PDS4 Standards Development• Held a DDWG face-to-face meeting to address outstanding PDS4
issues. • A simplified product model was proposed.• Several outstanding issues were either clarified or resolved.• Updates were made to the PDS4 Information Model.
• Held a two hour telecon almost weekly.• Generated a preliminary Data Dictionary.
• Registration Authority, Steward, and Namespaces policies are under discussion.
• Data Dictionary content cleanup was started.• Updated Data Provider’s Handbook.• Data standards builds are made about every other week to reflect
the latest design decisions and proposals.• Documents, schemas, and ancillary files are regenerated from the PDS4
Information Model• Alternate design options are often included in the model for testing.• The model driven design process promotes convergence towards a
consensus since an updated set of data standards are continually available for discussion, use, and testing.
Operations• Kept up with data sets(~230) and keywords ingestion, no back log• Managed subscription and ongoing data releases (Mars missions,
CAS, ODY quarterly), first release of LRO, all Phoenix & MESSENGER releases on time
• Supported peer review of ~80 data sets • Performed PDS web sites link check twice• Certified IT security plan and install required patches• Replenished and upgraded hardware/COTS software as needed• Supported Standards Coordination meetings• Participated in MIWG, NDWG• Facilitated contract set up with SDSC• Managed and monitored PDS infrastructure for day to day
operations (maintained high availability)• Handled user inquiries• Generated monthly and quarterly metrics reports• Managed document & hard media inventory
12
Performance Monitoring
13
Average Applications Average servers Mirror Site Uptime Downtime Uptime Downtime Uptime Downtime
Mar-09 99.99 0.01 100.00 0.00 96.34 3.66
Apr-09 99.98 0.02 100.00 0.00 99.27 0.73
May-09 99.78 0.22 99.98 0.02 93.44 6.56
Jun-09 99.73 0.27 100.00 0.00 95.67 4.33
Jul-09 99.84 0.16 100.00 0.00 99.73 0.27
Aug-09 99.98 0.16 100.00 0.00 99.83 0.17
Sep-09 99.29 0.71 99.45 0.00 92.66 7.34
Oct-09 99.37 0.63 99.45 0.00 99.39 0.61
Nov-09 99.49 0.51 99.57 0.29 99.18 0.82
Dec-09 99.85 0.15 100.00 0.00 99.56 0.44
Jan-10 99.87 0.12 99.87 0.13 99.23 0.77
Feb-10 100.00 0.00 100.00 0.00 99.18 0.82
0.0020.0040.0060.0080.00
100.00120.00
Monthly System Performance Average Uptime and Downtime Percentage
Average Applications Up-timeAverage Applications DowntimeAverage servers UptimeAverage servers Down-timeMirror Site UptimeMirror Site Downtime
Backup
14
URLsSchedules
EN Schedule http://pds-engineering.jpl.nasa.gov/about_eng/EN_FY10_schedule.pdf
StandardsStandards Changes
http://pds-engineering.jpl.nasa.gov/index.cfm?pid=2 Tasks
EN Tasks (EN Internal, includes milestones, current status and historical documents)http://pds-engineering.jpl.nasa.gov/index.cfm?pid=100
ReleaseData and Software
http://pds.jpl.nasa.gov/subscription_service/SS-Release.html
15
MC/Technical Group Engineering Concept
16
PDS Management Council
PDS Management Council
PDS Technical Staff
PDS Technical Staff Policy, Requirements
Recommendations/Needs,Implementations
Priorities, Actions, Policies, Requirements
Policy, RequirementsRecommendations/Needs,Implementations
Priorities, Actions, Policies, Requirements