Design and Analysis of Experiments for Europa Clipper’s Science Sensitivity Model Amy Braverman 1 , Manny Uy 2 , Vineet Yadav 1 and Kelli McCoy 1 1 Jet Propulsion Laboratory, California Institute of Technology 2 Applied Physics Laboratory, Johns Hopkins University April 11, 2019 1
26
Embed
Design and Analysis of Experiments for Europa Clipper’s Science … · 2019-06-24 · Europa Clipper’s orbit: ˘14 days; nominal mission is 46 orbits Jupiter Europa’s orbit:
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Design and Analysis of Experiments forEuropa Clipper’s Science Sensitivity Model
Amy Braverman1, Manny Uy2, Vineet Yadav1 and Kelli McCoy1
1 Jet Propulsion Laboratory, California Institute of Technology2 Applied Physics Laboratory, Johns Hopkins University
April 11, 2019
1
Outline
I Introduction
I The Europa Clipper mission
I Data acquisition
I Factors affecting data acquisition
I Mission requirements
I The experiment
I Experiment design
I Experiment analysis
I Conclusions
2
Introduction
Goals:
I Design, implement, and analyze a simulation experiment to quantify theprobability of mission success as a function of instrument fault andautonomous recovery rates.
I Mission success = meeting the set of “Level 1" Baseline requirements withprobability at least .95.
I Provide a tool to allow scientists and engineers to study how changes inboth requirements and hardware performance affect mission and Level 1requirement success probability.
3
Introduction
The Europa Clipper spacecraft will carry nine science instruments– thus thereare ten “systems":
System Code System Code
Spacecraft Sc Instrument 5 I5
Instrument 1 I1 Instrument 6 I6
Instrument 2 I2 Instrument 7 I7
Instrument 3 I3 Instrument 8 I8
Instrument 4 I4 Instrument 9 I9
4
Introduction
System Code System Code
Spacecraft Sc Instrument 5 I5
Instrument 1 I1 Instrument 6 I6
Instrument 2 I2 Instrument 7 I7
Instrument 3 I3 Instrument 8 I8
Instrument 4 I4 Instrument 9 I9
Each system’s transient fault behavior is modeled by two different exponentialdistributions, depending on orbital conditions.
Each system’s recovery time (after a fault) is modeled by a (shifted) betadistribution “outside" the flyby region, and as a constant “within" the flyby region(see next slides).
5
Europa Clipper Mission
Jupiter
Europa’sorbit: ∼3.5 days
Closest approach-3 days
Closest approach+2 days
Closestapproach
Closest approach+10 hrs
Closestapproach-10 hrs
High radiationzone: fault ratesare higher
6
Europa Clipper Mission
Europa Clipper’sorbit: ∼14 days;nominal missionis 46 orbits
Jupiter
Europa’sorbit: ∼3.5 days
Closest approach-3 days
Closest approach+2 days
Closestapproach
Closest approach+10 hrs
Closestapproach-10 hrs
High radiationzone: fault ratesare higher
7
Europa Clipper Mission
Europa Clipper’sorbit: ∼14 days;nominal missionis 46 orbits
Jupiter
Europa’sorbit: ∼3.5 days
Closest approach-3 days
Closest approach+2 days
Closestapproach
Closest approach+10 hrs
Closestapproach-10 hrs
High radiationzone: fault ratesare higher
Low-radiationzone: fault ratesare lower
Most importantscience dataacquired in high-radiation zone
8
Europa Clipper Mission
Europa Clipper’sorbit: ∼14 days;nominal missionis 46 orbits
Jupiter
Europa’sorbit: ∼3.5 days
Closest approach-3 days
Closest approach+2 days
Closestapproach
Closest approach+10 hrs
Closestapproach-10 hrs
Most importantscience dataacquired in high-radiation zone
“Outside" portion oforbit: recovery canbe slower
9
Europa Clipper Mission
Europa Clipper’sorbit: ∼14 days;nominal missionis 46 orbits
Jupiter
Europa’sorbit: ∼3.5 days
Closest approach-3 days
Closest approach+2 days
Closestapproach
Closest approach+10 hrs
Closestapproach-10 hrs
Most importantscience dataacquired in high-radiation zone
“Outside" portion oforbit: recovery canbe slower
“Within" portion oforbit: recoverymust be faster
10
Data Acquisition
I Each system will be in either an active or inactive state at every time pointduring the 46-orbit “tour".
I Let Xi(t) be the state of system i , i = 0, 1, 2, . . . , 9, at (continuous) time t ,t ∈ (0,T ), where T is the number of seconds in the 46-day tour (i = 0 forthe spacecraft).
I If system i is active at time t , then Xi(t) = 1. Otherwise Xi(t) = 0.
I If the spacecraft goes down (X0(t) = 0), all other systems go down.
I Ability to acquire high-value science data will depend on the complexsuper-positiion of the ten systems’ states during the period from -10 to +10hours of closest approach. (Caveat: two instruments do take datathroughout the whole orbit, and these are also critical for science.)
11
Data Acquisition
Sc
t=
0
t=
T
| | | |
I1 | | || | | | | | | | |
I2 | |
I3 | | | |
I4 | | | | | |
I5 | |
I6 | | | | | | | | | |
I7 | |
I8 | | | | | |
I9 | | | | | |
12
Factors affecting data acquisition
Sc
t=
0
t=
T
| | | |
I1 | | || | | | | | | | |
I2 | |
I3 | | | |
I4 | | | | | |
I5 | |
I6 | | | | | | | | | |
I7 | |
I8 | | | | | |
I9 | | | | | |
high-value data acquisition/high-radiation periods
Each instrument takes complete, partial, or no data during the critical periods.
13
Factors affecting data acquisition
Sc
t=
0
t=
T
| | | |
I1 | | || | | | | | | | |
I2 | |
I3 | | | |
I4 | | | | | |
I5 | |
I6 | | | | | | | | | |
I7 | |
I8 | | | | | |
I9 | | | | | |
“within" periods
14
Factors affecting data acquisition
Sc
t=
0
t=
T
| | | |
I1 | | || | | | | | | | |
I2 | |
I3 | | | |
I4 | | | | | |
I5 | |
I6 | | | | | | | | | |
I7 | |
I8 | | | | | |
I9 | | | | | |
“outside" periods
Different fault rates and recovery times apply (for each system) depending onwhether EC is in the red, orange, or green period.
15
Mission requirements
I Each instrument takes complete, partial, or no data during the criticalperiods.
I Each instrument has a set of instrument-specific requirements that areeither met or not met depending on the degree to which data are acquiredduring the critical periods.
I A tree-structured graph shows how instrument-level requirements areprogressively aggregated up through intermediate science objectivesthrough to Level 1 requirements.
I Determine maximum allowable fault rates and recovery times such that theset of Level 1 Baseline requirements have success probabilities thatexceed .95.
I Use Monte Carlo to simulate 1000 tours, each with faults generated byexponential distributions with specified parameters, and recovery timesgenerated as described earlier.
I These parameters are factors in the experiment.
I Responses are the probabilities of success for the Level 1 requirementsafter aggregating over the ensemble of 1000 tours.
I Build a response surface that relates factor levels to responses.
18
The experiment
timeline 1
timeline 2
timeline 1000
Y1 = (Y11,Y12, . . . ,Y1,91)′
Y2 = (Y21,Y22, . . . ,Y2,91)′
Y1000 = (Y1000,1,Y1000,2, . . . ,Y1000,91)′
Requirementschecker
Yjk = 1 if timeline j passes req. k ,
Yjk = 0 otherwise.
19
The experiment
Suppose p is the 91-dimensional vector of probabilities,
p =1
1000
1000∑n=1
Xn = (p1, p2, . . . , p91)′,
where pk is the probability of passing the k th (basic/“level 2"/leaf) requirement.
T = “transient" fault (the only kind here). H/L = high/low radiation. F = fault rate,R = recovery time. W = “within".
All other factor combinations are fixed, and treated deterministically.
Not all factors applicable to all Level 1 requirements.
21
Experiment design
I 30 factors (10 systems, three factors each). Experiment run for two levelsfor each factor (a high value and a low value).
I ∼ 9 Baseline Level 1 requirements.
I Definitive Screening Design (Jones and Nachtsheim, (2011). Journal ofQuality Technology, Vol 43, No. 1.) created in JMP 14 Pro. Requires only arelatively small number of runs (∼ 2× (number of factors)).
I We had sufficient computational power to augment the DSD with additionalspace-filling runs. Total number of runs = 577.
I Design space = 577 points in a 30-dimensional space. Responses = 577probabilities for each Level 1 req.
22
Experiment design
23
Experiment analysis
Fit full response surfaces and main effects only models.
24
Conclusions
I Tom Youmans to discuss substantive conclusions (next talk).
I The Europa Clipper requirements model and experiment provides aquantitative way to relate science outcomes to design choices in buildinginstrument and spacecraft systems.
I The Monte Carlo simulation experiment allows us to interrogate theserelationships, which are too complex to understand analytically.
I The experimental design is crucial to making the Monte Carlo-basedstrategy feasible: it ensures that the limited number of conditions underwhich the experiment can be run, is as informative as possible.
I Personal reflection: JMP is a great tool, but I wish it was programmable!
25
This work was partially performed at the Jet Propulsion Laboratory, CaliforniaInstitute of Technology, under contract with the National Aeronautics and SpaceAdministration.