TG08 TG08 [email protected][email protected]1 Toward Parallel Space Radiation Analysis Dr. Liwen Shih, Thomas K. Gederberg, Karthik Katikaneni, Ahmed Khan, Sergio J. Larrondo, Susan Strausser, Travis Gilbert, Victor Shum, Romeo Chua University of Houston Clear Lake
35
Embed
[email protected] Toward Parallel Space Radiation Analysis Dr. Liwen Shih, Thomas K. Gederberg, Karthik Katikaneni, Ahmed Khan, Sergio J. Larrondo, Susan.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
This project continues Space Radiation Research work preformed last year by Dr. Liwen Shih’s students to investigate HZETRN code optimization options.
This semester we will analyze HZETRN code using standard static analysis tools and runtime analysis tools. In addition we will examine code parallelization options for the most called numerical method in the source code: the PHI function.
Two major sources galactic cosmic rays (GCR) solar energetic particles (SEP).
GCR are ever-present and more energetic, thus they are able to penetrate much thicker materials than SEP.
In order to evaluate the space radiation risk and design the spacecraft and habitat for better radiation protection, space radiation transport codes, which depends on the input physics of nuclear interactions, have been developed
What about Galactic Cosmic Radiation What about Galactic Cosmic Radiation (GCR)?(GCR)?
A typical high energy particle of A typical high energy particle of radiation found in the space radiation found in the space environment is ionized itself and environment is ionized itself and as it passes through material as it passes through material such as human tissue it disrupts such as human tissue it disrupts the electronic clouds of the the electronic clouds of the constituent molecules and leaves constituent molecules and leaves a path of ionization in its wake. a path of ionization in its wake. These particles are either singly These particles are either singly charged protons or more highly charged protons or more highly charged nuclei called "HZE" charged nuclei called "HZE" particles.particles.
Radiation Fluence of HZE particles:Radiation Fluence of HZE particles:time-integrated flux of HZE particles per unit area.time-integrated flux of HZE particles per unit area.
Energy absorbed per gram:Energy absorbed per gram:first measuring energy amount left behind by first measuring energy amount left behind by
radiation in question and, then, amount and type radiation in question and, then, amount and type of material.of material.
Dose Equivalent:Dose Equivalent:A unit of dose equivalent A unit of dose equivalent amount of any type of amount of any type of
radiation absorbed in a biological tissue as a radiation absorbed in a biological tissue as a standardized valuestandardized value
HZETRN used for Mars HZETRN used for Mars Mission Mission
Thus, protection from the hazards of severe space radiation is of paramount importance for the new vision. There is an overwhelming emphasis on the reliability issues for the mission and the habitat. Accurate risk assessments critically depend on the accuracy of the input information about the interaction of ions with materials, electronics and tissues.
NASA has a new vision for space exploration in the 21st Century encompassing a broad range of human and robotic missions including missions to Moon, Mars and beyond. As a result, there is a focus on long duration space missions. NASA, as much as ever, is committed to the safety of the missions and the crew. Exposure from the hazards of severe space radiation in deep space long duration missions is ‘the show stopper.’
Project Goal: Project Goal: SpeedupSpeedup of Runtime via Analysis and of Runtime via Analysis and modification of HZETRN Code numerical algorithmmodification of HZETRN Code numerical algorithm
Runtime Profile Of HZETRN Functions
texp, 2.91
powf.J, 1.35
prpgt_, 0.93
od_, 0.73
cvtas_s_to_a, 0.73
PHI (Interpolation Function Most Time
Spent Here at: 34.5% of total
runtime)
13,220,184 calls made to this function over
program run!
Remaining Functions, 9.03
expf.J, 8.72%
iuni 4.36
prpli_, 1.97
logf.J, 30.43%
anu, 4.26
phi
logf.J
expf.J
iuni_
anu
texp
prpli_
powf.J
prpgt_
od_
cvtas_s_to_a
Remaining Functions
The major Space Radiation Code Bottleneck lies inside the function call to the PHI interpolation function
Code Optimization OptionsCode Optimization Options4028 C ************************************************************** 4028 C ************************************************************** 4029 C 4029 C 4030 FUNCTION PHI(R0,N,R,P,X)4030 FUNCTION PHI(R0,N,R,P,X) 4031 C4031 C 4032 C FUNCTION PHI INTERPOLATES IN P(N) ARRAY DEFINED OVER R(N) 4032 C FUNCTION PHI INTERPOLATES IN P(N) ARRAY DEFINED OVER R(N)
ARRAY ARRAY 4033 C ASSUMES P IS LIKE A POWER OF R OVER SUBINTERVALS4033 C ASSUMES P IS LIKE A POWER OF R OVER SUBINTERVALS 4034 C 4034 C 4035 DIMENSION R(N),P(N)4035 DIMENSION R(N),P(N) 4036 C4036 C 4037 SAVE4037 SAVE 4038 C4038 C 4039 XT=X4039 XT=X 4040 PHI=P(1)4040 PHI=P(1) 4041 INC=((R(2)-R(1))/ABS(R(2)-R(1)))*1.014041 INC=((R(2)-R(1))/ABS(R(2)-R(1)))*1.01 4042 IF(X.LE.R(1).AND.R(1).LT.R(2))RETURN4042 IF(X.LE.R(1).AND.R(1).LT.R(2))RETURN 4043 C4043 C 4044 DO 1 I=3,N-14044 DO 1 I=3,N-1 4045 IL=I4045 IL=I 4046 IF(XT*INC.LT.R(I)*INC)GO TO 24046 IF(XT*INC.LT.R(I)*INC)GO TO 2 4047 1 CONTINUE4047 1 CONTINUE 4048 C4048 C 4049 IL=N-14049 IL=N-1 4050 2 CONTINUE4050 2 CONTINUE 4051 PHI=0.4051 PHI=0.
1. Fix Inefficient code
2. Fix/Remove unnecessary function calls (TEXP) SAVE, and dummy arguments
3. Use optimized ALOG function
4. Use Lookup Table instead
5. Investigate Parallelization Of Interpolation Statements
PHI Routine OptimizationPHI Routine Optimization Bottleneck PHI routine being Bottleneck PHI routine being called so heavilycalled so heavily, message , message
passing overhead to parallelize would be passing overhead to parallelize would be prohibitiveprohibitive..
Simple Simple code optimizationscode optimizations of PHI routine resulted in: of PHI routine resulted in:
– 11.4 % speedup on home PC running Linux compiled 11.4 % speedup on home PC running Linux compiled
using the Sun Studio 12 Fortran compiler.using the Sun Studio 12 Fortran compiler.
– 3.85% speedup on an Atlantis node using the Intel 3.85% speedup on an Atlantis node using the Intel
Fortran compiler.Fortran compiler.
– Reduced speedup on Atlantis may be that the Reduced speedup on Atlantis may be that the Intel Intel
compilercompiler was already generating more optimized was already generating more optimized
DMETRIC RoutineDMETRIC Routine - Dependent? - Dependent? To determine if loop 5 is parallelizable, To determine if loop 5 is parallelizable, the outer the outer
loop was firstloop was first changed to decrement from changed to decrement from IIII to 1 to 1
rather than from 1 to rather than from 1 to IIII. The results were . The results were
identical identical outer loop of loop 5 should be outer loop of loop 5 should be
parallelizableparallelizable..
Next the inner loop was changed to decrement Next the inner loop was changed to decrement
from from IJIJ to 2 rather than from 2 to to 2 rather than from 2 to IJIJ. . Differences Differences
appear in the last significant digitappear in the last significant digit (see next page). (see next page).
These differences are due to These differences are due to floating point floating point
rounding differencesrounding differences during four summations. during four summations.
PRPGT RoutinePRPGT Routine PRPGT - propagate GCR's through the shielding and the target.PRPGT - propagate GCR's through the shielding and the target.
~ 82% of HZETRN processing is spent in PRPGT or routines it ~ 82% of HZETRN processing is spent in PRPGT or routines it
calls.calls.
At each propagation step from one depth to the next in the At each propagation step from one depth to the next in the
shield or target, the propagation for each of the 59 isotopes is shield or target, the propagation for each of the 59 isotopes is
performed in two stages:performed in two stages:
– The first stage computes the energy shift due to propagationThe first stage computes the energy shift due to propagation
– The second stage computes the attenuation and the The second stage computes the attenuation and the
secondary particle production due to collisionssecondary particle production due to collisionsTo test whether the propagation for each of the 59 ions could be done in parallel, the loop was broken up into four pieces (a J loop from 20 to 30, from 1 to 19, from 41 to 59, and from 31 to 40).If the loop can be performed in parallel, then the results from these four loops should be the same as the single loop from 1 to 59.
DependencyDependency The following compares the results of breaking up main loop into four The following compares the results of breaking up main loop into four
loops (on the left) with the original results.loops (on the left) with the original results.
Significant different results demonstrate that the propagation can not be parallelized
DependentDependent Identical to original results reversing inner 1Identical to original results reversing inner 1stst and 2 and 2ndnd stage I loops stage I loops
possible to parallelize the 1possible to parallelize the 1stst or 2 or 2ndnd stages stages..
However, to test data dependence from the 1However, to test data dependence from the 1stst stage to the 2 stage to the 2ndnd
stage, the main J loop was divided into two loops (one for the 1stage, the main J loop was divided into two loops (one for the 1stst
stage and one for the 2stage and one for the 2ndnd stage) stage)
Results changed Results changed the 2the 2ndnd stage is dependent on the 1 stage is dependent on the 1stst stage stage
A barrier to prevent execution of the 2A barrier to prevent execution of the 2ndnd stage until the 1 stage until the 1stst stage stage
completescompletes
24% of the HZETRN processing is spent on the 124% of the HZETRN processing is spent on the 1stst stage while less stage while less
than 2% of the time is spent on the 2than 2% of the time is spent on the 2ndnd stage. Therefore, parallel stage. Therefore, parallel
processing of both stages does not appear worthwhileprocessing of both stages does not appear worthwhile..
Parallel PRPLI RoutineParallel PRPLI Routine PRPLI is called by PRPGT after the 1PRPLI is called by PRPGT after the 1stst and 2 and 2ndnd stage propagation stage propagation
has been completed for each of the 59 isotopes.has been completed for each of the 59 isotopes.
PRPLI performs the propagation of the six light ions (ions Z < 5).PRPLI performs the propagation of the six light ions (ions Z < 5).
~ ~ 53%53% of total HZETRN of total HZETRN time is spent on light ions propagation.time is spent on light ions propagation.
PRPLI propagates 45 x 6 fluence (# particles intersect a unit PRPLI propagates 45 x 6 fluence (# particles intersect a unit
area) matrix (45 energy points for each of the 6 light ions) area) matrix (45 energy points for each of the 6 light ions)
named PSI.named PSI.
Analysis of the has shown that there is Analysis of the has shown that there is no data dependency no data dependency
among the energy grid pointsamong the energy grid points..
It should, therefore, be It should, therefore, be possible to parallelize the PRPLI code possible to parallelize the PRPLI code
across the 45 energy grid pointsacross the 45 energy grid points..
ReferencesReferences J.W. Wilson, F.F. Badavi, F. A. Cucinotta, J.L. Shinn, G.D. Badhwar, R. Silberberg, C.H. Tsao, L.W. J.W. Wilson, F.F. Badavi, F. A. Cucinotta, J.L. Shinn, G.D. Badhwar, R. Silberberg, C.H. Tsao, L.W.
Townsend, R.K. Tripathi, Townsend, R.K. Tripathi, HZETRN: Description of a Free-Space Ion and Nucleon Transport Shielding HZETRN: Description of a Free-Space Ion and Nucleon Transport Shielding
Computer ProgramComputer Program, NASA Technical Paper 3495, May 1995., NASA Technical Paper 3495, May 1995.
J. W. Wilson, J.L. Shinn, R. C. Singleterry, H. Tai, S. A. Thibeault, L.C. Simmons, J. W. Wilson, J.L. Shinn, R. C. Singleterry, H. Tai, S. A. Thibeault, L.C. Simmons, Improved Spacecraft Improved Spacecraft
Materials for Radiation ShieldingMaterials for Radiation Shielding, NASA Langley Research Center. , NASA Langley Research Center.
NASA Facts: Understanding Space RadiationNASA Facts: Understanding Space Radiation, FS-2002-10-080-JSC, October 2002., FS-2002-10-080-JSC, October 2002.
P. S. Pacheco, P. S. Pacheco, Parallel Programming with MPIParallel Programming with MPI, Morgan Kaufmann Publishers Inc.: San Francisso, , Morgan Kaufmann Publishers Inc.: San Francisso,
1997.1997.
S. J. Chapman,S. J. Chapman, Fortran 90/95 for Scientists and Engineers Fortran 90/95 for Scientists and Engineers, 2, 2ndnd edition. McGraw Hill: New York, 2004. edition. McGraw Hill: New York, 2004.
L. Shih, S. Larrondo, K. Katikaneni, A. Khan, T. Gilbert, S. Kodali, A. Kadari, L. Shih, S. Larrondo, K. Katikaneni, A. Khan, T. Gilbert, S. Kodali, A. Kadari, HIgh Performance Martian HIgh Performance Martian
Space Radiation MappingSpace Radiation Mapping, NASA/UHCL/UH_ISSO, pp. 121-122., NASA/UHCL/UH_ISSO, pp. 121-122.
L. Shih, L. Shih, Efficient Space Radiation Computation with Parallel FPGAEfficient Space Radiation Computation with Parallel FPGA, Y2006 – ISSO Annual Report, pp. , Y2006 – ISSO Annual Report, pp.
56-61.56-61. Gilbert, T. and L. Shih. "High-Performance Martian Space Radiation Mapping," IEEE/ACM/UHCL Gilbert, T. and L. Shih. "High-Performance Martian Space Radiation Mapping," IEEE/ACM/UHCL
Computer Application Conference, University of Houston-Clear Lake, Houston, TX, April 29, 2005.Computer Application Conference, University of Houston-Clear Lake, Houston, TX, April 29, 2005.
Kadari, A.. S. Kodali, T. Gilbert, and L. Shih. "Space Radiation Analysis with FPGA," IEEE/ACM/UHCL Kadari, A.. S. Kodali, T. Gilbert, and L. Shih. "Space Radiation Analysis with FPGA," IEEE/ACM/UHCL Computer Application Conference, University of Houston-Clear Lake, Houston, TX, April 29, 2005.Computer Application Conference, University of Houston-Clear Lake, Houston, TX, April 29, 2005.
F. A. Cucinotta, "Space Radiation Biology," NASA-M. D. Anderson Cancer Center Mini-Retreat, Jan. 25, F. A. Cucinotta, "Space Radiation Biology," NASA-M. D. Anderson Cancer Center Mini-Retreat, Jan. 25, 2002 <2002 <http://advtech.jsc.nasa.gov/presentation_portal.shtmhttp://advtech.jsc.nasa.gov/presentation_portal.shtm>.>.
Space Radiation Health Project, May 3, 2005, NASA-JSC, March 7, 2005 <Space Radiation Health Project, May 3, 2005, NASA-JSC, March 7, 2005 <http://srhp.jsc.nasa.gov/http://srhp.jsc.nasa.gov/> >
AcknowledgementsAcknowledgements NASA LaRC -NASA LaRC - Robert C. Singleterry JrRobert C. Singleterry Jr, PhD, PhD NASA JSC/CARR PVA&M -NASA JSC/CARR PVA&M - Premkumar B. SagantiPremkumar B. Saganti, PhD, PhD TeraGrid, TACC TeraGrid, TACC TLC2 -TLC2 - Mark HuangMark Huang & Erik & Erik EngquistEngquist Texas Space Grant Consortium ISSOTexas Space Grant Consortium ISSO