Data Analysis Software Development Hisanori Furusawa ADC, NAOJ For HSC analysis software team 1.

Data Analysis Software Development

Hisanori FurusawaADC, NAOJ

For HSC analysis software team

1

Status Updates

2

1. Prototype with Suprime-Cam Data

• Quick analysis with in-hose software with CFITSIO library & TERAPIX software

• Investigating RCM+Roobasf middleware – 1st-stage reduction, statistics measurement,

mosaic/stack, catalog making– managing analysis history and data tags in database

• Progress– First test in August 2008

• Basic processing of data is OK• Improving the system, especially in analysis results management

– Testing the assembled system in Sep 20093

Raw data

Processed data

Statistics measurement

A Snapshot of the prototype software

4

2. Data Simulation• A field of view of HSC with artificial objects• PA, CCD arrangement, Geometrical effects

(Distortion+ADC, Airmass), and Vignetting– Useful for development of a procedure of data

handling• Photon tracing (turbulence in the atmosphere) led by Hamana&Miyazaki

Example of FOV w/ vignetting

5

3. Discussion of Analysis Procedure

• Requirements on HSC data products– Hearing led by Strauss, Furusawa+ Specification & Design of HSC Analysis

• Determining analysis procedure & Breaking down development tasks

Determining collaboration framework System Design

6

Data Acquisition

CreateBias

CreateFlat CorrectFrame

FindBrightObjects

Bright object listwith Postage Stamps

Astrometry &Distortion

FluxCalib

MeasurePSF

FindFaintObjects

MeasureObjects

MeasureStats

Raw imagesUse every frames from 1 shot. Assume CCD alignment.

include spatial variationPSF matching detection

for single frameSky level, seeing, detection limit, …

Standard StarAnalysis

HSC Data Analysis Pipeline (1)From DAQ to Single Frame Analysis

7

HSCANA jobWL WG job

WG analysis tools can be added

CollectFrame

MosaicFrame 1. Select stars and identification of same object 2. Solve optical distortion, atmospheric effect, CCD alignment, offsets between shots 3. Solve relative flux scaling

StackFrame 1. Sky Subtraction 2. Match PSF 3. Warping and flux scaling

FindObjects

FindFaintObjects

MeasureObjects

Astrometry

FluxCalib

MeasurePSF

for stacked image

MergeObjects

x 5 bands

For the objects NOT detected in some bands,measure corresponding parameters at the positions of objects detected in other bands.

MeasureShape

Another PathWhen we process without stacking

Feed back to single frames

HSC Data Analysis Pipeline (2)From Mosaicking to Catalog Making

with m

osaic/stacked im

agesw/o

sta

ckin

g

8

Collaborative Framework (Tentative)

1. Analysis of Correcting Frames (1a) & Constructing Mosaic Images (1b) – NAOJ/IPMU

2. Photometric & Astrometric Calibration –

NAOJ/IPMU3. Science Archive – NAOJ 4. Detection of objects & Measurements of object

parameters & PSF mapping – Princeton 5. Analysis Framework – KEK, UT/Physics

9

Topics for Today’s Discussion

For Weak Lensing Analysis:1. Necessary Data Products (in what stage & in

what format)2. Necessary Extracted Parameters (Shape or

profile, Size, Brightness …) & What are to be in Science Data Archive

3. Roles of Analysis Software Team and WL-WG on WL analysis software

10

1. Data Products for WL Analysis Standard HSC-ANA will provide:1. Individual Corrected Frames (CCD-by-CCD) with ..

1. Photometric & astrometric calibration (tolerance?)2. PSF measurements across the fields (how&what?) 3. Mosaicing information, i.e., alignment & flux scaling4. No geometrical transformation done

(distortion, airmass effect remain) 2. Stacked Mosaic Image with Above 1 & 2.

(what format?)

3. Standard Object Catalog (what parameters?)

Any other? 11

2. Parameters to be Extracted & Science Archive

1. HSC-ANA will extract standard parameters in the next slide (tentative) from standard stacked images

2. Standard parameters & images (standard corrected frames & stacked images) are supposed to be in Science archive

Any other ideas?

12

Standard Parameters(errors are associated with all measurements)

id Unique ID to identify objectband Observed bandMJD Mean time during exposure

* Position *xc, yc center of objects in pixelra, dec Equatrial coordinategLong, gLat, eLong, eLat, …

* Type *type star / galaxy indicator

* Flag *flag object flag

* Flux * (circle or ellipse aperture?)psfMag PSF fitted magnitudepetroMag Petrosian magnitudepetroRad Petrosian radiuspetroR50, petroR90KronMag Kron magnitudeKronRad Kron radiusmodelMag Model integrated magnitudeextinction Galactic extinction

* Shape * Stokes Q and U 2nd order moment

*Profile *n Sersic indexreff Half light radiusAB a/b axis ratioPhi Position AngleRadial Profile -> Aperture mag

if needed

* Size *FWHMSome dimension indicator

* After MergeObjects *type -> utilize color informationphotozlink -> to external catalogs (NIR photometry, spectroscopic info)

HSC Data Analysis Pipeline - Standard Parameters to be measured

Shape measurement related

+ WL dedicated parameters?

13

3. WL Software & Our Roles

1. HSC-ANA is responsible to provide• Standard Images (Corrected frames & Stacked images)• Standard Object Parameters• Science Archive

2. WL-WG will determine data & parameters necessary for WL analysis and optimal methods to produce the data

3. WL-WG will lead Implementation of the methods, in collaboration with HSC-ANA members

4. The WL software can (& should) be added to the standard HSC analysis system by HSC-ANA members

14

End

15