STATUS OF APOGEE DATA PRODUCTS, REDUCTION, AND ANALYSIS Jon Holtzman (NMSU) David Nidever (UVa) Ana Garcia-Perez (UVa) Carlos Allende-Prieto (IAC) Szabolcs.

STATUS OF APOGEE DATA PRODUCTS, REDUCTION, AND

ANALYSIS

Jon Holtzman (NMSU)David Nidever (UVa)Ana Garcia-Perez (UVa)Carlos Allende-Prieto (IAC)Szabolcs Mezaros (IAC)The APOGEE team

2

Data Overview

• Target selection : input photometry, selection flags, extinction, etc.

• Observations: up-the-ramp readouts + telescope/weather information raw APOGEE data files. Exposure sets are taken in visits: multiple frames at different dither positions

• Data reduction: • Visit analysis: basic calibration, extraction, sky correction,

initial RV• Visit combination: wavelength resampling, refined RVs

• Chemical analysis (ASPCAP)• Stellar parameter determination• Individual element abundances

Most major results summarized in two data files: allVisit : merges target information, individual visit RVs,

individual visit stellar parameters. One record per visit per star

allStar : merges target information, average RVs (soon, combined frame RV), average individual visit stellar parameters (soon, stellar parameters from combined frames)

Available in both FITS table, and (wide!) ASCII file Files are sorted by RA, with accompanying RA “index” table

(FITS table version) for faster searches allStar file contains ID records (FITS table version) of

individual visits in allVisit file

Summary data files

objid : TMASS-style object name

file: apVisit file name

fiber: fiber number

plate: plate number

mjd: mjd observation date

location : field location ID

field: field name

ra : RA

dec : DEC

glon : galactic longitude

glat : galactic latitude

jmag : 2MASS J

hmag : 2MASS H

kmag : 2MASS K

primtarg : primary target flag

sectarg : secondary target flag

jd: JD of observations

bc: Barycentric correction

vtype: Radial velocity method type

vrel: derived radial velocity

vrelerr: error in radial velocity

vhelio: derived barycentric RV

vlsr: derived VLSR

vgsr: derived VGSR

allVisit contents

rv_teff: effective temperature used for RV templaterv_logg: log g used for RV templaterv_feh: [Fe/H] used for RV templaterv_chi2: chi2 of RV fitsnr: median S/N ratio of spectrumsynthfile: synthetic spectrum file namepsfile: plot fileteff : ASPCAP Teff from visitlogg : ASPCAP logg from visitvmicro : ASPCAP vmicro from visitmetals : ASPCAP metals from visitc : ASPCAP stepar carbon from visitn : ASPCAP stepar nitrogen from visitalpha : ASPCAP alpha from visitaspcap_chi2: ASPCAP chi2 from visit fitaspcap_vers : ASPCAP versionak : extinction used in target selectionakmethod : extinction method usedak_wise_allsky : extinction using WISE ALLSKY releasewash_m : Washington Mwash_m_errwash_t2 : Washington T2wash_t2_errddo51 : DDO 51ddo51_errirac_3_6 : IRAC 3.6micron magirac_3_6_errmag_4_5 : 4.5micron mag (IRAC or WISE)mag_4_5_errirac_5_8 : IRAC 5.8micron magirac_5_8_errirac_8_0 : IRAC 8.0micron mag

obj : TMASS-style object name

locid : field location ID

field : field name

jmag : 2MASS J

hmag : 2MASS H

kmag : 2MASS K

ra : RA

dec : DEC

glon : galactic longitude

glat : galactic latitude

ak : extinction used in target selection

akmethod : extinction method used

ak_wise_allsky : extinction using WISE ALLSKY release

primtarg : primary target flag

sectarg : secondary target flag

nvisits : number of visits

pre : =1 for pre-2012 shutdown, otherwise 0

vavg: average RV

vscatter : scatter in RV

snr : S/N of combined frame

teff_avg : Average ASPCAP Teff from multiple visits

logg_avg : Average ASPCAP logg from multiple visits

vmicro_avg : Average ASPCAP vmicro from multiple visits

metals_avg : Average ASPCAP metals from multiple visits

c_avg : Average ASPCAP stepar carbon from multiple visits

n_avg : Average ASPCAP stepar nitrogen from multiple visits

alpha_avg : Average ASPCAP alpha from multiple visits :

allStar contents

teff_sig : Sigma of ASPCAP Teff from multiple visits logg_sig : Sigma of ASPCAP logg from multiple visits vmicro_sig : Sigma of ASPCAP vmicro from multiple visits metals_sig : Sigma of ASPCAP metals from multiple visits c_sig : Sigma of ASPCAP stepar carbon from multiple visits n_sig : Sigma of ASPCAP stepar nitrogen from multiple visits alpha_sig : Sigma of ASPCAP alpha from multiple visits teff : ASPCAP Teff from combined frame logg : ASPCAP logg from combined frame vmicro : ASPCAP vmicro from combined frame metals : ASPCAP metals from combined frame c : ASPCAP stepar carbon from combined frame n : ASPCAP stepar nitrogren from combined frame alpha : ASPCAP alpha from combined frame aspcap_chi2 :ASPCAP chi2 aspcap_vers : ASPCAP version wash_m : Washington M wash_m_err wash_t2 : Washington T2 wash_t2_err ddo51 : DDO 51 ddo51_err irac_3_6 : IRAC 3.6micron mag irac_3_6_err mag_4_5 : 4.5micron mag (IRAC or WISE) mag_4_5_err irac_5_8 : IRAC 5.8micron mag irac_5_8_err irac_8_0 : IRAC 8.0micrton mag, irac_8_0_err

6

Other data products

• Data model available at http://data.sdss3.org/datamodel, data at http:/data.sdss3.org/sas/bosswork/groups/apogee (to be renamed…)

• Raw data: apR-[abc]-ID8.fits (in APOGEE_DATA/)• Reduced data (in APOGEE_REDUX/vers/)

• Basic calibrations: red/MJD5/ap2D-[abc]-ID8.fits• Extracted 1D: red/MJD5/ap1D-[abc]-ID8.fits• Sky corrected:

plates/PLATE4/MJD5/apCframe-[abc]-ID8.fits• Dither combined: plates/PLATE4/MJD5/apPlate-[abc]-ID8.fits,

or separated apVisit-PLATE4-MJD5-FIBER3.fits (with initial RVs in headers), native wavelength scale (with wavelength array)

• Visit combined: stars/LOC4/apStar-NAME.fits: rebinned to fixed log lambda dispersion

• Chemical analysis (in APOGEE_ASPCAP/vers/)• Best fit stellar parameters and model spectra from individual

visit spectra in results/aspcap-PLATE4-MJD5.fits

7

Sample spectra

Spectra plots available on embedded web pages, + RV plots, links to FITS files

8

DR10

DR10 contents still TBD, but will likely include•Table of all possible targets with target selection data•Design, field, plate data•allVisit table•allStar table•Separate ASPCAP tables for different versions? To include …?

9

Data quantity status

• Sky coverage• S/N and number of visits• Reduction processing: visit

processing routine and current, happens within 2-3 days of data taking; combination processing happening ~monthly

• ASPCAP processing: uniform (but non-optimal) processing through ~March 2012

• S/N vs H…

• Mostly relatively standard reduction steps• Basic calibration (flats, darks, linearity):

• flats seem stable, minor improvements have been made over current pipeline version

• Darks need some analysis• Linearity corrections not currently implemented

• Extraction• Currently done with empirical PSF, considering 3 fibers

at a time

• Sky correction: telluric and sky subtraction• Dither combination• Visit RV• Visit combination / refined RVs

Data quality: reduction

Data quality: sky and telluric

Velocity Uncertainty•Scatter from multiple measurements•Some real variability•Peak RV scatter ~100 m/s

Velocity zeropoint•Comparison to GC RVs•RV offset average -0.26 +/- 0.2 km/s

Data quality: RVs

• ASPCAP tuning/development still active• Tuning of linelists• Fit method (PCA vs flux space) being changed• Parameter space (7 param vs 6 param plus

fixed/coupled microturbulence) still under consideration• Fitting of combined spectra as well as individual visit

spectra

• Some tests of ASPCAP stellar parameters available• Repeatability: internal precision• Clusters • Kepler asteroseismic targets

Data quality: ASPCAP

ASPCAP precision

Sample: •Combined S/N>220•N(ASPCAP>=5)

ASPCAP accuracy: metallicities

• ASPCAP metallicities seem fairly good near solar, but are systematically high for metal-poor clusters

• TBD: understanding and decision on whether to implement calibration relations

ASPCAP accuracy: gravities

• ASPCAP gravities systematically high:• ~0.4 dex near

solar• ~1 dex for

metal-poor clusters

Significant issue for distances

Again, calibratable or need to solve?

Comparison with asteroseismic gravities

• Catalog RJCE extinctions derived assuming fixed color for all stars: expected metallicity dependence could give systematic errors of 0.1-0.2 mag in A(K)

• Simple to implement metallicity dependent A(K)• Still some uncertainties at metal-poor end: extinctions

derived from spectroscopic temperature and color-temp relation don’t quite match RJCE extinctions

Data quality: Extinctions

• APOGEE data is flowing through reduction pipeline• Much processed through ASPCAP, soon to be routine• Data products available in SAS

• Spectra• summary tables• ASPCAP fits

• Important to be aware of known issues: publication quality TBD?

• We’re interested to hear about unknown issues!

Conclusions

• Hope to implement late summer/fall

• Design work by Nick MacDonald + spare fibers: target + sky fibers

• Main goals:• Calibration star

targets• RV monitoring• Bright star survey

NMSU 1m – APOGEE Fiber feed

Challenge: robotic spectroscopy – acquisition and guiding

• Determine mean M_H of PDF given Te, log g, [Fe/H] and adopted errors, with prior on IMF and SFH; use H and WISE reddenings to get dist (similar to Santiago et al SEGUE distances)

• Coming: priors on spatial distribution, selection function, better error treatment (e.g., Burnett and Binney)

Preliminary distances and abundance gradients with APOGEE : Michael Hayden, Jon Holtzman (NMSU)

• Inner disk coverage with ASPCAP still limited but should be available soon

• Initial gradient results comparable with SEGUE (Cheng et al.)

Dec 8, 201122

Line Widths/ Resolution (1)

ThAr

UNe

Sky

Solid: fiber 150Open: fibers 10/290

Each point is median of results from lots of frames (few per night for lamps, one per night for skies)

Simple Gaussian fits

Some lines appear to be resolved (blends)

• Wavelength range/dispersion as expected• Resolution close to or achieves SRD specification of R~22500

(significant improvement from pre-shutdown in red)

Dec 8, 201123

Line Widths/ Resolution (1)

ThAr

UNe

Sky

Solid: fiber 150Open: fibers 10/290

Each point is median of results from lots of frames (few per night for lamps, one per night for skies)

Simple Gaussian fits

Some lines appear to be resolved (blends)

• Wavelength range/dispersion as expected• Resolution close to or achieves SRD specification of R~22500

(significant improvement from pre-shutdown in red)

Dec 8, 201124

Line Widths/ Resolution (2)

• Stability and monitoring: line widths appear to be constant in time (mostly?)

• Significant variation across chip not unexpected

Dec 8, 201125

Line Widths/ Resolution (3)

Issues•Moderate undersampling in blue•Resolution varies across chip (not always best in center!)•Beware sky lines for LSF characterization?Caveat:•Measurements from uniformly illuminated fibers•Possible LSF measurements from fits to telluric lines, or perhaps asteroid (or solar analog)

Dec 8, 201126

S/N and throughput

• SRD calls for S/N=100 at H=12.2 in 3 hr exposure per pixel at 1.6 microns, airmass=2, seeing=1.5”, clear conditions. Pixel is taken to mean a “Nyquist sampled pixel”; since dithering provides spectra at ~2X Nyquist sampling, this means S/N=70.7 per dithered pixel.

• S/N calculated for all 500s exposures taken to date, using median S/N for 12<H<12.2, where noise comes from noise model

• Some S/N verification from looking at “featureless” region of tellurics; ratio of observed/expected scatter generally between 1.0 and 2.0

• Dotted lines are SRD requirement assuming signal-limited exposures

Note that S/N in excess of SRD doesn’t necessarily imply we are overexposing because we could reduce number of exposures

Dec 8, 201127

Persistence

APOGEE detectors have noticeable persistence issues•Blue detector has “superpersistence” over top ~1/3rd

Some characterization done in spring, but little or no effort since then

Attempting to minimize as much as possible:•Cold shutter installed and in routine use•Dome flats are only nighttime frames taken that aren’t science frames; taken AFTER science exposures of a plate, followed by two short dark•Evening cals minimized, and taken early, with more complete calibration set taken in morning; kept cal exposures to a “minimum”

Dec 8, 201128

Dither pair accuracy

• Modifications made to mechanism algorithm between runs 2 and 3 made significant improvements!

• SRD spec: 0.5 micron accuracy or 0.028 pixels (dotted lines)

• Algorithm for combining dither frames should handle arbitrary shift size so long as it is well measured, although deviations from 0.5 lead to correlated noise issues

• Detailed tests on understanding whether 2 samples are sufficient for full LSF reconstruction have not yet been done (dither combination of monochromatic sources complicated by intensity variations): test on single line looks good

Dec 8, 201129

Observing software

Observing software functional and relatively stable:•ICS software STABLE : some modifications have been made for dithering and for safety interlock. Possible desire to implement additional exposure type keywords•Instrument control MOSTLY through SOP: few items still done with STUI scripts: cartridge change, evening cals, morning cals. Appears to be going smoothly, but possible desire to incorporate everything into SOP (priority? manpower?)•Python quicklook actor MOSTLY STABLE: handles header annotation of frames, checksum vs raw frames, initiates quicklook, quickred, bundling. Some issues with long term stability, so restarted every day automatically (TBD priority for modification? Manpower?)•Quicklook/webapp for observing support to be discussed later•Morning cron job checks for missing bundled frames, creates MD5 sums for SAS transfer•SAS transfer of raw data running stably during subsequent day

Dec 8, 201130

Observing support

Observers have been given primary contacts:•Frinchaboy: scheduling issues•Holtzman : actor / observing issues•Shetrone: observing issues

Alerts set up for instrument parameters, disk space

Things seem to be running fairly smoothly (from team perspective, but it’s really the observer perspective that is important here!)

Dec 8, 201131

Calibration data

Calibration data is taken on a regular basis:Daily internal cals:•ThArNe and UNe lamps at 2 dither positions•Quartz exposure•3 long darks•Internal flatsMonthly internal cals, plan not yet routine – needs improved procedure/responsible party•10 long darks•10 internal flats•Persistence monitor•Sparse pack quartzOn-telescope calibration data:•Mirror petal flats taken after every cartridge to 1) provide PSFs, 2) provide fiber to fiber throughput•Sky frames taken once per night (4 150s exposures) for LSF monitoring (but note possible issues with these)•Every plate has sky and telluric star fibers for sky subtraction and telluric correctionOverall approach has been relatively conservative, however, little or no impact on efficiency from internal cals, some from on-telescope cals

Dec 8, 201132

Calibration data (2)

Calibration data suggests instruments appears to be relatively stable:•Wavelength cals •Flat fields •Darks •LSFs Appears that current internal calibration data is sufficient (and taking more that might possible be relevant would probably imply taking an efficiency and persistence hit)

Reduction pipeline currently not doing great job at sky subtraction and telluric correction•Relatively little effort dedicated to date•Not yet clear if calibration data is sufficient•Have been using 35/35 sky/telluric as opposed to original plan of 25/25

Dec 8, 201133

Data inspection / QA

Data is now being routinely processed (although almost certainly not optimally!)•quick manual construction of pre-processing file (check on plate/exposure assocation, cal frames, etc.) done morning after observing (~15 minutes) based on observing logs•Raw data cubes downloaded from SAS, generally by night following observing•Reduction started second day after observing, processed through 3D->2D, 2D->1D, sky correction, dither combination, RV measurement in 6-8 hours per plate, all plates reduced simultaneously on multiple processors•Daily calibration data processed in parallel (not yet quite routinely implemented)

Dec 8, 201134

Data inspection / QA (2)

Data reduction pipeline produces web pages for quick inspection / QA•Summary MJD QA page:

• Highlights any frames for which no reduced frame exists• Frames with bad checksums• Missing sequence numbers• plot of zeropoints for all exposures, continuum sky levels for all exposures• Table of all science exposures, with derived S/N, zeropoints, attempts at

identifying missing and faint fibers•Summary web page for each plate:

• Zeropoint, continuum sky, and S/N plots vs mag for every exposure• Maps of zeropoints, sky as f (zeta, eta)

•Web page of all dither-combined spectra and error spectrum for each plate• Links to RV page showing best matching spectrum

Some of these are getting looked at now Need to develop specific checklist of what should be looked at (may motivate some reorganization of information on pages), but I think we are very close

This checks mostly issues with conditions, data issues, and data reduction issues, thus indirectly instrument health

Dec 8, 201135

Data inspection / QA (3)

Instrument health montoring through daily calibration monitor output: not yet fully implemented, but many pieces in place (few days work)•Wavelength calibration: start/end wavelengths, wavelength solution vs. fiducial solution•LSF: line width monitoring•Dither pair separation (should be included in plate pages)•Daily flat vs fiducial flat•Daily dark vs fiducial dark•Cartridge flats vs fiducial (perhaps should be included in plate pages)

Need to develop specific checklist of things to look at and simple web interface (high priority, getting close to implementation)

Increment of survey reduced data statistics•RV histogram for plate•Sky map of observed plates•S/N histogram•Delta mag histogram

Dec 8, 201136

Possible areas for improvement

EFFICIENCY•Shortening default exposure time•Implement variable exposure time based, e.g., on quicklook•Use of twilight time•Additional cartridges or nighttime replugging of a cartridge•Reduction of number of sky and/or telluric fibers•Reducing/eliminating daily sky exposures•Developing refined S/N criteria, e.g., based on actual ASPCAP results

ANALYSIS•Develop methods for addressing persistence•Improve extraction methods•Improve telluric correction, and perhaps, sky subtraction•Improve construction of dither-combined frames•Implement combination of multiple visit spectra