TIPS, Mar 19, 2009 - Toma s Dahlen 1 New static DQ masks for NICMOS Tomas Dahlen & Elizabeth Barker TIPS 3/19/2009 QuickTime™ and a TIFF (Uncompressed) d are needed to see t QuickTime™ and a TIFF (Uncompressed) d are needed to see t
Feb 06, 2016
TIPS, Mar 19, 2009 - Tomas Dahlen 1
New static DQ masks for NICMOS
Tomas Dahlen & Elizabeth Barker
TIPS 3/19/2009
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
TIPS, Mar 19, 2009 - Tomas Dahlen 2
NICMOS Data Quality (DQ) extension of calibrated images (*_cal.fits or *_ima.fits) contains information about which pixels may be problematic and should/may be excluded when combining dithered images.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
DQ Meaning0 No known problems1 Reed-Solomon decoding error in telemetry2 Poor or uncertain Linearity correction4 Poor or uncertain Dark correction8 Poor or uncertain Flat Field correction16 Pixel affected by "grot" on the detector32 Defective (hot or cold) pixel64 Saturated pixel128 Missing data in telemetry256 Bad pixel determined by calibration512 Pixel contains Cosmic Ray1024 Pixel contains source (see Section 3.4)2048 Pixel has signal in 0th read4096 CR detected by Multidrizzle8192 User flag16384 Curvature in detector response
• Most flag values are dynamic, i.e., are set during observation/calibration.• Static flag values are DQ=16 (“grot” affected pixels) and DQ=32 (“bad” pixels).• Each NICMOS camera has its own static mask given in the header keyword MASKFILE.
TIPS, Mar 19, 2009 - Tomas Dahlen 3
• The current “post-NCS” static masks were created 2002.
• We now have a large set of calibration images obtained during NICMOS monitoring programs 2002-2008.
• We have used this data to create new static masks for the three NICMOS cameras based on better statistics.
• The temporal coverage also allows us to investigate any changes in the masks with time or detector temperature.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
New static DQ masks for NICMOS
TIPS, Mar 19, 2009 - Tomas Dahlen 4
• We use long dark exposures (>1000s) to identify “Bad” pixels as pixels with a dark current deviating from what is expected.• Hot pixels are defined as pixels with excessive charge compared to surrounding pixels.• Cold pixels have extremely low(/zero) dark current (i.e., “dead” pixels).• To quantify the number of bad pixels we use the recipe in Sosey (2002, NICMOS ISR 2002-001 “ Updating the NICMOS Static Bad Pixel Masks”).
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
DQ=32 “Bad” pixels
1. A median dark image is made using available monitoring dark data after excluding CR affected pixels
2. A smoothed image is made and subtracted from the median image
3. The resulting image is rescaled in units of its rms4. Bad pixels are defined as pixels outside 5
Sosey (2002)
5
Since we have monitored the darks continuously from 2002 to 2008, we can look for changes with time/temperature.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
DQ=32 “Bad” pixels
NIC 1 NIC 2 NIC 3
Old DQ=32 193 (0.29%) 656 (1.0%) 446 (0.68%)
2002 46 17 16
2003 49 21 21
2004 70 28 28
2005 69 24 30
2006 70 24 29
2007 66 29 34
2008 70 31 36
All new 88 (+0.13%) 40 (+0.06%) 42 (+0.06%)
Old DQ=32: Number of DQ=32 pixels in existing mask.For each year the number of additional hot pixels is tabulated.All new: Total number of different pixels flagged as hot during at least one year.
Black dots: number of DQ=32 pixels in existing mask.Red dots: number of DQ=32 after adding new bad pixels
Num
ber o
f bad
pix
els
6
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
DQ=32 “Bad” pixels Plots show pixels that are not flagged in existing DQ mask, but pass the criteria for being bad in this investigation.
Number of pixels that “suddenly” turn bad and can be “saved” by using multiple DQ masks is ~5 in each camera. (In addition a few bad pixels drop below the selection with time.)
We find that having multiple DQ masks to “save” at the most a few pixels per camera is not “worthwhile” and
only a single mask is created for each camera.
7
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
DQ=16 “grot” pixels
• The grot consists of flecks of anti-reflective paint on the detectors (that were scraped off the optical baffles between the dewars due to the expansion of the solid nitrogen in Cycle 7).
• Grot leads to areas with reduced sensitivity.
• Size of grot ranges from 25m to over 100m and since NICMOS pixels are 40m on a side, grot can affect regions of several pixels, as well as a fraction of a pixel.
The largest example of a grot region is the "battleship" feature in NIC 1 which affects approximately 35 pixels.
Flat-field image DQ mask
8
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
DQ=16 “grot” pixels
• The grot affects the incoming light onto the detectors.• Flats are ideal to estimate the effects of grot due to their high “uniform” counts over the whole detector.• To quantify the grot we use the recipe in NICMOS ISR 2003-003 (Schultz et al.).
1. We use a well sampled non-inverted flat-field (F160W filter).2. A smoothed flat image is subtracted from this flat.3. Grot pixels are defined as a pixels deviating more than 4 in the
subtracted image (excluding bad DQ=32 pixels).
9
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.DQ=16 “grot” pixels
Existing New
NIC1 180 170
NIC2 243 123
NIC3 249 113
Table: The number of grot pixels. The number found here is significantly lower than in the existing mask for nic2 and nic3.
To investigate this we look at the relative DQE response of the flagged pixels - this is the pixel value of a ratio image created by dividing the flat image with a smoothed flat image. “Normal” pixel has response=1.
Q: what are the characteristics of the pixels not selected using the new selection?
10
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.DQ=16 “grot” pixels
Existing New
NIC1 180 170
NIC2 243 123
NIC3 249 113
The new selection suggest that grot pixels have response ~< 0.8.The pixels that are flagged in the existing mask and NOT flagged in the new selection have a fairly normal response in the range 0.8-1.2.
Existing New Existing - New
Relative DQE response
11
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.DQ=16 “grot” pixels
Existing New
NIC1 180 170
NIC2 243 123
NIC3 249 113
The new selection suggest that grot pixels have response ~< 0.8.The pixels that are flagged in the existing mask and NOT flagged in the new selection have a fairly normal response in the range 0.8-1.2.
Furthermore, the rms of the previously flagged pixels look normal
Previously flagged
Relative DQE response
Black dots: rms of non-flagged normal pixels
Green dots: rms of previously flagged pixels
Rms look similar, green dots should be OK
We should therefore be able to “de-flag” these
Rms vs. response
12
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.DQ=16 “grot” pixels
Q: Does the number of grot pixels change with time?
A: No.But there are somewhat more flagged pixels in NIC2-2008, but these have fairly normal response 0.7-0.9. We will not include a time dependence in the grot mask (i.e., not create multiple masks).
Existing New 2002 2008
NIC1 180 170 172 174
NIC2 243 123 119 139
NIC3 249 113 113 112
As a final selection, we include all “4 pixels” from above as grot pixels, except those that have a response >0.8 and a normal signal-to-noise.
The new MASKFILEs for the post NCS era will be delivered by end of March -09.
FINAL SELECTION
13
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.New static DQ masks for NICMOS
Summary:Grot, DQ=16 Bad, DQ=32
Existing New Existing New
NIC 1 180 163 193 281
NIC 2 243 119 656 696
NIC 3 249 96 446 448
Existing:
New:
NIC 1 NIC 2 NIC 3
DQ=16, grot pixels