Multispectral Format from Perspective of Remote Sensing Rulon E. Simmons [email protected] (585) 253-5382
Dec 14, 2015
Multispectral Formatfrom Perspective of
Remote Sensing
Rulon E. Simmons
(585) 253-5382
Outline
• Color Issues– Steps to color processing– Color Accuracy in Remote Sensing– Handling color display of non-visible bands
• Format issues– Similarities and differences with other
multispectral applications– XML for Metadata– Compression
Color Issues
Contributing Persons:
Rulon Simmons
Tim Elder
Scott Bennett
Michael Vaughn
Color Processing Approach*
*Note: This is a standard approach used with digital photography.
Display
OutputPreparationRendering
ColorCorrection
Pre-Processing
DRAChannel Balance
RGB’ XYZ XYZ LABTTCMTFCLAB XYZ’
XYZ’ RGB”Gamma
RGB raw
Pre-Processing
Black Point
Calibration Point
Pre-Processing (Calibration from Digital Counts to Reflectance)
• Empirical Line Method*– Using one or more calibration patches– Using best guess of reflectance of one or more
objects
• Atmospheric modeling (e.g., MODTRAN)
*Note: The ELM method of calibration is applicable to any discipline, not just Remote Sensing.
Color Correction(Causes of Color Inaccuracies in Remote Sensing)
• Multispectral band passes in the visible region not equivalent to HVS response functions – IKONOS, QuickBird
• Skylight illumination– Common phenomenon: Materials in
shadow appear bluish
• Dynamic Range Adjustment (DRA)– Varying approaches produce different
color presentations
Color Correction(Problem 1: Different Spectral Sensitivities)
E
E E
II I
I
E = Eye SensitivityI = IKONOS Sensitivity
Color Correction(Ikonos Vs. HVS Color Example)
IKONOS Image Simulated HVS Image
Color Correction (GretagMacbeth Color Checker: Truth & IKONOS Simulation)
Color Correction(Eckerd’s Blue Roof Turns Purple)
HVS IKONOS QuickBird
Color Correction(Image Example and Simulation of IKONOS Color)
Ground Truth:Digital Camera
Simulated IKONOS Color
Color Correction(Problem 2: Shadow Illumination)
• White reference chip in direct sunlight– Used to calibrate the ASD
spectrometer
• White reference chip in shadow– Illuminated by light scattered
by the atmosphere
Rendering
• Convert from XYZ to LAB (to get to a color appearance space)• Apply a Tone Transfer Curve (TTC) to the “L” channel to stretch
midtones while compressing highlights and shadows
• Apply Modulation Transfer Function Compensation to the “L” channel to sharpen image
• Convert back to XYZ
Tone TransferCurve
Luminance (in)
Lum
inan
ce (
out)
Output Preparation
Phosphor Gamut will allow reproduction of most colors seen by IKONOS, but many colors seen by the eye cannot be reproduced by IKONOS.
SO HOW CAN YOU DO COLOR CORRECTION?
Color Correction Approach
• There is no perfect solution, however, all of the problems are addressed by creating a transform that minimizes the Delta E between a set of color patches as seen by the eye and by the sensor
– Use real patches were possible
– Use synthesized data based on system specs
Color Correction(Beyond Visible)
• Near IR and other non-visual spectral regions can be handled in the same way as shown on previous slide. (Note: UV is rarely used in Remote Sensing because it is heavily absorbed by the atmosphere.)
• The transformation seeks to minimize the Delta E between an acquired or synthetically generated image of a set reference panels and some aim colors
• For IKONOS, a “false-color IR” image is produced by mapping the IR channel to the red display, red to the green display, and green to the blue display
Color Correction (for “hyper”-visible imagery)
• Currently in Remote Sensing there is no standard method of mapping color for systems that have more than three spectral channels covering the visible range of the spectrum– Current visualization packages just display
three bands (either user selected or chosen by the program)
– A useful option would be to smartly combine the bands to make an image that is as close as possible to what the human visual system would see
Format Issues
Contributing Persons:
Rulon Simmons
Bernard Brower
Similarities and Differences between Remote Sensing and other
Applications of Multispectral Imaging
TOPIC Textiles Paints Digital Cameras Remote Sensing
Need to Transmit Data Yes Yes Yes YesNeed to Store Data Yes Yes Yes YesNeed for Color Accuracy Yes Yes Yes YesNeed to Compress Data No No Yes YesNeed to Sharpen Data No No Yes YesNeed for Geolocation Data No No No YesNeed for Format Standardization Yes Yes Yes YesCurrent Standards TIFF TIFF TIFF GeoTIFF
JPEG NITF / BIIF (NATO)HDF5
Future Standards J2K J2K J2K J2K
Reconciling Differences• Consider requirements for all disciplines to be a
subset of requirements for Remote Sensing
– Note 1: This approach does not require that all Remote Sensing issues be addressed in the first release of the standard as long as it is extensible at a later date
– Note 2: A basic set of Remote Sensing requirements may not be too difficult to accommodate
• Build upon a standard that will be acceptable to all parties in the future
• Standardize an approach to recording metadata that can be used within any file format
Standard Approach to Metadata (Why Use XML?)
• Standard method of coding metadata
• Can be used regardless of format
• Is human readable as well as machine readable
XML for Pre-Processing
<PRE-PROCESSING>
<DRA>
<PARAM NAME=“GAIN” VALUE=“1.5/>
<PARAM NAME=“OFFSET” VALUE=“10”/>
</DRA>
</PRE-PROCESSING>
XML for Size
<SIZE>
<PARAM NAME=“NBANDS” VALUE=“4”/>
<PARAM NAME=“NROWS” VALUE=“1000”/>
<PARAM NAME=“NCOLUMNS” VALUE=“1000”/>
<PARAM NAME=“NBITS” VALUE=“12”/>
<PARAM NAME=“SEQUENCE” VALUE=“BIP”/>
</SIZE>
XML for Location
<LOCATION>
<PARAM NAME=“COORD PROJECTION” VALUE=“UTM”/>
<PARAM NAME=“LAT” VALUE=“XX.XX”/>
<PARAM NAME=“LONG” VALUE=“YY.YY”/>
</LOCATION>
XML for Color Correction
<COLOR MATRIX>
<PARAM NAME=“RGB TO XYZ” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>
</COLOR MATRIX>
XML for Rendering
<RENDERING>
<PARAM NAME=“XYZ TO LAB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>
<PARAM NAME=“TTC” VALUE=“X1, Y1, X2, Y2, X3, Y3, …”/>
<PARAM NAME=“MTFC” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>
<PARAM NAME=“LAB TO XYZ” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>
</RENDERING>
XML for Display(In general would not be included in metadata.)
<DISPLAY>
<COLOR MATRIX
<PARAM NAME=“XYZ TO RGB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>
<PARAM NAME=“LAB TO RGB” VALUE=“X1, X2, X3, X4, X5, X6, X7, X8, X9”/>
</COLOR MATRIX>
<PARAM NAME=“GAMMA” VALUE=“2”/>
</DISPLAY>
Compression
• Compression is required for some, but not all multispectral applications
• JPEG 2000 is an emerging international standard that is gaining wide acceptance (can be used in lossy or lossless mode)
• JPEG 2000 can be used to stream very large data sets in real time to small computing devices such as PDAs (using the feature that the whole image does not have to be decompressed before it can be sent)
• JPEG 2000 can be accommodated within the current NITF/BIIF standard, making it attractive to many currently doing Remote Sensing
Conclusions / Recommendations• Use a four-step approach to color management currently used with digital
photography
• Non-visible channels can be displayed in predictable ways using standard color science
• “Hyper”-visible channels can be combined to give a better representation of true color (from HVS perspective)
• Multispectral format requirements for Remote Sensing are more similar to than different from other applications
• Select a standard that will be acceptable to all disciplines, taking into account widely-used standard formats and one that will likely be used in the future.
• Use XML as a standard for recording metadata