IST-083 Symposium Progressive Still Image Transmission over a TDL Network A Study Case: JPEG2000 Compressed Images over a Link 16 Network Author: Cdr,
Post on 29-Mar-2015
215 Views
Preview:
Transcript
IST-083 Symposium
Progressive Still Image Transmission over a TDL Network
A Study Case: JPEG2000 Compressed Images over a Link 16 Network
Author: Cdr, Manuel Martinez, PHD (SPN)Co author: Prof, Antonio Artes, PHD (UC3M)Co author: Maj, Roberto Sabatini, PHD (IT AF)
Scope
• Operational Requirements
• Image Technical Requirements
• JPEG2000 Standard
• Link 16 Case Study
• Some Examples
Imagery Operational Requirements
• Target images required before authorizing weapon release.
• Strike Aircraft aircrew using an image depicting a time critical target (TCT) for identification.
• Damage Assessment.
• UAV imagery supporting urban/roads battlefield.
• To develop a “composite tactical picture” available to all Joint Units by using “multimedia” information content provided by TDLs in support of network centric warfare.
Why in TDLs?
Why in TDLs?
• Extend TDLs concept from “textual data” to “multimedia elements” including still image & stream-video for a variety of image types– Visual– IR/FLIR– SAR
• Each image shows different characteristics based on the sensor performance and image type.
• Don’t ask for new “spectrum allocation” in dense urban areas– Use the current TDL’s (Link 16) allocated spectrum.– Use current AJP capabilities provided by TDLs (Link
16) to avoid jammers /civilian interferences.
“Still Image” Compression Requirements
• Compression Efficiency.– Excellent performance at low or very low data rate (disadvantage
links).• Random access to the “code-stream”.
– Embedded code stream that allows to extract the image in different ways.
– This shall allow to reconstruct an image based on a progressive scheme.
• Scalable.– Resolution.– Quality: definition of QoS and quality layers.– Components: color, B&W,…– Position.
• Lossless and Lossy Compression.• Open architecture and standardized. • Definition of Region of Interest (ROI), Zoom, Tiling,..• Robustness in channel error conditions.
Current Efforts. DLWG
• DLCP NUMBER: ML277-054-P33-US-M32-R6• J16.0 Image Transfer Message definition.• NPG 11 dedicated for Imagery• Minimum 32 Time Slots required per frame.• Recommended P4 Packing Limit (Link 16).• Supplement messages from surveillance and
control.– Secondary role.
• Imagery and image amplification information defined in the standard.
• Voice coordination required.
Current Efforts. DLWG (2)
• Fulfill NATO Secondary Image Format (NSIF).– STANAG 4545, Edition 1 Amendment 1 dated
14 April 2002 – NATO Secondary Imagery Format (NSIF) Version 1.0.
• Compression Algorithms: JPEG & EagleEye.• Image packet data of 45 bytes per J 16.0
message.• Geo-position Information is included as a non
displayable image segment.• File Transfer protocols: one way & two ways.
Proposed Still Image Compression Standard:JPEG2000 (1).
• Defines a “canvas coordinate system” based on tiles – Since different components can have different sizes.
• Each tile is DWTd Provides “spatial” access to the image (can be operationally required).– Discrete Wavelet Transform (DWT).
• Wavelets provides a robust tool for signal and image processing.
• More accurate time-frequency representation Can handle signals with discontinuities.
• Transform Domain: each sub-band of every resolution level is partitioned into blocks/precints/packets.
Proposed Still Image Compression Standard:JPEG2000 (2)
• Embedded block coding algorithm (EBCOT): one code-stream can be decompressed in many ways.– Each sub-band image divided into blocks that
are coded independently. Introduces quality for “layers” concept. – Progressive compression.
• By quality / resolution / components and position
• State of the art compression efficiency.
JPEG2000 Canvas Coordinated System
T19T18T17T16T15
T14T13T12T11T10
T9T8T7T6T5
T4T3T2T1T0
Xsiz
XOsiz
XTOsiz
Ysi
z YO
siz
YT
Osi
zXTsiz
YT
siz
(0,0)
JPEG2000 Block Diagram
ICT[RGB YCbCr]
DWT_Irr[Num Niv, D]
DZQ
RCT[RGB Y’DbDr]
DWT_Rev[Num Niv, D]
Ranging[eb]
Offset
ROI[Max-Shift, U]
Transf_Color Transf_Wavelet Quantif
Image Samples Quantification Indexes
Muestras Sub_bd
EBCOTCoder
ICT: Inter-component TransformationDWT: Discrete Wavelet Transform: Daubechies 9/7 (NR) & Daubechies 5/7 (Reversible) DZQ: Dead Zone Quantification
JPEG2000: Wavelet Transform
•Sub-band Images from low & high pass filtering and sub-sampling•Same samples as in original image•Transform is reversible•Sub-bands are quantized
(Image from David Taubman)
JPEG2000: Wavelet Transform (2)
•Multi-resolution representation of the image•Image is LL/HL/LH/HH filtered in various levels
•LL: Approximation of image (low frequency)•LH: Horizontal details.•HL: Vertical details.•HH: Diagonal Details (high frequency)
Note: Image by Konstantinos Kamaras Thesis, March 2002.
JPEG2000: EBCOT(Embedded Block Coded with
Optimal Truncation)• EBCOT defines quality layers: each block
stream is optimally truncated.– From low quality imagery (coarse
quantization) to the best image (finest quantization)
JPEG 2000 Standard. Different Parts
• Part #1: Core coding & basic technology (JP2)• Part #2: Extensions (JPX)• Part#3: Motion JPEG2000 (MJ2)• Part #5: Reference SW (Java & C)• Part#8: Encryption and Watermarking (JPSEC)• Part #9: Interactive Protocols and API (JPIP)• Part#11: robust communication for wireless
(JPWL)• Part #12: ISO Base Media File Format (Common
with MPEG-4)
JPEG2000 Code stream (JP2)
• JPEG2000 coder produces an embedded code-stream that can be decoded at any length to produce an image.– From 0 bit rate to the maximum (lossless).
• JPEG produces an independent code-stream per “target bit-rate”.– Implies multiple coding passes less efficient.
• Embedded code stream support random access to image properties:– Region of Interest (ROI).– Zoom-in / out.
JPEG2000 Code stream (JP2)
MainHeader
Tilestream
Tilestream
Tilestream
EOC
Tileheader
packetstream
packet
packet
SOC
SIZ
COD
QCD
+
COCQCCRGNPOCPPMPLMTLMCRGCOM
SOT
CODQCDCOCQCCRGNPOCPPTPLTCOM
SOD
+
OptionalMarkers Packet 0
TileHeader
Packet i
Packet j
Packet k
Packet l
Packet m
Tile Stream: t
TileHeader
Packet 0
Packet i
Tile part 0
Packet l
Packet m
Tile-partHeader
Tile part N-1
POC
PPT
SOT
PLT
COM
SOD
TileStream
Geo Referenced Images(Geography Markup Language)
• JPEG2000 doesn’t specify mechanism for geo-referencing the image, describing the sensor characteristics,…
• JPEG2000 provides room for “boxes” containing arbitrary XML data.
• GML (by Open Geo-Spatial Consortium) provides geo referencing information as XML encoded metadata.
• GML & JPEG2000 are interoperable and compatible.
JPEG2000/JPEGComparative Analysis
• Progressive lossy to lossless compression– JPEG: not integrated capability.
• Random Access to the code-stream:– Implementation of Region of Interest capability.– Better data rate control.
• JPEG2000: better compression efficiency.(Image1 & 2 are referenced in paper)
Bpp 0.125 0.5 2.00
Img1 JPEG 24.42 31.17 35.15
Img1 JPEG2000 28.12 32.95 37.35
Img2 JPEG 22.60 28.92 35.99
Img2JPEG2000 24.85 31.13 38.80
JPEG2000/JPEGComparative Analysis in Noisy Channels
• Better performance in noisy channels. NOT EDC Technique Implemented WORS CASE SCENARIO better if R-S is modeled.
– See example: BER: 10^ (-4).– Error Symbols in the code-stream:
• JPEG2000: 28• JPEG: 18
• Method:– JPEG2000 code stream is built (*.J2C)– A Binary vector representation of the code stream is developed. – A binary vector with the same BER channel is generated: x=(rand(tamano,1)<=ber)– An exclusive OR is made between both vectors for generating a LINK-16 channel corrupted
JPEG2000 code stream.– Image reconstruction with “Kakadu”
JPEG JPEG2000
Requirements Flow Down: From Design to Operation
Link 16NetworkRqmts
ImageryRqmts
-One / Two ways-Image Type:SAR/FLIR/Visual-Quality Min_PSNR
-% TS Imagery-Relé-VoIice- AJP
NETWORK DESIGN PHASE
OPERATION PHASE
Link 16TX (BER)
JPEG2000CodingPARAMETERS
-Link 16 (TSR)-Interactive Mode (TWO WAYS)Voice / data coordination (service channel)
ImageryCompression
Note: Link 16 Service channel: 8 TS assigned if required (Free Text)
Req
uire
men
ts f
low
dow
n
JPEG2000 Compressed Image Over Link 16
• Method proposed– Transmit the most sensitive part of JPEG2000 code-stream with
maximum Link 16 Anti-Jamming Protection: STD Packing Limit.• Headers/markers/first layers/tiles/
– Rest of the image with a less robust Packing Limit. (less AJP) • JPEG2000 Coding option: progressive by quality.
– Quality Layers. • Link 16.
– “free text” unformatted messages/ JXX.YY standard format message.
– NPGs: from 1..4 (NPG 11 –assigned- & spare NPGs: 15 to 18)– Max TSBs: 64 per Link 16 Terminal (48 in TSR)– Time slots available for imagery
• Two study cases: NMT 10% and NMT 5%.
JPEG2000 Quality Layer toTime Slot Allocation Algorithm
2.-Time Slots per NPG: I1: X_TS I2: Y_TSI3: Z_TSI4: T_TS
4. Image SizeM x N bpp
1. NPGsavailable for Imagery1..4
3.-Bit Rate Computing (bits)
-Layer #1: TB#1=X (TS)*PL_Std / TS................
-Layer #N: TB#N =i (TS) * PL_j / TS
5.-BPP computing
BPP#1=TB#1 / MxN....................
BPP#N=BPP#(N-1)+TB#N / MxN
Definition OriginalImage
ORIGINAL IMAGELINK 16 RESOURCES
6.-JPEG2000:LRCP
Match Link 16 available resources with JPEG2000 LRCP “quality layers”
Matching Numbers: An Example
• Step 1 &2. Define the bit rate iaw network design: 32+x+y+z (time slots available for imagery)
• Step 3. Bit rate layer#1: 32 TS *XXX bits/TS_PL=YYY bits.
• Step 4. Size of the image=640 x 437 pixel.• Step 5. Bit rate layer #1: r_1=YYY/640*437= 0.02574
bpp (P2DP).• Step 6. Layer #n: r (n)= r(1) + (r), with (r ) iaw time
slots allocation by network design• Step 7. Image JPEG2000 coded LRCP iaw layer(n) bpp.
– Nota: Each additional quality layer can be allocated to a different
NPG or TSAB of the corresponding NPG.
Communication Architecture& Protocol
• Sensor Node (SN)– Captures the image.– Image Preprocessing & Compression:
filtering/PSNR/Quality Criteria.– Build Geo referencing code stream (If required).
• Imagery Management Node (IMN)– Receive the image and request retransmission or
changes in quality based on R-S Symbol threshold. (Errors in channel)
– Request for ROI.– Request for more time-slots if in TSR.– Reconstruct the original image based on code stream
received and operational/tactical requirements.
Protocol
Mgmt Unit Sensor Unit
CS: Msg: “Req_Img”
Capture of Img
Preproc Img: filtering, zoom, etcCompres_Img
CP: -Preparation Tx link 16 channel.-Msg: “Tx_Img”
CP: RX_ImgProces_Img
CS: Msg: “Mod_Img” “change_prm”Interactive Mode
CS: Msg + Q:“Mod_Img”
Modify Compression (+ F_C)CP: “(Tx_Img)´”
(Proces_Img)´
CP: (Tx_Img)´´
CS: Msg: “Balance_TS”TSR processing
(TSR)
CS: Msg confirmation
Post Proces: PSNR If > TSendIf <=T Q.M.)
CS: “Param_Img”
Quality Mngmt)
Note: All joint units receive the same code-stream but only IMN can interact with Sensor Node unless otherwise specified.
Quality Management (QM).(In terms of PSNR)
Reconstructed image JPEG2000 Decoded iaw Time Slots available+Original Image.
PSNR>T YN
Transfer toInteractiveto modifyCompression param.
EXAMPLE: -Size-Zoom-TILES-ROI
PSNR / MSE
*.J2C
Threshold: T(OperationalRequirement)
(1)
(2) (3)
Ready toTX L16
InteractiveMode
Y
Request moreLink 16 T-S :
AP 257-259-261J0.6 OR J0.7
(TSR)
N
Reduce T
N
Service MSG
Ready toTX L16
Y
Concept. Example 1
JPEG2000Coder
J2C bitstream
#1 #2 #3 #4
Mode LRCP:Quality Layers
#1
#2
#3
#4
NPG_V1 (Dedicated)
NPG_V2 (Dedicated)
NPG_V3 (TSR)
NPG_V4 (TSR)
Alg #1
Alg #2TSR
L16Transmission
Layers jpeg2000:
•Picture JPEG2000 coded•Progressive LRCP Mode: 4 layers•NGP allocation (if available): Packing Limit and Access Mode
•Layers 1 & 2: Dedicated Algorithm #1•Layers 3 & 4: TSR Algorithm #2
•Link 16 Transmission
Example 2: P/L increased by layer
JPEG2000Coder
Code streamJPEG2000: *.J2C
#1 #2 #3 #4
Progressive #1:XYZT:
#1
#2
#3
#4
NPG_V1 (Ded)
NPG_V2 (Ded)
NPG_V3 (TSR)
NPG_V4 (TSR)
Alg #1
Alg #2
TXL-16
jpeg2000 layers:
#1’ #2’ #3’
#1’
#2’
#3’
1 2 Note:1.- QL Image 12.- QL Image 2
Img #1(FLIR_Land)
Img #2(FLIR_Naval)
Img #1.j2c
Img #2.j2c
JPEG2000TransCoder
(MGMI)
TS available: 128•Img#1: 3 layers (96 TS/27.26 dB)•Img#2: 1 layer (32 TS/23.21 dB)
JPEG2000 & Link 16 Combined Error Detection and Correction Tecniques
Bit to Chip for MSK Modulation
Spread Spectrum: FH-DS
Symbol interleaving
Jitter at the start of the MSG
EDC Techniques: -Reed-solomon-Polinomial
LINK 16
JPEG2000Error D&C
Packet Organization:Markers: SOP-EPH
SEGMARK-ERTERM-RESTART
MSG Type-Std-P2SP-P2DP-P4
Block Coding
PacketSequentialOrganization:-Resincronization.
Error Resilient Mode (Symplified).• JPEG2000 Coded Image: Always include SOP-EPH
markers in the code-stream.Much better syncronization mechanism.
• Error Management Protocol Implemented.• Monitor Link 16 Parameter MER (Message Error Rate)
– Provides Reed-Solomon Coder Errors and Erasures.
“If MER >Threshold (iaw AJ enviroment): Retransmit same packet with a more robust P/L”.– Increase AJP.– Decrease BPP.
Commitment: Binary Rate AJ Protection
Error Recovery. An Example (1)
Pq#1 Pq#2 Pq#3 Pq#4 Pq#N....
STD
X- TS
P2DP
Y- TS
P4
Z- TSCode stream partFor Quality of Service
P4_NEDC
Q- TS
JPEG2000: j2c code-stream
LINK 16
X-TS
Packetwitherrors
Time Slot with errors
TS retransmitted with PL STD
..............................
JPEG200 Code-stream
#Bytes P/LResilient Mode # Time Slots P/L No resilient Mode #Time Slots
MAIN HEADER 96 STD 4 STD 4
TILEPART #1 12 STD 1 STD 1
PACKET #1 HEADER 11 STD 1 P2 1
PACKET#1 13 P2 1 P2 1
……………….. …. …………… …………. ……………… …
TOTAL Time Slots 25 24
1 packet header corrupted +1 TIME SLOT
Error Recovery. An Example (2)True JPEG2000 Coded Image
Trama Codificada JPEG2000
#Bytes Packing Limit Resilient # Time Slots Re-TX Delta #Time Slots
MAIN HEADER 96 STD 4 STD =
TILEPART #1 12 STD 1 STD =
PACKET #1 HEADER 11 STD 1 STD =
PACKET#1 13 P2 1 STD (0.462)= 1 (+0)
PACKET #2 HEADER 17 STD 1 STD =
PACKET#2 38 P2 1 STD (1.3511)= 2 (+1)
PACKET #3 HEADER 17 STD 1 STD =
PACKET#3 95 P2 2 STD (3.3777)= 4 (+2)
PACKET #4 HEADER 17 STD 1 STD =
PACKET#4 207 P4 2 P2DP (3.68)= 4 (+2)
PACKET #5 HEADER 17 STD 1 STD =
PACKET#5 392 P4 4 P2DP (6.96)=7 (+3)
PACKET #6 HEADER 17 STD 1 STD =
PACKET#6 440 P4 4 P2DP (7.82)=8 (+4)
TOTAL Time Slots 25 37 (+12 time slots)
ALL packet corrupted +12 TIME SLOTS
Some Simulations and Results
• Great amount of simulations has been conducted:– One NPG STD P/L (by default)– Two NPGs STD/P2 – Three NPGs STD/P2/P4– Four NPGs STD/P2/P4/P4NEDC
• DLWG: Only one NPG allocated for imagery (NPG11)– But several NPGs available for future use.
• Packing limit can also be changed before each Link 16 transaction via AP: Only one NPG will be required More “host overhead”.
119 TS/3 Layers/ 31.53 dB.
140 TS/ 9 Layers/ 37.97 dBm
FLIR Original Image
32 TS (STD) / 1 Layer/ Cr=706.71
128 TS (P4) / 3 Layer/ Cr=78.5
Change in ResolutionAlgorithm
stepPixels bytes BPP Time Slots. NPGs/PL
Original Image 640x480 921600 1
#1 Resolution: 4 40x30 3600 0.7138 4 NPG V1 (STD-TSR)
#2 Resolution 3 80x60 14400 0.7277 +16 =20 NPG V1 (STD-TSR)
#3 Resolution 2 160x120 57600 0.7851 +34 (TSR) =54 NPG V2 (P2DP-TSR)
#4 Resolution 1 320x240 230400 0.8830 +37 (TSR) =91 NPG V3 (P4-TSR)
Original Resolution 1 Resolution 2 Resolution 3
JPEG In Noisy Channels BER: 10^-4Original JPEG
JPEG 2000 In Noisy Ch. BER: 10^-4
ORIGINAL
JPEG JPEG2000
Original SAR Image: 8 bpp
JPEG2000 SAR Images
1 Layer / 30 TS (P4)/0.05088 bpp/18.84 dB 5 Layers/ 80 TS (P4)/ 0.1384 bpp/ 19.84
JPEG2000 Filtered SAR Images
Image BPP PSNR(Filt)
PSNR(Orig)
T/S P4 (Filtd)
T/S P4(Orig)
Original
8 bpp - -
CompLay #1
0.0505 22.97 18.84 30 30
CompLay#5
0.1395 25.48 19.84 81 80
CompLay#10
0.5143 -- 22.30 --- --
SAR Image median filtered. 5 layersPSNR regarding to filtered image
SAR Image. Region Of InterestLAYERS BPP PSNR db Link-16 Time Slots
Layer 1 0.02783 64
Layer 2 0.083497 64+64=128
Layer 3 0.1948341 10.93 128+32=160 > 154
Layer #1 Layer #3
SAR Image. Tiling
Lay bitrate TS PSNR Tiles
1 0.0139 32 10.39 T0
2 0.0695 96 11.44 T0 a T3
3 0.1252 128 12.60 T0 a T5
4 0.18 144 20.07 T0 a T8
Layer #1 layer #2
Layer #3 layer #4
Summary (1)
• Recent conflicts have shown that real time imagery availability is one of the most demanding capabilities required by operational/tactical commanders.
• Integrating “imagery content” into the “common tactical picture” provides a better target identification, weapon release criteria, damage assessment and surveillance data in support of network centric warfare. Also avoids “request for frequency allocation.”
• Current efforts are focused mainly to support JPEG/EagleEye compressed images packed into J16.0 messages and released in the NPG 11.
Summary (2)• JPEG2000 is a more robust open source standard that produces a
totally embedded code-stream. • Method proposed to send the JPEG2000 image in a antijamming
adaptative way– Most important part is more protected.– Rest of the image in a less protected packing limit.
• Algorithm Proposed to adapt JPEG2000 quality layers to Link 16 available time slots.
• Results shown a very good compression ratio with less than 10% and good performance with 5% of the available Link 16 time slots.
• JPEG2000 also offers a great variety of new capabilities well suited for tactical exploitation such as– ROI– Watermarks– Geo-referencing– More robust error and detection schemes.
Way Ahead
• Develop a technological demonstrator– Plan on going.
• Propose an ad-hoc J message J XX.YY or modify J16.0 specification for including JPEG2000 codification syntax.
• Implement a GML compatible geo-referencing capabilities as a part of XML boxes available at JPX file format.
• Implement security and encryption capabilities iaw JPSEC file format.
• MPEG2000 for stream-video under assessment.• And more…
Some of these new proposals require an “enhance throughput” Link 16 capability.
QUESTIONS ?QUESTIONS ?
top related