Digital Scanner

Digital Capture DevicesHigh End Scanners

Agenda

• The genesis of digital capture devices• Market forecasts, trends, and key conclusions• Digital capture basics• Digital capture devices• System analysis and block diagrams• Programmable logic in digital capture applications• Xilinx Solutions• Xilinx eSP

The Genesis of Digital Capture

The Evolution of Digital Capture

• 1960s - Remote Sensing – Analog capture– Required film return– Slow

• 1970s - CCD invented– Birth of digital capture– Enabled deep-space imaging– Instantaneous– Very expensive

The Evolution of Digital Capture

• 1980s - The relentless advance of Moore’s Law– CCD devices become affordable

• 1990s - The advent of…– Graphical computing– Digital communications– The Internet– Consumer capture devices emerge

Digital Capture Today

• 2000s - Digital Video Consumer Electronics Explodes– DVD, digital cameras, digital camcorders, ILink, HDTV, video phones, DVR,

Firewire, USB 2.0, motion JPEG, web cams, broadband Internet, GPS, etc.

Digital Video Everywhere!

The New GenerationDigital Capture Technologies

Digital Camcorders

DigitalCameras

Scanners

Why Digital Capture?

• Quality– Digital content enables superior display processing

• Filtering, color correction, image enhancement

• Convenience– Digital enables “instantaneous interchange”

• Affordability– No photo processing fees– Ability to leverage semiconductor economies of scale

Market Forecasts,Trends, and KeyConclusions

Video Equipment Forecast

Worldwide Video Equipment Production Revenue Forecast (Millions of US Dollars)Source: Gartner Dataquest (Nov.2000)

2000 2001 2002 2003 2004 CAGR(%)Color TV 28,676 29,218 28,710 28,856 28,609 0.5Digital TV 1,175 2,469 3,813 6,358 10,732 93.5VCR 8,339 7,683 7,313 6,549 6,177 -6.7Analog Camcorder 3,085 2,512 1,875 1,186 924 -22.4Digital Camcorder 3,904 5,426 6,789 8,503 10,465 28.7DVD 4,044 6,254 8,507 10,791 12,232 37.8Analog Set-Top-Box 431 229 136 103 79 -34.2Digital Set-Top-Box 6,179 7,488 8,640 9,728 10,331 16Video Game Console 5,170 6,927 6,117 5,502 5,458 7.8Digital Still Camera 2,205 2,393 2,630 2,818 3,383 17.8Other Video 6,144 6,841 7,435 7,828 8,286 7.5Total Video 69,351 77,438 81,964 88,223 96,677 9.2

Worldwide Scanner Shipment Forecast

Source: IDC (Nov. 1999)

02468

101214161820

2000 2001 2002 2003 2004

Year

Un

its

(M)

Scanner

Scanners - Two Categories

• There are two broad classes of Scanners– SOHO and Office Scanners

• Fairly inexpensive - $50 - $200• 36 bit color (12 bit grayscale), Maximum resolution: 600 x 1200 optical• SCSI, Parallel port, USB

– High-end Scanners• Expensive - $200 - $5000• 8,000 element CCD • 24 bit color to 48 bit color • Maximum resolution: 1200 x 3048 optical, 2400 x 1600 optical, 5,000 optical,

15,000 interpolated • Dynamic range: 3.9, DMax: 4.1 • Largest reflective original: 13.8 x 18.5, largest film size: 13.8 x 17.9 (inches)

Interface(s): SCSI, USB, IEEE-1394• Feeder handles up to 200 sheets of 8.5x11• Scans at rates more than 40 pages per minute

Image Sensor Forecast by Application (K Units)

Area Image Sensor Forecast by Application (K Units)

2000 2001 2002 2003 2004 CAGR (%)Analog Camcorder 8,685 8,142 7,990 7,755 4,157 -15.4Digital Camcorder 6,107 8,056 9,751 10,959 12,793 25.5Security Cameras 9,600 12,100 15,300 19,400 24,500 26.4Digital Still Cameras 12,700 18,100 23,350 29,100 34,920 37.9PC Cameras 9,000 15,000 22,000 30,000 35,000 58.5Toys 2,700 3,800 5,500 7,800 11,100 42.3Cell Phones 100 900 3,100 17,800 59,500 393.9PDAs/Handhelds 40 220 1,100 2,400 5,800 247Automotive 20 75 225 1,550 5,320 303.9Biometric 50 275 730 1,500 2,800 173.6

Source: Cahners In-Stat (Oct. 2000)

Digital Capture DevicesKey Conclusions

• Digital video consumer electronics are growing explosively

• Digital capture segment is growing at 9.2% CAGR per year (Gartner Dataquest)– $69,351 M in 2000 to $96,677 M in 2004

• Consumer growth is fueling more affordable sensors for industrial and non-consumer applications which of themselves are large and substantial markets– For instance, biometrics growing from 50K to 2.8M in 2004– And these segments are even more attuned to the advantages

of programmable logic solutions

Digital Capture Basics

The CCD Device

The CCD device acts as the “eye” of the digital capture device. It literally sees the object in focus

CCD (Eye)

The CCD Device

CCD (Eye)

The CCD device acts as the “eye” of the digital capture device. It literally sees the object in focus

The CCD Device

CCD (Eye)

The CCD device acts as the “eye” of the digital capture device. It literally sees the object in focus

The CCD Device

CCD (Eye)

The CCD device acts as the “eye” of the digital capture device. It literally sees the object in focus

The CCD Device

CCD (Eye)

The CCD device acts as the “eye” of the digital capture device. It literally sees the object in focus

The CCD Device

CCD (Eye)

The CCD device acts as the “eye” of the digital capture device. It literally sees the object in focus

The A/D Converter

• The analog to digital or A to D converter acts as the “optic nerve of the eye” in a digital capture device. It literally converts the object in focus into digital signals that the digital capture system can process.

CCD (Eye)

R

GB

A/DConverter

1010101

0010001

Dots Amazing!

• The captured digital images are comprised of many tiny colored dots

• “Pixel” stands for Picture Element• Images are made

from captured 2D pixel maps– X pixels wide– Y pixels tall

• These dimensionsdefine the resolutionof the image capturede.g. 1024 X 768

These Colored Dots are Called Pixels

Y

X

Zooming In

• Each pixel is comprised of three sub-pixel elements: one each for Red, Green, and Blue, each respectively represented by separate binary (digital) values

Digital Image Processing

Image Processing Functions

• Pixel processing– Color space conversion– Scaling– Color/Gamma correction– Brightness– More colors through dithering

• Frame buffer processing – Contrast enhancement– Shadow enhancement– Sharpness enhancement– Chroma key compositing– Graphic overlay

Basic Image ProcessingColor Space Conversion

• To speed many processing tasks, an alternate color space is often preferable to RGB. Many video and imaging standards use a luminance and color difference color space such as YCrCb or LUV.

• To support real-time image processing, high performance conversion is necessary between the source and target formats– RGB2YCrCb: Converts RGB to Luma - Croma Red - Chroma Blue– YCrCb2RGB: Converts YCrCB to RGB– RGB2LUV: Converts RGB to Luma - U color diff. - V color diff.– LUV2RGB: Converts LUV to RGB

Basic Image Processing Scaling

• Fractionally enlarges the incoming data stream as necessary to match the target display resolution

Basic Image ProcessingColor/Gamma Correction

• Adjusts RGB intensities through correction tables• Required to account for technology specific RGB

characteristics (CRT vs. LCD vs. PDP etc.)

Basic Image ProcessingBrightness

• Increases the RGB intensity to the viewer’s taste

Basic Image ProcessingMore Colors Through Dithering

• Smooths out color transitions (banding) in bit-depth limited displays • Generates patterns of pixels which the eye blends together into

colors the display cannot generate

Advanced Image ProcessingContrast Enhancement

• Adjusts RGB intensities to control the degree of difference between light and dark image areas

Advanced Image ProcessingShadow Enhancement

• Selectively adjusts RGB intensities in order to lighten dark grayscale regions

Advanced Image ProcessingSharpness Enhancement

• Adjusts RGB intensities to sharpen the transition between adjacent color regions

Advanced Image ProcessingChroma Keyed Compositing• Composites 2 images together, replacing a specific RGB

value in one image (chroma) with the data pixel from the other

+ =

Advanced Image ProcessingGraphic Overlay

+ =

Advance Image ProcessingDCT-IDCT

• Wide range of signal and image processing applications– JPEG, JPEG2000, MPEG1, MPEG2, H.261, H.263– Video phone, video conferencing– Progressive image transmission (PIT) systems:

• Teleconferencing• Medical diagnostic imaging• Security services

• Implementation Examples

Digital Capture TechnologySide by Side Comparison

Digital Still Camera Digital Camcorder Flat Bed ScannerStill Resolution 640x480 to > 2832x2128 640x480 to > 1360x1020 600x1200 to > 19200x19200

Video Resolution N/A 500 Horizontal Lines or > N/APixel Count < 1Megapixel to > 4 Megapixels <1 Megapixel to > 4 Megapixels < 1 Megapixel - Linear ArrayFile Format JPEG JPEG, MPEG JPEG

Image Sensor Types

• CCD (Charge-Coupled Device)– Electronic pulses are applied which cause each pixel to

transfer its charge to the next pixel, in what is called a bucket brigade transfer

• Eventually each line of pixels transfers its collection of charges into an output amplifier, where each charge is converted into a voltage

– Uses MOS manufacturing process• CMOS

– Are active devices that read out their charge via transistors placed in each pixel

– Uses CMOS manufacturing process

How an Image Sensor is UtilizedScanner Example

Courtesy: www.pctechguide.com

A light source illuminates the piece of paperplaced on the scanner’s glass plate.Blank or white areas reflect more light.

The scan head moves below thepaper and receives the lightreflected from the paper.

The scanner’s lens passes the light onto light-sensitive diodes which translateit into electrical current.

The light is reflectedby a series of mirrors.

Digital ScannerSystem Analysisand Block Diagrams

Digital Scanner System Data Flow

Image Source

Image Processing- DCT/IDCT, color space conversion, compression, etc.- Gamma/color correction, half-toning, brightness, contrast, sharpness, etc.

High-speedTransport

Microcontroller- Scheduler, task mgr., resource allocator, menu mgr., etc.

PHY- LVDS, BLVDS

CCD through A/D conversion- Captured raw digital RGB values of analog image from CCD

Fast Image Distribution- USB 2.0, IEEE-1394, Ethernet MACs

System Controller- Hardware I/O & memory decoding, synchronization, status, interrupts, etc.

System Transport- DMA access to system memory resources, PCI, local bus, dual-port memories

SystemControl

Image Processing

SystemController

Generic Digital Scanner System

OptionalDigital

Encoding High Speed

I/O

DigitalRGB

Digital

uC

Image Processing

CCD I/FCCD

A/D

PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

Raw Analog Capture

CCD

Creates analog RGB image representation of object in focus

SystemController

OptionalDigital

Encoding High Speed

I/O

DigitalRGB

uC

Image Processing

CCD I/FCCD

A/D

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

A/D Conversion

A/D Converter

Converts analog RGB values tosampled digital values

SystemController

OptionalDigital

Encoding High Speed

I/O

DigitalRGB

uC

Image Processing

CCD I/FCCD

A/D

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

CCDA/D

DigitalRGB

Digital Image Capture

CCD Interface

Buffers and sorts sampleddigital RGB values

SystemController

OptionalDigital

Encoding High Speed

I/O

uC

Image Processing

CCD I/F

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

DigitalRGB

CCD I/FCCD

A/D

uC

System Control

System Controller

Provides system level hardware interface between all functional blocks

SystemController

OptionalDigital

Encoding High Speed

I/O

Image Processing

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

SystemController

DigitalRGB

CCD I/FCCD

A/D

Image Processing

Image Processing

Provides improved system levelperformance via hardwareaccelerated image processing

OptionalDigital

Encoding High Speed

I/O

uC

Image Processing

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

SystemController

DigitalRGB

uC

Image Processing

CCD I/FCCD

A/D

Optional Digital Encoding

Optional MPEG/JPEG Encoder

Encodes digital sampled andstreaming images into MPEG/JPEG compliant format

OptionalDigital

Encoding High Speed

I/ODigital

PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

ROM FLASH DRAM SRAM

MemoryController

Supervision & Control

Microcontroller

Orchestrates proper and timelysystem interactions betweenhardware and software

SystemController

OptionalDigital

Encoding High Speed

I/O

DigitalRGB

uC

Image Processing

CCD I/FCCD

A/D

Digital PHY

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

SystemController

uC

Memory Control

Memory Controller

Provides system level interfacesignals for accessing memorydevices and resources

OptionalDigital

Encoding High Speed

I/O

DigitalRGB

Image Processing

CCD I/FCCD

A/D

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

High-Speed Transport

SystemController

OptionalDigital

Encoding

DigitalRGB

uC

Image Processing

CCD I/FCCD

A/D

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

Digital Transport MACs

USB 1.1/2.0,IEEE 1394, Ethernet

High Speed

I/O

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

High Speed

I/O

High-Speed Transport

SystemController

OptionalDigital

Encoding

DigitalRGB

uC

Image Processing

CCD I/FCCD

A/D

Digital

ROM FLASH DRAM SRAM

MemoryController

Digital Transport PHYs

USB1.1/2.0, IEEE-1394, Ethernet

PHY

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

High Speed

I/O

High-Speed Transport

SystemController

OptionalDigital

Encoding

DigitalRGB

uC

Image Processing

CCD I/FCCD

A/D

Digital PHY

ROM FLASH DRAM SRAM

MemoryController

LVDS/BLVDS I/O

Provides high-speed serial transport of captured and or streaming images

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

High Speed

I/O

Digital Image

SystemController

ROM

OptionalDigital

Encoding

DigitalRGB

uC

FLASH DRAM SRAM

MemoryController

Image Processing

CCD I/FCCD

A/D

PHY

Digital RGB

MPEG, JPEG, TIFF, etc

Digital

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

Digital Scanner System DiagramWhere FPGAs Can Add Value

ROM

FLASH

DRAM

SRAM

Memory Controller

CCD I/F

Image Processing

uC

OptionalDigital

Encoding

SystemControl

High Speed

I/O

DigitalRGB

CCDA/D

Digital PHY

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

Lens

Xilinx Solutions

High VolumeLow Cost

High PerformanceHigh Density

Low PowerLow Cost

The Xilinx Product Portfolio

Software and IP Core Solutions

Virtex, Virtex-E, Virtex-EM, Virtex-II,

Virtex-II Platform FPGA

Spartan, Spartan XL, Spartan-II, Spartan-IIE

XC9500, XC9500XV, XC9500XL, CoolRunner

XPLA3

High PerformanceSystem Features

Software and Cores

Smallest Die SizeLowest Possible

Cost

Low Cost Plastic PackagesStreamlined Testing

Xilinx Spartan Series FPGAs

CLB

IOB

CLB

CLB

CLB

IOB

IOB IOB

CLB

CLB

CLB

CLB

CLB

CLB

CLB

CLB

CLB

IOB

CLB

CLB

CLB

IOB

BRAM

DLL

DLL IOB

CLB

CLB

CLB

CLB

CLB

CLB

CLB

CLB

CL

IOB

CL

IOBIOB

IOBIOB

BRAM

DLL

CLB

IOB

CLB CLB

IOB

IOB IOB

BRAMCLB

CLB

CLB

CLB

BRAMCLB

CLB

CLB

CLB

IOBIOB

IOBIOB

IOBIOB

CL

CL

BRAM

IOB

CLB

DLL

BRAM

IOB

2ns2ns

2ns

CLB Tiles• Fast, predictable interconnect

CLB

Differential I/O• 400 Mbps• LVDS• Bus LVDS• LVPECL

IOB

Delay Lock Loops• 200+ MHz performance• 4 DLLs in every device• Deskew 4 system Clks• Zero-delay clock conv.

CLKIN

CLKFB

RST

CLK0CLK90

CLK180CLK270CLK2XCLKDV

LOCKED

DLL

Dual-Port4KbitBRAM P

ort

B

Block RAM• Up to 64Kbits• 200 MHz

BRAM

System I/O™

• 19 signaling standards• Chip to Backplane• Chip to Memory• Chip to Chip

IOB

Spartan-IIE Technology

Standard System I/O Features

• LVDS, BLVDS (Spartan-IIE)• LVTTL• LVCMOS2• LVCMOS18• PCI 33/66• GTL, GTL+

• HSTL I, III, & IV• SSTL3 I & II• SSTL2 I & II• CTT• AGP-2X• LVPECL (Spartan-IIE)

Memory Controller IP

• Content Addressable Memory (CAM)

• DDR-SDRAM Controller• Quad-Data-Rate SRAM

Interface• Single-Port Block

Memory• Registered single port

RAM

• Registered ROM• Dual-port Block Memory• 200MHz ZBT SRAM

Interface• SDRAM Controller

— Timer/Counter Block — Watchdog Timer/Timebase — Interrupt Controller— 16550/16450/Lite UART— ZBT Memory Controller— SRAM Controller— Flash Memory Controller

— IIC*— SPI*— Ethernet 10/100 MAC*— More to come

* Licensed for a fee

MicroBlaze

• 32-bit fully synthesized RISC processor• Fast

– Twice the performance at half the logic area vs. competition

• Supported by an integrated IP library

Xtreme DSP

• Industry first System Generator for Simulink® bridges gap between FPGA and conventional DSP design flows

• Unique constraint-driven Filter Generator allows optimization between performance and cost

• Power estimator tool (Xpower™) for power-sensitive DSP implementations

• Eleven optimized DSP algorithms (cores) that cut development time by weeks

• DSP features added to ChipScope ILA tool dramatically accelerate hardware debugging time

Video/Image Processing IP

• Inverse Discrete Cosine Transform (IDCT)

• 1-D Discrete Cosine Transform

• 2-D DCT/IDCT Forward & Inverse Discrete Cosine Transform

• JPEG CODEC• FastJPEG Color Decoder• Fast JPEG B/W Decoder

• logiCVC - Compact Video Controller

• RGB2YCrCb Color Space Converter

• YCrCb2RGB Color Space Converter

• RGB2YUV Color Space Converter

• YUV2RGB Color Space Converter

High-Speed Transport MACs IP

• USB 2.0• IEEE-1394• Ethernet

FPGA Standard Features and IPAccelerating Time-to-Market

OptionalDigital

Encoding

ROM DRAMFLASH

Hard Disk

Encoder&

Encryption

Image Processing Digital

Compact FLASH

SRAM

DigitalRGB

Sub-System Controller

Digital DisplayController

Analog DisplayController

BufferMemory

High Speed I/O

LCD

TV/CRT

User

Des

igne

d I/O

User

Des

igne

d I/O

FPGA Capture System Utility

User Designed I/O

User Designed I/O

LVDS, BLVDS

CCDA/D

BufferMemory

SystemControl DAC

Clock Mgmt

Nonvolatile Storage

HighSpeed

Bus

RGB2YCrCb

RGB2YUV

YCrCb2RGB

YUV2RGB

2D FFT2D FIR Filter PCI

DCT

IDCT

JPEG

System I/O

DLL

Storage Controller

ROM Controller

System I/O

FIFO

Dual PortBlock RAM

Distributed RAM

Syst

em I/

O

Syst

em I/

O

MicrocontrolleruC

DLL

DLL

DLL

FLASH Controller

SRAM Controller

DRAM Controller

PHY

DES 3DES

Digital uP or uC Mixed Signal ProgrammableMemory IP Block

Xilinx CPLDs

XL9500 FamiliesHigh PerformanceLow Cost Solution

CoolRunner FamilyLowest Power

Highest Reliability

System Integration and Peripheral Interfaces

CPLD Solutions For Every Need

XC9500 families • Voltage flexibility

– 9500 - 5v / 9500XL - 3.3v / 9500XV - 2.5v

• 36-288 macrocell densities• Ultra-high performance• Low cost• Superior pin-locking

CoolRunner XPLA3 family• Ultra-low power with high speed• 3.3V• 32-512 macrocell densities• Advanced architecture

Summary

• Digital scanners are one of the more popular consumer devices– Wide acceptance in all geographies

• The scanner marketplace is growing at a significant pace– Expected to grow from 10M units in 2001 to 20M in 2004

• Xilinx role in scanners– CCD interface, timing generation, memory controllers, hard

disk interface, display interface, and system control• Xilinx fit

– Primarily used for interface between ASSP chips/chipsets– For external connectivity - Memory, USB, IEEE-1394