Camera Driver Porting MTK SZ 06/08/02
Camera Driver Porting
MTK SZ
06/08/02
2
Outline
IntroductionBase band Camera featureMT6217/18+BackendMT6219+SensorCode Review
3
CameraBase band feature(MT6217)
Jpeg DecoderHas to decode full imageDestination has to be sequential
Multi-layer 4 layers
LCD ControllerNeed an extra Camera Backend
OSDJpeg DecoderLCD ControllerSensorImage ProcessorJpeg Encoder
4
Camera
Base band feature(MT6219)Jpeg Decoder/Encoder
May decode partial image
Mpeg 4 Decoder/EncoderMulti-layer
4 layers
LCD ControllerImage ProcessorNeed an extra Sensor
Support up to 1.3 Mega Pixel (1280x1024)
5
Camera
Camera MT6217 Slave backend
MT6217
PAP1301 (PixelArt)
LCMBase band
Sensor
Camera Backend
6
Camera
CameraMT6217 Master Backend
MT6217
VC0558(ViMicro)
LCM
Base band
Sensor
Camera Backend
7
Camera
Camera MT6219 structure
MT6219
LCM
Base bandSensor
8
Camera Software ArchitectureMMI Task
Camera ApplicationLogic, Interrupt handling
MDI_CameraAPI interface to use camera features
GDIMulti-layer (frame buffer) management
Media TaskInterface to access camera driverCamera Driver
Control Camera backend
LCD InterfaceControl LCM
MMI Task
MDI Camera GDI
LCD InterfaceCamera Driver
HW- (LCM, ISP, MT6218, MT6219...)
MEDIA Task
Camera MMI
9
Camera
Slave backend
MMI Task
MDI Camera GDI
LCD InterfaceCamera Driver
MEDIA Task
Camera MMI
LCMAIT103
Layer 0
MT6218B
Layer 1
Layer 2
10
Camera
Master backendMMI Task
MDI Camera GDI
LCD InterfaceCamera Driver
MEDIA Task
Camera MMI
LCM
Layer 0
Layer 1
MT6218B
Layer 0Layer 1
AIT700
Intermediate Buffer
AIT700 OSD Buffer
11
Camera
MMI Task
MDI Camera GDI
LCD InterfaceCamera Driver
MEDIA Task
Camera MMI
LCM
ISP
Layer 0blt_layer_flag == LAYER 0 | LAYER 1 | LAYER 2
preview_layer_flag == LAYER 0
Layer 1
Layer 2
MT6219
Layer 0Layer 1Layer 2
12
Camera (MT6217)
Files:\custom\drv\cam_module\[prj]\cam_module.c\custom\drv\cam_module\[prj]\cam_module.h
Function:Function:
13
Camera (MT6219)
Camera Task6219 ISP
isp_if.c.h,ae_awb.c.h,
Driverimage_sensor.c.h
Tuningcamera_para.c
AE AWB
6219 ISP
control tuning
14
HW Interface
Signal Bus
VSYNC
HSYNC
SCLK
SDATA
XCLK
PCLK
8 or 10
DATA
PWDN
POWER
6219ISP
Sensor
15
SW Interface
void (* sensor_init)(void)Initialize sensor and power on sensor module.Sensor default Configuration.The first executed API.
void (* sensor_power_off) (void)Clean sensor state and power off sensor module.
16
SW Interfacevoid (* get_sensor_id)
kal_uint8 *sensor_write_idkal_uint8 *sensor_read_id
To report sensor R/W ID.SCCB(I2C) R/W has been customized.
17
SW Interface
void (* get_sensor_size)kal_uint16 *sensor_widthkal_uint16 *sensor_height
To report image sensor resolution.IMAGE_SENSOR_SXGA_WIDTHIMAGE_SENSOR_SXGA_HEIGHT
18
SW Interface
void (* get_sensor_period)kal_uint16 *pixel_numberkal_uint16 *line_number
To report the period of image sensor in a frame.VGA_PERIOD_PIXEL_NUMSVGA_PERIOD_LINE_NUMS
These are the parameters of frame rate, base-shutter and exposure control.
Base-shutter affects the banding effect.
19
SW Interfacevoid (* sensor_capture_setting)void (* sensor_preview_setting)
image_sensor_exposure_window_struct *image_windowkal_uint16 grab_start_x;kal_uint16 grab_start_y;kal_uint16 exposure_window_width;kal_uint16 exposure_window_height;kal_uint16 image_target_width;kal_uint16 image_target_height;kal_uint16 exposure_pixel;kal_uint16 exposure_line;kal_uint8 digital_zoom_factor;
kal_uint8 image_mirrorNormal, H-Flip, V-Flip, HV-Flip.
20
SW Interface
void (* write_sensor_reg)kal_uint32 addr;kal_uint32 para;
kal_uint32 (* read_sensor_reg)kal_uint32 addr;
SCCB R/W interface for CCT access.Sensor address/parameter bit width ?
Micron(9M011,9M111)8bits/16bits(9M111 has 3 page address i.e. 3*0xFF)
OV(OV9650,OV7660)8bits/8bits
21
SW Interfacevoid (* set_sensor_eshutter)
kal_uint16 eshutter;
To set sensor exposure time.Important API associated with AE algorithm, banding.Depends on sensor characteristic.
22
SW Interface
void (* set_camera_night_mode)kal_bool enable;
To turn ON/OFF sensor night mode.Double MAX_EXPOSURE_LINES.Increase sensor global gain.
23
SCCB(I2C)Micron 8bit
Write(PARA to ADDR)
Read(PARA from ADDR)
Micron 16bitWrite(PARA to ADDR)
Read(PARA from ADDR)
S WID PARA1 EA ADDR A S AA WID A 0xF1 PARA2 A
S WID A ADDR A S RID A PARA1 N
S WID A 0xF1 A S RID A NPARA2 E
S WID A ADDR A PARA1 A PARA2 A E
S WID A ADDR A S RID A PARA1 NPARA2A E
24
Image sensor data input
V _Sync
H _Sync
Clk
CMOS sensor data input timing
Fram e cycle
Line cycle
25
Sensor&ISP timing
Sync. Signal
Set VSYNC polarity LOWSet HSYNC polarity LOW.
VSYNC
HSYNC
26
Sensor&ISP timingTiming adjustment
ENABLE_CAMERA_PIXEL_CLKIN_ENABLE;//MT6226 or laterSET_TG_OUTPUT_CLK_DIVIDER(1);//XCLK=MCLK/(1+1)=26MSET_CMOS_RISING_EDGE(0); //PCLK rise at MCLK 0SET_CMOS_FALLING_EDGE(1); //PCLK fall at MCLK 1SET_TG_PIXEL_CLK_DIVIDER(3);//PCLK=MCLK/(3+1)=13MSET_CMOS_DATA_LATCH(2); // PCLK latch at MCLK 2
PCLK
XCLK
MCLK0 1 2 3
27
Sensor&ISP timing
First Valid Pixel/Line
VSYNC
HSYNC
First Valid Pixel
XCLK
PCLK
image_window->grab_start_x = dummy 的 lines,
base on datasheetFirst Valid Line
28
Sensor&ISP timing
Bayer pattern
SET_BAYER_PATTERN(FIRST_GRAB_COLOR)#define FIRST_GRAB_COLOR XX
Gr00
R01
B10
Gb11
Base on datasheet
29
Sensor&ISP timing
Valid Frame
VSYNC Valid Frame
SENSOR_PERIOD_PIXELSHSYNC
HBLANKING
VBLANKING
30
AE/AWB Algorithm
AE/AWB Window
AE “Window size”, “region” and “weighting” are configurable, and reports the luminance.AWB window reports the R, G, B accumulation.
w1 w2 w3
w4 w5 w6
w7 w8 w9
31
AE/AWB Algorithm
AE Process(FW)AE banding factor.
Base-shutter should be at lease 1/2f.Base-shutter=PIXEL_CLK/(2f*SENSOR_PERIOD_PIXELS)
VSYNC ValidFrame
SENSOR_PERIOD_PIXELSHSYNC
32
AE/AWB Algorithm
AE Process(FW)AE computation.
Exposure time(shutter), pre-gain.Linear interpolation.
avg_lum = weighted average of 9 windows.
Exposure time 和 Luminance 不是线性的,要逐次计算AE逼近
Exposure time
Luminance
Y1
Y2Yt
T2 T1Tt
Desired Luminance
33
AE/AWB Algorithm
AE/AWB parameter setting timing(HISR)
Three of them should be set precisely, or AE vibration occurred.
AE_AWB_CAL_PERIOD=3 AE_AWB_CAL_PERIOD=3
VSYNC
AE_GAIN_DELAY_PERIOD=2AE_SHUTTER_DELAY_PERIOD=1
34
Code ReviewPreview
ISP/Sensor CLK setting.Mode/Resolution configurationActive window configuration
CaptureAll you did in “Preview”Mode switching
VGA/SXGAzoom<2x and 2x<=zoom<=4x.
Set ShutterControl the exposure time precisely.Depends on sensor characteristic.
35
Case Study
Step 1: define sensor power enable GPIO pin in Camera_para.c
36
Case Study
Step 2: set I2C read ID and write ID
37
Case Study
Step 3: Implement write_cmos_sensor() and read_cmos_sensor()
38
Case Study
Step 4: set Vsync/Hsync polarity and first grab color
39
Case Study
Step 5: insert initial code to CAMERA_PARA_DEFAULT_VALUE or write it directly in init_OV9650()
40
Case Study
Step 6: implement OV9650_preview()
MCLK=13MHzPCLK = 3.25MHz
If latch point wrong pic will be rad/blue
41
Case Study
Step 7: implement OV9650_capture()
VGA & zoom<20PCLK = MCLK/2MCLK=13MHz
PCLK = 6.5MHz
VGA & zoom > 20PCLK = MCLK/4MCLK=13MHz
PCLK = 3.25MHzincrease dummy
42
Case Study
Step 7: implement OV9650_capture()
SXGA & zoom<20PCLK = MCLK/2MCLK=13MHz
PCLK = 6.5MHz
SXGA & zoom > 20PCLK = MCLK/4MCLK=13MHz
PCLK = 3.25MHzincrease dummy
decrease shutter
43
Case Study
Step 8: implement set_OV9650_shutter()set_OV9650_gain()OV9650_night_mode()power_off_OV9650()
44
Q&A