Tae-Shick Wang; Kang-Sun Choi; Hyung-Seok Jang; Morales, A.W.; Sung-Jea Ko; IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010 ENHANCED FRAME RATE UP-CONVERSION METHOD FOR UHD VIDEO
Tae-Shick Wang; Kang-Sun Choi; Hyung-Seok Jang; Morales, A.W.; Sung-Jea Ko;
IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010
ENHANCED FRAME RATE UP-CONVERSION METHODFOR UHD VIDEO
Introduction(1/2)Motion blur caused by inherent characteristic of LCDFRUC
MCFI consists of ME and MCI.
Introduction(2/2)The motion vector field(MVF) is estimated
between two successive frames.MVF should represent the actual object motion
Several ME methods for MCFI have been proposed to find the true motion based on BMA Multi-size block matching algorithmBi-directional Motion Compensate
Interpolation(MCI)
Motion analysis for UHD video(1/2)
Enlarge SR can span regions which belong to another object but are more similar to the current block
Motion analysis for UHD video(2/2)Full search, sort all MVs for each block in
UHD Increasing order of SAD
Search range(SR) should be kept as small as possible if a reliable MV is initially given
Large block size gives a more correct motion
Proposed ME algorithmThe blocks with similar motion are
merged to an entity.Introduce a segment-based ME algorithm
Divide the frame into several segments Estimate the motion for each segment
Large region can be obtained more reliablyfn: current
frame
Block-based segmentationClassify each block into three patterns:
edge, plane, and texture
Segments are obtained by mergingadjacent blocks with an identical pattern
Gradient calculationFor each block, the gradient is calculated
by using the Sobel operator [14].
Gradient
gradientimage
x
fg x
y
fg y
If large=>significant pixel
[14] R. C. Gonzalez and R.E. Woods, Digital Image Processing, 2nd Edition, Prentice Hall, New Jersey, 2002.
Gradient Direction Histogram
Largest number in the histogram # of significant pixels
gx
gy
= r
Segmentation result
MVF of fn-1
: k-th segment from fn
knS
Segment selectionFind segments whose motion can be
obtained accurately.Temporal consistency of the segment
MV of the segment with high reliability
1. Large segment is more reliably,
2. The pattern of segment is plane or texture
3. has dominant MV pass through Dominant MV : over 70% MVs are identical
4)( TSN knB
knS
knS
knS
Efficient true motion estimation(1/3)Full search with low computation
complexitySSAD (subsample SAD) as matching criterion
# of blocks in the segment
=> sub-sample rate σ
)( knB SN
Efficient true motion estimation(2/3)Determine the search range
Reliable block (RB) : SSAD < 5(B/σ)^2 => the MV’s similarity to the actual
motion of the segment
=> search range
Search range size LSR in the segment:
ε =8
)(
)(knB
knRB
SN
SN
Efficient true motion estimation(3/3)Three stage ME
Refine dominant MV within LSR
Refine the received MV (if average SSAD < 5(B/σ)^2) and within LSR
No initial MV within max SR
Experimental resultThree UHD video sequence:
Toy and Calendar, Table Setting, and TractorCompare Ha’s [2] and Huang’s [10]
methods
[2] T. Ha, S. Lee, and J. Kim, "Motion compensated frame interpolation by new block-based motion estimation algorithm," IEEE Trans. Consumer Electron., vol. 50, no. 2, pp. 752-759, May 2004.[10] A.-M. Huang and T. Nguyen, "A multistage motion vector processing method for motion-compensated frame interpolation," IEEE Trans. Image Process., vol. 17, no. 5, pp. 694-708, May 2008.
Subjective comparison
Objective comparison
Improve quality for the interpolated frame by 2~3 dB
Reduce the computation load.
RC: relativecomplexity
ConclusionThe block-based segmentation method
was confirmed to produce meaningful segment information with low complexity.
The proposed method can also be successfully employed for various applications includingDe-interlacing View interpolation for multi-view video.