Enhancing the Visual Experience Enhancing the Visual Experience on the Mobile Computing and on the Mobile Computing and Communications Platforms Communications Platforms Achin Achin Bhowmik, Ph.D. Bhowmik, Ph.D. Senior Manager Senior Manager Advanced Video & Display Technology Advanced Video & Display Technology Intel Corporation, Mobile Platforms Group Intel Corporation, Mobile Platforms Group Santa Clara, CA 95054 Santa Clara, CA 95054
41
Embed
Enhancing the Visual Experience on the Mobile Computing ... · •Web video growing in importance in Mainstream Consumer experience Growth in Worldwide Broadband Penetration is ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Enhancing the Visual Experience Enhancing the Visual Experience
on the Mobile Computing and on the Mobile Computing and
Communications PlatformsCommunications Platforms
AchinAchin Bhowmik, Ph.D.Bhowmik, Ph.D.
Senior ManagerSenior Manager
Advanced Video & Display TechnologyAdvanced Video & Display Technology
Intel Corporation, Mobile Platforms GroupIntel Corporation, Mobile Platforms Group
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEINFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTELL®®PRODUCTS.PRODUCTS. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLPRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY AIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND/OR USE OF INTELEXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND/OR USE OF INTEL PRODUCTS, PRODUCTS, INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTINCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, ICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT, OR OTMERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT, OR OTHER HER INTELLECTUAL PROPERTY RIGHT.INTELLECTUAL PROPERTY RIGHT.
Intel may make changes to specifications, product descriptions, Intel may make changes to specifications, product descriptions, and plans at any time, without and plans at any time, without notice.notice.
All dates provided are subject to change without notice.All dates provided are subject to change without notice.
Intel, Intel logo, Centrino are trademarks or registered trademaIntel, Intel logo, Centrino are trademarks or registered trademarks of Intel Corporation or its rks of Intel Corporation or its subsidiaries in the United States or other countriessubsidiaries in the United States or other countries
Actual measurement results may vary depending on the specific haActual measurement results may vary depending on the specific hardware and software rdware and software configuration of the computer system measured, the characteristiconfiguration of the computer system measured, the characteristics of those computer cs of those computer components not under direct measurement, variation in processor components not under direct measurement, variation in processor manufacturing processes, the manufacturing processes, the benchmark utilized, the specific ambient conditions under which benchmark utilized, the specific ambient conditions under which the measurement is taken, and the measurement is taken, and other factors.other factors.
*Other names and brands may be claimed as the property of others*Other names and brands may be claimed as the property of others..
–– HighHigh--quality video decode and postquality video decode and post--processing processing architecture and algorithmsarchitecture and algorithms
–– Display with high brightness, high contrast & Display with high brightness, high contrast & color gamut, fast response time & blurcolor gamut, fast response time & blur--free free motion performancemotion performance
�� Display with formDisplay with form--factor and attributes for factor and attributes for mobile usagesmobile usages
–– Thin & lightThin & light
–– Viewable in both dark and bright ambient Viewable in both dark and bright ambient
–– Wide viewing angleWide viewing angle
�� New consumer usages (e.g. lowNew consumer usages (e.g. low--power highpower high--quality fasterquality faster--thanthan--realreal--time time encoding/encoding/transcodingtranscoding, etc.), etc.)
17
LCD Panel
DisplayController
Notebook Video & Display Subsystem: Platform View
GraphicsController
Display Interface
DisplayController
Memory
Data,Buffers
LCDDrivers
RX/TCON
GPU &Media Accel.
Engines Backlight
� Major elements
- Video & Display processor in Graphics Controller
LCD as the Mobile Computer DisplayLCD as the Mobile Computer DisplayFirst, The Good:First, The Good: LCD Made LCD Made ““MobilityMobility”” PossiblePossible
Issues with LCD:Issues with LCD:1. LCD optical stack has very poor efficiency1. LCD optical stack has very poor efficiency
Consumes almost half of platform average power at full brightness
2. LCD exhibits severe motion blur due to2. LCD exhibits severe motion blur due toa. Slow LC response timea. Slow LC response time
b. Sampleb. Sample--andand--Hold characteristicsHold characteristics
time
Intensity
Lo Gray
Hi Gray
Visual Error
Visual Error
Ideal
Response
Hi Gray
time
Intensity
CRT-like
Impulse
LCD-like
Drive-Hold
Frame Interval:1/60Hz
Blurred Video
MobilityMobility’’s Nemesis: s Nemesis: Battery LifeBattery Life
XGA display, 60 GB Hard Disk, 512 MB DDRII-533 memory, CD/DVD Drive, Microsoft* Windows* XP SP2 Operating System
Note: The subsystem percentages noted in the chart are representative figures; they may vary from platform to platform.
0%
5%
10%
15%
20%
25%
30%
35%
Average Power (%)
CPU
GMCH
ICH
Comm
Display
HDD
ODD
Memory
Clock Gen
Audio
VR
Other
Display is the largest consumer of notebook
average power!
##Intel research Dec 2005, US, Japan, Europe, China Intel research Dec 2005, US, Japan, Europe, China
–– User research study to better understand laptop purchase User research study to better understand laptop purchase process, usage and satisfactionprocess, usage and satisfaction
–– New buyers: New buyers: Top three factors in laptop purchasesTop three factors in laptop purchases–– #1: High Performance or Speed#1: High Performance or Speed
–– #2: Long battery life#2: Long battery life
–– #3: Wireless Capability#3: Wireless Capability
–– Existing users: Desired improvements with current laptopExisting users: Desired improvements with current laptop–– Battery life ranked #1Battery life ranked #1
Strong Market Demand for Battery LifeStrong Market Demand for Battery Life##
Desired Improvements w/Current Laptop
Make it Smaller
Improve HD Capacity
Improve Speed
Make it Lighter
Improve Battery Life
Long battery life: a high priority end-user requirement
23
System-Level Display Optimization for Mobile Platforms
Frame Frame
BufferBufferLCD
Panel
Image
Analysis
Image
Analysis
Backlight Backlight
ControllerController
ProcessingProcessing
Backlight Backlight
HotHot--KeysKeys
Image
Adaptation
Image
Adaptation
Backlight
User & User &
Platform Platform
PolicyPolicy
Dynamic and seamless refresh rate management
Refresh rate change associated with glitch on display
LCD Driving
Dynamic backlight level and image adjustment
Static settings independent of content and ambient lighting
LCD Backlighting
Intel® Centrino® PlatformsTraditional Approach
IntelIntel®® Display Power Saving Technology (DPST)Display Power Saving Technology (DPST)
Original ImageOriginal Image
Enhanced ImageEnhanced Image
ReducedReduced
Backlight Backlight
PowerPower
Normal Normal
BacklightBacklight
Visually EquivalentVisually Equivalent
�� DPST digitally enhances the graphics and video images prior to DPST digitally enhances the graphics and video images prior to transmission to display, simultaneously adjusts the backlighttransmission to display, simultaneously adjusts the backlight
�� ~25% backlight power saved: >1W at full panel brightness~25% backlight power saved: >1W at full panel brightness
�� Most major OEMs have adopted DPST on laptopsMost major OEMs have adopted DPST on laptops
–– Automatic Display Brightness feature uses this Automatic Display Brightness feature uses this information to adjust backlight brightness as information to adjust backlight brightness as appropriate for current environmentappropriate for current environment
–– In a dark environment, decreases backlight In a dark environment, decreases backlight
brightness to account for increased sensitivity of brightness to account for increased sensitivity of
human eye as pupil dilateshuman eye as pupil dilates
–– In a bright environment, increases backlight to In a bright environment, increases backlight to
adjust for decreased human eye sensitivityadjust for decreased human eye sensitivity
Higher MotionHigher MotionIncreased PowerIncreased Power
60Hz 50Hz 40Hz …
Lower MotionLower MotionDecreased PowerDecreased Power
ExampleExample Refresh RatesRefresh Rates
Power conservation by dynamically switching between multiple Power conservation by dynamically switching between multiple display refresh rates depending on content and power policydisplay refresh rates depending on content and power policy
Reduced power consumption in panel electronics,
display controller, display link, and memory
60Hz
Progressive
60Hz
Interlace
0
1
2
3
0
2
1
3
Dynamic seamless switching based on motion detection
High motion
contentStatic / low-motion
content
Progressive: nth “frame”
60 “frames” per secondInterlace: nth “field” (n+1)th “field”
60 “fields” per second
Reduced power consumption in panel electronics,
display controller, display link, and memory
Dynamic Display Power Optimization*Dynamic Display Power Optimization*using Multiusing Multi--Field Drive TechnologyField Drive Technology
* Jointly developed by Intel and Toshiba Matsushita Display Technology Co., Ltd.
LED is Emerging as LCD Backlight LED is Emerging as LCD Backlight
CCFL vs LED 消費電力比較11.1WXGA+で比較
0
50
100
150
200
250
300
350
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5
Power consumption Power consumption Power consumption Power consumption ((((InclInclInclIncl....INVINVINVINV))))
Brightn
ess
Brightn
ess
Brightn
ess
Brightn
ess
[[ [[ cd
cd
cd
cd// //㎡㎡ ㎡㎡
]] ]]
CCFLCCFLCCFLCCFL
LEDLEDLEDLED
>1w
Chart and data from Toshiba Matsushita Display Technology Company Ltd.
29
Intel® Graphics Media Accelerator
� Array of Cores- Multi core
- Multi-thread per core
- Advanced SIMD ISA
� Fixed Functions- Sampler: texture engine
- Math: transcendental
- ITC: Inter thread communication
- CC: Color Calculator
- 3D: 3D fixed function pipe
- Media: Media fixed function pipe
Advanced graphics media architecture balancing
programmable and fixed function acceleration
30
Processing Pipeline in Intel® Clear Video Technology
Video Processing Pipeline
� Change size to match display size/type
� Enhance picture contrast and color
� Sharpness and detail enhancement
� Convert Interlaced video to Progressive Scan video.
� Includes 24 to 30 frame rate conversion and Film mode detection.
� Remove video sensor, analog and digital noise
Description
Advanced
Deinterlacing
Temporal/Spatial
Adaptive Noise Reduction
Scaling
Film Mode
Detection
Color
Enhancement
Adaptive Detail Enhancement
31
Hollywood Quality Video (HQV*)What is it?
� Current “De Facto” benchmark intended to evaluate video quality in the consumer electronics industry
- Created and published by Silicon Optix*
� Subjectively scored
- Each test contains brief visual descriptions of potential video artifacts and scoring guidelines for the resulting image
� 10 different test patterns designed to test variety of scenarios
- De-interlacing
- Detail Enhancement
- Noise reduction
- Film cadence detection
* I
ma
ge
co
py
rig
ht
of
Silic
on
Op
tix
* Other names and brands may be claimed as the property of others
32
Improving Video Quality
110
100
0
10
20
30
40
50
60
70
80
90
120
Intel® GM965At launch(15.2.3)
Intel® GM945
HQV
Scores
~4x
increase
Intel® GM965with potential SW optimization*
� Significant video quality enhancement from last generation chipsets
� Performance scalability provided through flexible graphics architecture
* Please see demos in the Mobility Zone
33
Advanced De-interlacing
Intel® GM945 without
Intel® Clear Video
Technology
Intel® GM965 with
Intel® Clear Video
Technology
* Images obtained from Silicon Optix HQV benchmark
34
Film-Mode Detection & Reconstruction
� Detects when progressive “film-mode” content is encoded as interlaced
� Intelligently combines fields to recreate original progressive frames
Typical deinterlacing of 3:2 pulled down content can result
in artifacts
Detecting and Correcting Film Mode Cadences recreates original
progressive images
* Images obtained from Silicon Optix HQV benchmark
Without Intel® Clear Video Technology
With Intel® Clear Video Technology
35
Detail Enhancement
� Detects and sharpens edges of image increasing visibility of scene detail
� Reduces perceived image softness
� Adaptive technology to sharpen image w/o emphasize noise
* Images obtained from Silicon Optix HQV benchmark
Without Intel® Clear Video Technology With Intel® Clear Video Technology
36
GM965
Comp A
Comp B
Example Comparison (Out-of-Box)
* Images obtained from Silicon Optix HQV benchmark
37
GM965
Comp A
Comp B
Example Comparison (Custom Player)
* Images obtained from Silicon Optix HQV benchmark
38
Transition from SD to HD…
Intel will continue innovation to bring power efficient,
high quality, HD media experience to mobile platforms
Graphics/Media Processor
with Media Hardware Assist
Video Codec Pipe
>5X Codec complexity:>5X Codec complexity:
From 1From 1--stream MPEG2 to stream MPEG2 to 22--stream MPEG2/AVC/VC1stream MPEG2/AVC/VC1
>4X Usage complexity: >4X Usage complexity:
From 2From 2--plane nonplane non--interactive interactive to multito multi--plane interactiveplane interactive
6X Resolution:6X Resolution:
from SD (720x480) from SD (720x480) to HD (1920x1080)to HD (1920x1080)
Future Graphics Generations
Compensating for LCD Slow Response TimeCompensating for LCD Slow Response Time
Gray
Level
Luminance
Response
Deficit at
End of Frame
Interval
Compensation:
Over-Drive to
overcome deficit
Deficit at
End of Frame
Interval
Compensation:
Under-Drive to
overcome deficit
Frame
Interval
N N+1 N N+1
w/o overdrive
w/ overdrive
Slow graySlow gray--toto--gray response gray response time of mainstream laptoptime of mainstream laptop
Improved with OverdriveImproved with Overdrive
SummarySummary
�� Rapid growth continues in the mobile computing Rapid growth continues in the mobile computing market, aided by exploding internet usage and market, aided by exploding internet usage and technology advancementstechnology advancements
�� Intel is leading the mobility wave with powerIntel is leading the mobility wave with power--performance optimized platformsperformance optimized platforms
�� Significant challenges and opportunities for the Significant challenges and opportunities for the video & display technologies to continue enhancing video & display technologies to continue enhancing the visual experience for the consumerthe visual experience for the consumer
�� We look forward to collaborative R&D with We look forward to collaborative R&D with innovators in the academiainnovators in the academia