This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
MRL/TRLAcknowledgements*Acknowledgements* Yen-Kuang Chen, Yen-Kuang Chen, Ph.D.Ph.D. Matthew HollimanMatthew Holliman Rainer Lienhart, Rainer Lienhart, Ph.D.Ph.D. Skip Macy, Skip Macy, Ph.D.Ph.D. Igor Kozintsev, Igor Kozintsev, Ph.DPh.D Andre Zaccarin, Andre Zaccarin, Ph.D.Ph.D. Valery Kuriakin and the iNNL-MPL teamValery Kuriakin and the iNNL-MPL team
(Russia)(Russia) And many othersAnd many others
* Key collaborators/contributors to the work mentioned in the talk. * Key collaborators/contributors to the work mentioned in the talk.
MRL/TRLDigital Media: from desktop, to Internet, to Digital Media: from desktop, to Internet, to hand-helds, to wireless, and to Peer-to-Peerhand-helds, to wireless, and to Peer-to-Peer
MRL/TRLMultimedia dataMultimedia dataSame content can exist in different Same content can exist in different
resolutions, bit rate and qualityresolutions, bit rate and qualityCan be gracefully degradedCan be gracefully degradedCan be transformed to multiple Can be transformed to multiple
MRL/TRLP2P for MultimediaP2P for Multimedia Maintain key features and extendMaintain key features and extend
– Supports traditional Supports traditional (existing)(existing) file sharing systems file sharing systems– Facilitates sharing of resources like CPU cycles, storage, Facilitates sharing of resources like CPU cycles, storage,
computation process or arbitrary functionalities that can be computation process or arbitrary functionalities that can be abstracted by the user as resources.abstracted by the user as resources.
Facilitates exchange of data with real time Facilitates exchange of data with real time requirements, exemplified by video or audio requirements, exemplified by video or audio transmission.transmission.
– Choice and flexibility of transport mechanisms Choice and flexibility of transport mechanisms – Meeting diverse application requirementsMeeting diverse application requirements
Efficient multimedia servicesEfficient multimedia services– Integrates media transformation/transcodingIntegrates media transformation/transcoding
Flexible query interfaces to facilitate multimedia Flexible query interfaces to facilitate multimedia searchsearch
MAPS: Media Accelerating Peer ServicesMAPS: Media Accelerating Peer Services
Technology, platform architecture, and Technology, platform architecture, and software infrastructuresoftware infrastructure– facilitates facilitates efficientefficient, , transparenttransparent and and high-high-
performanceperformance media content delivery & processing media content delivery & processing
– can take advantage of, integrate into, and bring out can take advantage of, integrate into, and bring out the benefits of the P2P computing infrastructurethe benefits of the P2P computing infrastructure
– Detect and analyze the optimal platform configuration and Detect and analyze the optimal platform configuration and operation at a particular time instantoperation at a particular time instant
– Dynamic optimization via cost-function analysis across a Dynamic optimization via cost-function analysis across a globalglobal or or local local set of peer nodes: the result determines a set of peer nodes: the result determines a bestbest operational mode for a particular media delivery or operational mode for a particular media delivery or processing operationprocessing operation
– OptimizationOptimization is built upon (and calling) a set of support is built upon (and calling) a set of support modules, like streaming, transcoding, etc. for multimedia modules, like streaming, transcoding, etc. for multimedia delivery operations, and other support modules like delivery operations, and other support modules like distributed image processing primitives for multimedia distributed image processing primitives for multimedia processing operations.processing operations.
– InvisibleInvisible and and seamlessseamless to the applications to the applications– Overall results cascaded into an enhanced service Overall results cascaded into an enhanced service
experience experience – fast delivery, higher satisfaction, lower abort rates, and better fast delivery, higher satisfaction, lower abort rates, and better
MRL/TRLCost AnalysisCost Analysis Request to evaluate cost Request to evaluate cost
of Rof R If the file is not cached If the file is not cached
(local), then find the (local), then find the minimal cost from the set minimal cost from the set of neighbors (can extend of neighbors (can extend from local to global)from local to global)
Cost of transcoding Cost of transcoding computationcomputation
Cost of transmission Cost of transmission available network available network bandwidthbandwidth
Other cost factors can be Other cost factors can be formulatedformulated
MRL/TRLMinimal Cost to Perform Minimal Cost to Perform a computationa computationFor a computation task on data (resource R), For a computation task on data (resource R),
optimize with respect to several variables optimize with respect to several variables such as delay, power consumption, temp such as delay, power consumption, temp storage, etc. (constraint parameters P)storage, etc. (constraint parameters P)
To find the minimal ofTo find the minimal of– cost of performing the computation locally, cost of performing the computation locally, – cost of transmitting the data to a peer (or multiple cost of transmitting the data to a peer (or multiple
peers), perform the computation there and peers), perform the computation there and transmitting the result backtransmitting the result back
– cost of finding the same data on a peer, perform cost of finding the same data on a peer, perform the computation there and transmitting the result the computation there and transmitting the result backback
What is the cost?What is the cost?–Cost should be dynamicCost should be dynamic
– Resource availability at “server”Resource availability at “server”– Resource availability at “client”Resource availability at “client”– Network availabilityNetwork availability
How to measure?How to measure?How to estimate?How to estimate?Methodology?Methodology?Tools?Tools?
MRL/TRLMethodology for Estimating End-to-End Computing Performance and Analyzing Network Computing Models
Methodology for Estimating End-to-End Computing Performance and Analyzing Network Computing Models
In a networked & distributed computing environment, end-to-end performance of applications depend on capability of components in the system and their interaction.
Need methodology to estimate end-to-end system performance.
Results of performance estimation used to make computing decisions.
– Best distribution of computational tasks.– Best algorithm choice.– Most effective hardware improvement.
MRL/TRLSample Results: Quality as a Performance MetricSample Results: Quality as a Performance Metric Compare JPEG2000, SPIHT and JPEGCompare JPEG2000, SPIHT and JPEG
– Order of processing rate: JPEG, SPIHT, JPEG2000 Order of processing rate: JPEG, SPIHT, JPEG2000 – Order of compression ratio for given PSNR: JPEG2000, SPIHT, JPEG Order of compression ratio for given PSNR: JPEG2000, SPIHT, JPEG
ServerServer and and Client Client can have equal capability (measured on PIII 600 MHz systems for the following) can have equal capability (measured on PIII 600 MHz systems for the following)
MRL/TRLMultimedia SupportMultimedia Support ““Malleability”/“flexibility” of (bulky) MultimediaMalleability”/“flexibility” of (bulky) Multimedia
– Graceful degradationGraceful degradation– Various representationsVarious representations– Complexity Complexity b/w and compute intensive! b/w and compute intensive!– More flexibility More flexibility more opportunities more opportunities
Special technology components in MAPSSpecial technology components in MAPS– Universal across multiple platformsUniversal across multiple platforms– Media Adaptive TranscodingMedia Adaptive Transcoding
– e.g. MPEG2 (6Mbps) e.g. MPEG2 (6Mbps) MPEG4 (300kps) MPEG4 (300kps)
– Audio/video Streaming Audio/video Streaming – Robust, error resilient coding and transmissionRobust, error resilient coding and transmission– Enhanced Multimedia ProcessingEnhanced Multimedia Processing– Technology modules for media security & tracking Technology modules for media security & tracking
(encryption/watermarking)(encryption/watermarking)– Technology that facilitates multimedia browsing/search/managementTechnology that facilitates multimedia browsing/search/management
Intel® MPL can serve as a basic media software infrastructure that provides some core media technology components while facilitates the building of the other ones
Intel® MPL can serve as a basic media software infrastructure that provides some core media technology components while facilitates the building of the other ones
MRL/TRLIntel MPL - Media Processing LibraryIntel MPL - Media Processing Library(presented at IDF Fall’99, Fall’00)(presented at IDF Fall’99, Fall’00) General purpose, high performance software infrastructure General purpose, high performance software infrastructure
with API’s for MPEG encode/decode and with API’s for MPEG encode/decode and processingprocessing BUILDING BLOCK enabling NEW rich-media apps: targeted for BUILDING BLOCK enabling NEW rich-media apps: targeted for
applications beyond standard encode/decode/display, e.g., applications beyond standard encode/decode/display, e.g., ease-of-use of video, allow interactivity with digital videoease-of-use of video, allow interactivity with digital video
Universal & Ubiquitous:Universal & Ubiquitous:– Common API’s across multiple OS: Windows, LinuxCommon API’s across multiple OS: Windows, Linux– Optimized for multiple architecture: IA, XScale™ Optimized for multiple architecture: IA, XScale™ (in progress)– Can extend to network applications (Can extend to network applications (Internet, wireless, etc.)Internet, wireless, etc.)
Scalable solutionScalable solution– Low bit rate to high bit rateLow bit rate to high bit rate– Low resolution to high resolutionLow resolution to high resolution– 1 processor to multiple processors1 processor to multiple processors– Small devices to high end serversSmall devices to high end servers
Rich features, high performance, ease-of-developmentRich features, high performance, ease-of-development
MRL/TRLMPL – MPL – PC and Internet Multimedia S/W PC and Internet Multimedia S/W infrastructure for Diverse Platformsinfrastructure for Diverse Platforms
MRL/TRLReference: MPL featuresReference: MPL features Support MPEG-1 and MPEG-2 up to HDTV, MPEG4Support MPEG-1 and MPEG-2 up to HDTV, MPEG4 Optimized with Optimized with MMX™ technology and SIMD Streaming Extensions, Pentium™ 4MMX™ technology and SIMD Streaming Extensions, Pentium™ 4 Advanced Features - the highlights:Advanced Features - the highlights:
– Random access to any frames with near constant-timeRandom access to any frames with near constant-time– Fast extraction of reduced framesFast extraction of reduced frames– Simultaneous decode of multiple MPEG sequences. Simultaneous decode of multiple MPEG sequences. – Access via callbacks (API) to non-frame level information in the MPEG bitstream, Access via callbacks (API) to non-frame level information in the MPEG bitstream,
such as raw bits, blocks, macroblocks, GOP and slice, etc. [structured access to such as raw bits, blocks, macroblocks, GOP and slice, etc. [structured access to compressed video/audio]compressed video/audio]
Ease-of-development and deploymentEase-of-development and deployment– Windows*, Linux* Windows*, Linux* – MPL as Component: COM objects (for VB, VC++, DirectShow*)MPL as Component: COM objects (for VB, VC++, DirectShow*)– IPP support for audio & video (2001)IPP support for audio & video (2001)
– cross-platform optimization: Pentium™ processor families, Xscale™ core cross-platform optimization: Pentium™ processor families, Xscale™ core architecture based processors (in planning)architecture based processors (in planning)
* Other names and brands are property of their respective owners* Other names and brands are property of their respective owners
MRL/TRLMPL Next StepsMPL Next Steps Extend to embrace & integrate more technology Extend to embrace & integrate more technology
building blocksbuilding blocks– wirelesswireless
– P2P – embrace technology catering to and take advantage of P2P – embrace technology catering to and take advantage of the “inequalities” inherent in the computing platformsthe “inequalities” inherent in the computing platforms
– low power considerationslow power considerations
– Merging of video & graphicsMerging of video & graphics
Strive toStrive to– Cater to diversity (universal)Cater to diversity (universal)
– Be among the best in Be among the best in performanceperformance
– Be flexible, and maximize utility/usabilityBe flexible, and maximize utility/usability
– Be a key building block infrastructure for emerging Be a key building block infrastructure for emerging applicationsapplications
Facilitating Multimedia SearchFacilitating Multimedia SearchAutomatic media content analysis Automatic media content analysis
algorithms AND meta-data can improve algorithms AND meta-data can improve search efficiency.search efficiency.
New browsing tools and methodologies can New browsing tools and methodologies can be deployed for distributed media sharing, be deployed for distributed media sharing, storage and retrieval, and collaboration.storage and retrieval, and collaboration.
Flexible query interfaces to facilitate Flexible query interfaces to facilitate multimedia searchmultimedia search
Search can be based on BOTH the file Search can be based on BOTH the file names and content descriptions.names and content descriptions.– e.g. MPEG7 or XMLe.g. MPEG7 or XML
– Compute spatial low resolution signature - for MPEG video, extract/estimate Compute spatial low resolution signature - for MPEG video, extract/estimate DC images for I/P/B frames, on compressed dataDC images for I/P/B frames, on compressed data
– Fit polynomial of N degrees over M samples in timeFit polynomial of N degrees over M samples in time– Describe temporal trace using the N degree polynomial for each M samplesDescribe temporal trace using the N degree polynomial for each M samples
Achieved matching rate of 6000+ frames/sec on a low-end PCAchieved matching rate of 6000+ frames/sec on a low-end PC Reduce complex MM search/matching to a fast string-matchingReduce complex MM search/matching to a fast string-matching Same signature & compact size regardless of resolution of videoSame signature & compact size regardless of resolution of video
MRL/TRLBuilding MAPS into a P2P Building MAPS into a P2P computing platformcomputing platform The enhanced multimedia peer service layer can be The enhanced multimedia peer service layer can be
integrated into existing/emerging P2P integrated into existing/emerging P2P infrastructureinfrastructure
– To make P2P media-aware and optimized for media delivery and media applications…
APIs can be defined on P2P platform to facilitate APIs can be defined on P2P platform to facilitate multimedia applications developmentmultimedia applications development
Prototype & experimentation at Intel MRLPrototype & experimentation at Intel MRL
MRL/TRLExtending Multimedia Apps and Extending Multimedia Apps and Technology from Desktop to P2PTechnology from Desktop to P2P
Emerging media-centric applications:Emerging media-centric applications: Content Transaction, Tracking and MonitoringContent Transaction, Tracking and Monitoring Media content indexing, browsing & organizationMedia content indexing, browsing & organization Interactive video, and merged video and graphicsInteractive video, and merged video and graphics Enhanced virtual realityEnhanced virtual reality Enhanced video conferencingEnhanced video conferencing Networked multimedia gamesNetworked multimedia games
Goal: to enable multimedia apps development and deployment on P2P with enhanced user experience.Goal: to enable multimedia apps development and deployment on P2P with enhanced user experience.
MRL/TRLWireless Multimedia:Wireless Multimedia:Robust Video Delivery over Wireless Robust Video Delivery over Wireless Networks (IP, Bluetooth, and beyond)Networks (IP, Bluetooth, and beyond)
MRL/TRLLooking forward…Looking forward… What is the optimal multimedia content distribution What is the optimal multimedia content distribution
mechanism?mechanism? What is the optimal multimedia processing What is the optimal multimedia processing
methodology?methodology? What are the technology building blocks? What are the technology building blocks? Many more questions…Many more questions…
– What are the computing models of the future?What are the computing models of the future?– What are the best algorithms?What are the best algorithms?– What are the What are the performanceperformance criteria? criteria?
Many good problems for researchMany good problems for researchNew framework, new formulation, …New framework, new formulation, …
MRL/TRLSummer Internships @ Intel MRLSummer Internships @ Intel MRL May/June to August/September 2001 (12-13 May/June to August/September 2001 (12-13
weeks, can be renewed to 9 months)weeks, can be renewed to 9 months) Looking for outstanding candidates (MS, Looking for outstanding candidates (MS,
Ph.D., and very exceptional BS students)Ph.D., and very exceptional BS students) Areas:Areas:
1.1. Algorithms research and development (general, Algorithms research and development (general, in various fields) – video, audioin various fields) – video, audio
2.2. Low power researchLow power research
3.3. P2P System researchP2P System research
4.4. Wireless multimedia, communicationsWireless multimedia, communications– Email: Email: [email protected]@intel.com and Matt and Matt
will forward to appropriate researcherswill forward to appropriate researchers