Nov 22, 2014
Introduction
• Mime360.com is a Digital Media Distribution Company that performs activities of ingesting new data as and when received from clients.
• The Data has to be Normalized, Sanity Checked, Catalogued and finally Indexed before it can be delivered to the various clients. With Data coming in through various sources at different times, it is imperative to track and analyse the progress of data at various stages in order to accurately predict timelines and deliverables.
Objective• The objective of this project is to create a Data Analytics tool
that will keep track of the data in the various stages of the Ingestion Process. The tool will enhance the current Data Ingestion Process.
• This project will involve i) Triggering and monitoring the transcoding video clips.ii) Automating creation and parsing of Log Files to extract dynamic data. iii) Creation of pertinent Dashboards to Display the Data.
Steps in Data Transcoding
Data input
• Sources of Data input :1. Hard Drive Disk2. FTP3. Auto FTP4. CMS• Format of input1. .mp42. Clipped .mp33. .avi4. Etc.
•Data Enters into the system via FTP, HDD or batch inputs
DataInput
Choosing format of video• Multiple video formats are
available for transcoding
• This tool provides the user freedom to choose desired format from the options:
LQ = Low quality MQ = Medium quality HQ = High quality
Android LQ Android MQ Android HQ
Blackberry MQ Blackberry MQ Blackberry HQ
PC HQ PC MQ PC HQ
Video transcoding
• The video transcoding process in the tool deals with the software `HandBrakeHandBrake`
• HandBrake: open-source
(licensed under GPL) multiplatform,
multithreaded Available for MacOS X,
Linux and Windows
Monitoring
• Video transcoding in HandBrake is triggered by php script
• The CPU usage is tracked along with percentage of Memory usage
• All related information to video conversion process is displayed during and after conversion.
• Status of on-going, failed or completed videos can be checked at all times
Process_master
Process_id
Queued_file_id
Op_format_id
Status_id
Process_info
Process_id
Progress
ETF
CPU_usage
Input_data_master
File_idFile_nameFile_path
Src_formatDte_orig
Dte_updateContent_owner_idReleasing_banner_
id
Format_lookup
Format_idPlatform_idQuality_id
Format_queryDte_orig
Dte_update
Status-lookup_table
Status_id
Status_desc
DB
TablesPlatform_lookup
Releasing_banner_master
Quality_lookup
Content_owner_master
Technologies Used
Media TypeMedia Sub
Type Stages What to DisplayGranularity of
DisplayTo be
included
Audio Unprocessed Count of Files
For all Content Owners Error Files
Songs Converted List of Files For Single CO Log Files
Clips Processed
Time to Process (for
unprocessed and converted
files)For Single
Label
Ringtones
For Processed Files (How many processed last 24 hours, last 7
days and last month)
CRBTs
Project Management and tracking
GIT Repository
END