Transcript
Distributed Video Encoding: Methods to Distribute Computationally Difficult Work
Clayton E. HarperMatthew Chittum
Advisor: Dr. Wenjun Zeng7/30/08
Abstract
The proliferation of mobile media devices has cultivated the desire to have
video on the go. These devices are ultra-compact and have a fraction of the storage
space of a modern computer. For these devices to hold onto a reasonable amount of
video, such video must be encoded or transcoded to a format compatible with most
devices and that takes up very little space. This process of video encoding is
extremely difficult work for the processor and takes long amounts of time. Finding
an ability to distribute this work among several computers would drastically reduce
the encoding time for any given file. Utilizing open source tools, we developed a
method for splitting a video file and distributing the encoding process across several
computers resulting in a smaller total encode time compared to a single computer
doing the same work. It is clear that distributing this computationally difficult work
can potentially deliver higher quality video more quickly than current standards.
Introduction
Video transcoding currently serves three purposes: to reduce the size of a
video, to make the video compatible with computer software and/or mobile
hardward, and to meet the specifics of the user’s quality settings (by means of
bitrate, resolution, etc). By transcoding the video properly the user can play the
video not only on his personal computer, but also on mobile devices, such as cell
phones and other portable devices. Video transcoding also allows providers of
services such as Video on Demand (VoD) to quickly stream their content. With
websites such as youtube, mobile devices that have the capability to play videos, and
VoD services providing streaming video content the importance of video in today’s
society becomes apparent. It is therefore important that the video transcoding
process happens as quickly and as seamlessly as possible.
Unfortunately, the process of video transcoding is computationally difficult
and processor intensive, meaning that video transcoding can take long amounts of
time. As shown in our data a 1GB file takes around 1 hour and 30 minutes to
transcode using an average computer that could be found in most homes today.
While 1 hour and 30 minutes is not a bad time, most movies are not 1 GB. In fact,
most movies today are at least 7GB, which means that transcoding one of these can
take 10 hours or more. This time is simply unacceptable for people who want to
quickly place movies on their mobile devices, or for video on demand providers to
transcode their streams on the fly.
Thus a solution that reduces video transcoding time is needed in order to
satisfy users of mobile devices and VoD service providers. While many solutions
exist there is only one viable solution that can be quickly developed and will deliver
shorter transcoding times. That solution is to distribute the transcoding across
multiple computers. So, instead of one computer doing all of the work, multiple
computers will work as a unit to complete the same job but in a drastically shorter
time period. While other distributed video transcoding solutions exist, such as
x264farm or QMaster from Apple, these are either too expensive or don’t deliver the
performance increase that is needed. Therefore, a better solution is needed, one
that is open-source and free but can compete with the performance of
professionally developed software.
The framework for creating a distributed video transcoding system is
relatively simple: The server splits the original source file into chunks, the server
sends each unique chunk to a client computer, each client computer encodes the
received chunk to user specifications, the client then sends the chunk back to the
server, and then the server joins all chunks once they’ve been received. The
description makes this process look simple, but in reality the final product is quite
complex relying on several open source projects as well as a multi-threaded
server/client application. The program will be discussed in much more detail in the
“Methodology” section, but this framework provides a relatively simple overview
for such a complex problem.
Methodology
Overview of our Application
The program we developed during the internship incorporated several open
source tools. The goal of this project was to measure the performance difference
when adding multiple computers to an encoding job, not to fundamentally change
the encoding process. To make our program achieve this goal, video splitters,
joiners, and encoders were used that had already been developed. The basic flow of
our algorithm was as follows:
1. Capture Source – Obtain the source (.vob) files, most likely by ripping
from a DVD.
2. Spilt Source – Once the source .vob file has been ripped it is split into
multiple chunks.
3. Distribute Split Files – Once the .vob file has been split the files are sent
to agent computers, trying to assign work to the least worked client.
4. Encode/Transcode Split Files – Once a file has been received by the
agent, the agent transcodes the file into a new file format specified by the
server.
5. Download and Join Split Files – Once a file has been transcoded by an agent it is sent back across the network to the original server. Once the server has received all files it joins the files together in sequential order. The resultant video file is transcoded into a smaller file size and in considerably less time.
Details of Application Flow
The overall application we developed had two distinct programs running on
separate computers. The “server” program was the essential starting and ending
point of the application. The server would start and split the source video file into
separate chunks and send them to the “client” program. The client program was run
on several computers and would connect to the server program and receive the files
it sent. These files were transcoded by the client computer and sent back to the
server, which would join them back into a playable video file.
For our project, a 1GB .vob file was used to as our source image. The .vob file
contained 15 minutes of MPEG-2 encoded video at DVD quality. VStrip was used to
split the file into 20MB chunks. VStrip was executed at the beginning of every
transcode and the time taken to split the file was constant for every test run. After
VStrip was finished running, our server program would build a queue of all the
chunks that needed to be transcoded.
At this point, the server program would begin to distribute these chunks to
available clients. The server side of the application, the program that distributed the
chunks, would talk to the client side, running on separate computers transcoding
received files. The server distributes chunks based on the number of jobs a client
computer currently has, i.e. the computer running the fewest number of jobs
receives the next job. Upon receiving a file, the client computer starts transcoding
the file to the .mp4 file container using the h264 video format using handbrake,
another open source tool. After finishing a transcode job, the client transmits the
file back to the server.
After the server receives all of the transcoded files, they are concatenated
together using MP4Box into a single .mp4 file.
Structure of Program
Both the client and server utilize posix threads to run multiple sections of
code at the same time. The following diagrams show how each program works:
X264Farm
X264farm is another application that distributes video encoding. Instead of
splitting up files into individually encodable chunks, it decompresses a video file to
the raw YUV file format. This file format is completely uncompressed and an initial
first pass determines which sequences of frames are GOPs. Once the second pass
begins, the program sends these groups of pictures to agent computers to encode.
Because of the nature of the YUV format, network transmission speed is a concern.
X264farm is generally slower than our application because the decompression of
MPEG-2 takes time, the first pass can only be done by one agent, and transferring
raw video over a network is slow.
X264Farm and Our Application
Both X264Farm and our application suffer from some common limitations.
Typically, source video files are high quality and are large in size, meaning the time
to distribute the source file over the network is significant. Depending on the size of
the file and the speed of the network, the time added to the encode will negatively
affect performance. A typical DVD is 8GB in size. According to x264farm.com,
x264farm transmits DVD video at around 20 fps on a 100MB Ethernet connection1.
If the encode settings are low enough that the encoding process exceeds 20 fps,
sending it over the network is worthless.
Another problem that both applications share is longevity. In the future,
computers will have more physical cores limiting the use of distribution of work. A
16 core computer would be far more ideal that using either x264farm or our
application with 16 client computers2. Also, increased storage space may reduce the
desire for encoding altogether.
Details
Method for Data Collection
Our testbed consisted of 12 computers with Pentium 4 processors clocked
between 2 and 2.4 GHz. To measure the performance of our application and
x264Farm, we recorded the amount of time taken to transcode a 1GB DVD quality
video. We changed the bitrate and resolution of the transcode settings to view the
impact of transcode quality on time. We also changed the number of computers
used in the system to measure how performance would improve when using more
and more computers.
The total time for a complete transcode in our application includes the time
to split the source file and join the transcoded chunks. The time taken to split and
join these files was remained constant independent of the transcode settings and
number of computers used.
Data
Time (min) of transcodes with the following settings
QualityPC # 480x320@960kbps 640x432@960kbps 720x480@960kbps
1 77.22 79.21666667 83.666666672 42.96566667 43.66566667 45.708833334 21.95383333 26.345 26.100166678 16.66666667 15.957 16.125
12 15.1777 14.898 14.6803
QualityPC # 480x320@1500kbps 640x432@1500kbps 720x480@1500kbps
1 85.833 86.73333333 100.08333332 46.94483333 48.2835 46.996333334 25.31683333 26.13 28.166666678 16.514 17.46 18.184
12 14.256 14.17118333 13.5782
As the data above shows, sharing the work with multiple computers
significantly reduces encode time. The performance of the system increases with
every computer added. The theoretical performance should double as the number
of computers doubles with any given set of transcode settings. However, overhead
on the server side along with delays in pushing the files from the server reduces the
percentage gained each time computers are added.
There is an upper limit to how much performance can be gained by adding
computers. The time it takes to send out files means that by the time the last few
clients are receiving their first batch of work the first computers to have received
work are finished. This effectively makes the extra computers worthless. Using
higher transcode settings increases this upper limit, as well as increasing the chunk
size of a split file. However, increasing the chunk size means the time taken to send
that file over the network increases, increasing the overall time of the transcode.
Data Collection with x264Farm
The nature of x264farm makes transcoding take much longer than our
application. As described in the methodology, the first pass of x264Farm along with
the decompression of the source file amount to very large transcode times.
PC # 2 4 8 12Time (min) 133.041 108.03 102.916 98.0539
Conclusions on Performance
Our application suffers from diminishing performance returns as more
computers are added However, the performance gained when using multiple
computers compared to a single computer are significant, cutting the total time
almost to a tenth when using 12 computers. X264Farm sufferes similarly, but the
total time taken when using this application makes it not desirable in most
scenarios. One way to improve the effciency of both applications would be to have
multiple jobs working at the same time. If client computers were working on jobs
from multiple transcode servers, the delay on waiting for jobs would be reduced as
would be the time taken for the overall transcode.
Conclusions
The three purposes of video transcoding are to reduce the size of a video, to
make the video compatible with computer software and/or mobile hardware, and to
meet the specifics of the user’s quality settings (by means of bitrate, resolution, etc).
Since the process of video transcoding is computationally difficult, it can take long
amounts of time. Splitting up the file and distributing it to multiple computers can
drastically reduce this unacceptable transcode time.
As with anything improvements can always be made, and this program is no
exception. By using the P2P protocol the program can improve file transfers and
group communication. Instead of just sending files across the network a P2P
implementation would allow for all clients and the server to communicate with
other. It could also allow for multiple servers to send data as well as make it
possible for more than one client to receive the same data improving redundancy.
While VStrip and MP4Box are powerful pieces of software they introduce many
limitations that hamper not only the functionality, but also the usability of the
program. Implementing our own version of those tools would give us the reliability
needed for a project such as this. Every year Windows continually loses some of the
OS market share, basically meaning that more and people are turning towards other
operating systems to satisfy their needs. In order for this program to reach as many
people as possible it will need to be cross compatible with operating systems such
as Mac OS X and Linux. The program can only currently handle converting .vob files
into .mp4 using the h264 standard. In order for the program to be truly useful to a
large audience it would need to handle not only multiple source file types but also
be able to transcode into multiple file types.
Although this software is in a pre-alpha stage, it is still capable of showing
that the concept of distributed video transcoding is the ideal solution for shortening
the lengthy process of video transcoding using one computer. By adding some of
the aforementioned features distributed video encoding has the potential to become
successful in any field that uses video transcoding. From its uses in academia, to its
uses for commercial applications, and even its uses on home networks it’s easy to
see that distributed video encoding can meet the needs of a wide variety of people in
a wide variety of fields.
References
Wilson, Reed “x264Farm: A Distributed Video Encoder” <http://omion.dyndns.org/x264farm/x264farm.html>
Krazit, Tom “Intel shows off 80-core processor” <http://news.cnet.com/Intel-shows-off-80-core-processor/2100-1006_3-6158181.html>
top related