Gstreamer: an Overview

GStreamer: An overview

Rodrigo Costa

Pontifical Catholic University of Rio de JaneiroTeleMı́dia Lab.

[email protected]

April 16, 2015

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 1 / 22

Overview

1 GStreamerOverviewPipelinesBasic ConceptsSynchronizationExamples

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 2 / 22

What is GStreamer?

A framework for multimedia applications

Linux, Windows, OS X, . . .

10+ years old project

Similar to DirectShow, QuickTime, etc.

Designed to make it easy to write applications that handle audiovisualcontents

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 3 / 22

GStreamer overview

What GStreamer NOT is?

A codec implementation

An application itself

Features

Fully extensible by plugins

Flexible

Low and high level API

Often wraps other libraries (decoders, encoders, filters, etc.)

Integration with other frameworks and projects

WebKit, Firefox, Gnome, Android, iOS, . . .

Pipeline-based

Bindings to multiple languages

C/C++, Python, Java, Ruby, Pearl, etc.

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 4 / 22

GStreamer overview

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 5 / 22

Design

The fundamental design comes from the video pipeline at OregonGraduate Institute: Infopipes

Infopipes (2001)

High-level abstraction for building blocks that handle informationflows

Goal: simplify the task of building distributed streamingapplications by providing basic elements such as pipes, filters, buffers,and pumps

Motivated by the lack of support of RPC-like technologies to developstreaming and processing of distributed information

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 6 / 22

Infopipes

Infopipe example:

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 7 / 22

GStreamer pipeline

GStreamer pipeline example:

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 8 / 22

Basic Concepts

Elements

The most important class of objects in GStreamer

Each one has a specific function: read a file, receive data fromnetwork, decode, multiplex, present the data, . . .

A chain of elements is created and linked together for the data toflow through it

GStreamer ships with a large set of elements

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 9 / 22

Basic Concepts

Elements (cont)

Source

Filters, converters, demuxers, muxers, codecs

Sink

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 10 / 22

Basic Concepts

Pads

Connection points between elements

Source pads produce data

Sink pads consume data

Data always flows from src to sink pads

Can operate in pull or push mode

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 11 / 22

Basic Concepts

Pads (cont)

Each pad have a predefined set of properties called Capabilities orCaps

Caps are used to validate the communication between elements (capsrestrict the type of data that flows through it)

A source pad can only be linked to a sink pad if their allowed datatypes are compatible

For instance, a source pad that produces a video stream cannot belinked to a sink pad that consumes audio streams

Elements can negotiate with each other on format to use (capsnegotiation)

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 12 / 22

Basic Concepts

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 13 / 22

Basic Concepts

Bins

Elements can be grouped into a container called bin

It can be manipulated as it were an element

It also manages the state of its children elements

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 14 / 22

Basic Concepts

Pipeline

A pipeline is a top-level bin

It provides a bus for communication purposes

It manages the playout synchronization

It runs in a separate thread

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 15 / 22

Basic Concepts

Communication:

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 16 / 22

Synchronization

GstClock objects provide clock time

GstClock always returns the absolute-time

The pipeline maintains a GstClock object and a base-time (time whenit goes to PLAYING state)

running-time = absolute-time - base-time

Sink elements are responsible for present the buffers in theirrespective presentation time

If a buffer is delayed, the sink drops it

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 17 / 22

Pipeline Example

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 18 / 22

Example 1

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 19 / 22

Example 2

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 20 / 22

Example 3

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 21 / 22

Example 4

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 22 / 22