What's inside 09 2013: gstreamer: black magic and witchcraft

GStreamerBlack Magic and Witchcraft

--what it is not--

Marc LeemanBarco Networking Division

Content● Goal● Introduction● Common misconceptions● Fun and Games● Development

Goal● The presentation should allow the audience to

leave the room and start playing with it○ No fancy products○ No hocus-pocus with hardware acceleration○ No real-life examples○ Really, nothing useful...

Introduction● What is GStreamer

○ set of libraries with a couple of example programs

○ highly configurable○ streaming framework, modular

■ autoplugging■ dynamic loading■ caps, incremental buildup pipeline,

visualisation of procedure, ...

Common misconceptions● GStreamer is slow

○ Pretty harmless: educate them○ Performance issues are mostly due to bugs (e.g.

alsa, h.264 parsing, …)● GStreamer is not for embedded

○ Stuck in the previous century?○ Axis encoders, cell phones, Settop boxes, ...

Basic Tools● What GStreamer instance do I have?

○ (jhbuild)marc@drd1812:~$ gst-inspect-1.0 |grep barco○ barcortp: rtpsink: RtpSink○ barcortp: rtpsrc: rtpsrc○ barco: edgeblend: EdgeBlend○ barco: vbcpsrc: VBCP client source○ (jhbuild)marc@drd1812:~$ gst-inspect-1.0 rtpsink○ Factory Details:○ Rank: none (0)○ Long-name: RtpSink○ Klass: Generic/Bin/Sink○ Description: Barco Rtp sink○ Author: Thijs Vermeir <[email protected]>, Marc Leeman <[email protected]>, Paul Henrys

<[email protected]>○○ Plugin Details:○ Name: barcortp○ Description: Barco RTP Plugins○ Filename: /home/marc/gstreamer.jhbuild/build/lib/gstreamer-1.0/libgst-plugins-barcortp.so○ Version: 1.0.0.11.1○ License: LGPL○ Source module: gst-plugins-barcortp○ Binary package: Barco○ Origin URL: http://www.barco.com/

….● Experiment with gst-launch

Fun and Games● Get a copy of GStreamer 1.x

○ apt-get install …○ make menuconfig && make (hardcore)

● Here we go…● Capture from the V4L2 interface, and show

○ $ gst-launch-1.0 v4l2src ! videoconvert ! xvimagesink

● Scale the video○ $ gst-launch-1.0 v4l2src ! videoscale ! video/x-raw, width=640, height=360 ! videoconvert !

xvimagesink

● Do both○ gst-launch-1.0 v4l2src ! tee name=t t. ! queue ! videoscale ! video/x-raw, width=640, height=360 !

videoconvert ! xvimagesink t. ! queue ! videoconvert ! videoflip method=2 ! xvimagesink

Fun and Games● What is happening here?

○ gst-launch-1.0 v4l2src ! tee name=t t. ! queue ! videoscale ! video/x-raw, width=640, height=360 ! videoconvert ! xvimagesink t. ! queue ! videoconvert ! videoflip method=2 ! videocrop left=400 right=400 ! xvimagesink

● Capture red colour, live○ gst-launch-1.0 v4l2src ! tee name=t t. ! queue ! videoscale ! video/x-raw, width=640, height=360 !

videoconvert ! xvimagesink t. ! queue ! videoflip method=2 ! videocrop left=400 right=400 ! alpha method=custom target-r=245 target-g=0 ! videoconvert ! xvimagesink

● Use camera and overlay “snow in a box”○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-

raw,width=1280,height=720 ! videoconvert ! ximagesink sync=0 videotestsrc pattern=snow timestamp-offset=3000000000 ! "video/x-raw,format=AYUV,width=640,height=480,framerate=(fraction)30/1" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.

Fun and Games● What about other formats?

○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-raw,width=1280,height=720 ! videoconvert ! fpsdisplaysink sync=0 videotestsrc timestamp-offset=3000000000 horizontal-speed=1 ! "video/x-raw,format=AYUV,width=640,height=480,framerate=(fraction)30/1" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.

● Something more nerdy?○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-raw,

width=1280,height=720 ! videoconvert ! fpsdisplaysink sync=0 filesrc location=/home/marc/battlestar.galactica.blood.and.chrome.2012.1080p.bluray.x264-geckos.mkv ! decodebin ! videoconvert ! videoscale ! "video/x-raw,format=AYUV,width=640,height=480" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.

● More work○ gst-launch-1.0 videomixer name=mixer sink_1::alpha=0.5 sink_1::xpos=50 sink_1::ypos=50 ! video/x-raw,

width=1280,height=720 ! videoconvert ! fpsdisplaysink sync=0 filesrc location=/home/marc/battlestar.galactica.blood.and.chrome.2012.1080p.bluray.x264-geckos.mkv ! decodebin ! videoconvert ! videoscale ! "video/x-raw,format=AYUV,width=640,height=360" ! timeoverlay ! queue2 ! mixer. v4l2src ! timeoverlay ! queue2 ! videoconvert ! mixer.

Development● Develop against git master

○ lots-a-fixes, lots-a-features○ cooperation on IRC○ avoid fixing something that was fixed○ jhbuild (cerbero): standalone development

environment, free from system configuration○ 0.10 is dead, long live 1.0

● Runs on lots-a OSes (Linux, OSX, Windows) on lots-a architectures○ Active development is mostly done in Linux○ There is hardware impact