Dec 26, 2015
Introduction• What is areaDetector?
– Supported detectors
• Example processing chain• Why do we need a streaming video server?• ffmpegServer components
– Performance
• The future…
What is areaDetector?• An EPICS module for controlling area (2-D)
detectors, including CCDs, pixel array detectors, and online imaging plates.
• Architecture– Plugin based system– Plugins can be configured to create processing chains– Configurable at runtime– Plugins can be used on many detectors– Processing can be spread over multiple threads– Consistent data format (NDArray)– NDArray objects passed between plugins
Supported Detectors• Firewire (IEEE 1394)• Prosilica GigE cameras• Rayonix (Mar USA) CCD• Marresearch GmbH 345• Dectris Pilatus• ADSC• Roper CCD detectors• PCO imaging pcocam• Thales Pixium flat panel detectors• Perkin Elmer flat panel detectors• Bruker SMART CCD• Generic GigE vision cameras (prototype)• XIA DXP-XMAP/Jupiter/Saturn
Example processing chain
• Camera Driver• Processing
– Background subtraction– Flat field correction– Scale and offset– Recursive average
• Statistics:– Histograms– Centroid calculation– Min and max values
• Overlay– Crosses or rectangles– Selectable colours
• MJPG web server
CAM
PROC OV
STAT MJPG
Example processing chain
CAM
PROC OV
STAT MJPG
Why do we need a streaming video server?
• Scalability
Server:
CPU: 32%
Rx: 15MB/s
Tx: 15MB/s
Each client:
CPU: 15%
Rx: 15MB/s
Camera:
1024x768
Mono
20 fps
Server:
CPU: 86%
Rx: 15MB/s
Tx: 60MB/s
Why do we need a streaming video server?
• Scalability
Server:
CPU: 61%
Rx: 15MB/s
Tx: 8MB/s
Each client:
CPU: 29%
Rx: 2MB/s
Camera:
1024x768
Mono
20 fps
ffmpegServer components• FFmpeg
– A complete, cross-platform solution to record, convert and stream audio and video.
• Nullhttpd– A very small, simple and multithreaded web server for
Linux and Windows.
• ffmpegStream– areaDetector plugin that compresses NDArrays using
the MJPG codec from FFmpeg and serves them up over TCP using Nullhttpd
• ffmpegFile– areaDetector plugin that compresses NDArrays using
any codec Ffmpeg supports and writes them to file
Performance• Tests with prosilica driver and ffmpegServer compression
– 3 cameras, 1024x768 Mono @ 30fps, 105% of one core CPU usage
– 3 cameras, 1024x768 Bayer -> RGB @ 30fps, 212% of one core CPU usage
– Raw data from cameras -> server = 70MB/s
– No dropped frames
• Same raw data rate from more cameras doesn't seem to work– 4 cameras, higher res @ 10fps, 67%, 55MB/s raw data
– 5 cameras, 1024*768 @ 10fps, 42%, 39MB/s raw data
– Increasing the frame rates or res produces dropped frames
• Adding more cameras with lower res more successful– 8 cameras, 400x300 @ 70-100 fps, 87%, 81MB/s
Manufacturers suggestions• Use Intel Pro 1000 GT/PT/CT NIC.• Enable JumboFrames on NIC. PacketSize should
= 8228.• Run as root• PvAPI 1.24 with filter 1.20 recommended.• Use a dedicated LAN/NIC/Switch for the
camera(s). If using multiple cameras, on a single port NIC, be sure the sum of StreamBytesPerSecond on all cameras doesn't exceed 120,000,000.
The future…• Prosilica driver
– Replace with open source reverse engineered generic GigE Vision driver: aravis
– Prototype of areaDetector driver complete, camera producing frames
– Need to understand the GeniCam XML of each vendor camera
• ffmpegServer– Use ffserver instead of the nullhttpd– Not cross platform– Supports multicast– Support for MPEG4 over rtsp link
Other developments• Gaussian fitting
– x, y, base, height, minor axis, major axis, angle.– Uses MPFIT, so easily extensible to any other defined
function
• Gaussian simulator– Gaussian noise
• Pixel noise and on function parameters
• Function parameters have optional IIR low-pass filter.
• For further info, contact James O’Hea– [email protected]
Introduction• An effort was started at the last codeathon to
automate the testing of EPICS base. This has continued since then at Diamond. There are two threads:– Convert the ‘soft tests’ to the Diamond testing
framework.– Extend the Diamond framework to support the ‘make
runtests’ and present results as a web page.
Results• The results are available at
http://controls.diamond.ac.uk/downloads/other/epics-base-report/
• Tests are run regularly on the 3.14 branch of the launchpad repository.
• Targets so far supported are:– Linux-x86 (host): ‘run tests’ and ‘soft tests’– vxWorks-ppc604_long: ‘run tests’ only– RTEMS-mvme5500: ‘run tests’ only
Code Coverage
• A statement based, code coverage report is also available. This indicates the coverage obtained during the linux-x86 host execution of ‘make runtests’ and ‘soft tests’.– Uncovered lines are indicated with a ‘#####’
and are highlighted.– Lines marked ‘-’ are not a complete statement
(continuation lines, declarations, etc.).– A decimal number indicates the number of
times the statement was executed.
Coverage Pitfalls• Be aware of the shortcomings of statement
based, code coverage analysis.– Just because it was executed does not mean it’s
correct. That’s the job of the test case.– Code can be structured in such a way that it hides
untested code. Single line ‘if’ statements are a big culprit here.
– Insensitive to the ‘short-cut’ features of ‘&&’ and ‘||’ operators.
Further Developments• More targets.
– A Windows target will be tackled next.
• Extending ‘soft tests’ to the embedded targets.• Cross target testing, especially between targets
with differing endianity.
Any questions?