ImageTerrier www.imageterrier.org OpenIMAJ Intelligent Multimedia Analysis In Java www.openimaj.org OpenIMAJ and ImageTerrier Java Libraries and Tools for Scalable Multimedia Analysis and Indexing of Images Jonathon Hare, Sina Samangooei & David Dupplaw The University of Southampton
18
Embed
OpenIMAJ and ImageTerrier: Java Libraries and Tools for Scalable Multimedia Analysis and Indexing of Images
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ImageTerrierwww.imageterrier.org
OpenIMAJIntelligent Multimedia Analysis In Java
www.openimaj.org
OpenIMAJ and ImageTerrierJava Libraries and Tools for Scalable Multimedia
Analysis and Indexing of Images
Jonathon Hare, Sina Samangooei & David Dupplaw The University of Southampton
What is OpenIMAJ?
Scalable Clustering
Scalable KNN
OpenIMAJIntelligent Multimedia Analysis In Java
Image
Analysis
LibrariesCommand-line Tools
Hadoop Map-Reduce Implementations
HardwareMachine Learning
Video
Web Audio
Analysis
AnalysisProcessing
ProcessingProcessing
Analysis Rendering
Kinect Audio & Video
CaptureExotic
Hardware
LibrariesCommand-line Tools
Hadoop Map-Reduce Implementations
Efficient, Compressed
Inverted Index
Query Engine
Term Metadata
IntegrationBased on
Terrier Constructable from Images
Distance Metrics
scaleRe-Ranking affine
orientation
Compressed
Extensible
OpenIMAJIntelligent Multimedia Analysis In Java
Scalable
e.g SIFT BoVW (x, y, scale, etc)
What is ImageTerrier?Efficient inverted index construction for image/video features (and potentially audio features too)
Augmented inverted index, capable of storing almost any kind of meta information per term
Easily extensible
Provided implementations contain things like (x,y,scale) for SIFT BoWV
Indexing is scalable across machines using Hadoop
Complete OpenIMAJ integration
Included tools can build an index straight from a set of images using common techniques like SIFT BoVW or read features extracted from other tools.
This can even be done on Hadoop!
Easy to define searching and re-ranking strategies
Various distances, weightings and constraints (such as Affine & Homographic) are provided
Tested with corpuses in excess of 10M images
A Little history...IAMMediaEngine
c. 2005
MapSnapc. 2006
JCVMLTKc. 2009
OpenIMAJ & ImageTerrier
May 2011
MoVEc. 2010
Pure-Java SIFT & Matching
Extensible Java (& JNI) Content-
Based Retrieval System
Refactoring of older code, addition of face detection
and machine learning components
Initial attempts at scalable BoWV indexing using terrier,
shape analysis, etcMany new features added.
Improvements to ImageTerrier, face pipeline, demos, video capture and
hardware, ... PhotoCopainc. 2005
Image classifiers
SMS-NLPc. 2010
Machine learning, clustering
Massive code cleanup and documentation effort. First
Open Source Release
It’s pure Java... (with the exception of thin interfaces for hardware)
Isn’t it slow?
Why another set of libraries?Scalability + Portability. OpenIMAJ projects work on:
Android mobile devices
Hadoop clusters
...without recompilation!
Easy to use and extend
Implement many important Multimedia algorithms
While allowing easy adaption and extensions.
Software engineering principles adopted
clear separations of concerns
maintainable, understandable
We simplify data representations:Images are 2D-arrays of pixels;
Videos are iterable streams of Images
Audio is an array of samples
Both OpenIMAJ and ImageTerrier have been designed from the ground-up to be easily extensible.
We’ve made it easy to modify and understand algorithms
and techniques by separating different components.
Dominant(orienta+on(of(an(orienta+on(histograms(
Replace'with'a'Null'orienta1on'extractor.'
Regular(SIFT(feature(provider(
Replaced(with(Irregular(Binning(
Rapid Application Development
“Make it easy to use”
This philosophy lets researchers build complex applications quickly