The State of Image Annotations Robert Casties Max Planck Institute for the History of Science, Berlin
The State of Image Annotations
Robert Casties Max Planck Institute for the History of Science, Berlin
What we do• Work with historical sources
• scanned books, manuscripts, notebooks
• digital photos
• scanned drawings and paintings
• digital texts
• data
Galileo Galileis MS72 "Notes on motion" (1998) http://www.mpiwg-berlin.mpg.de/Galileo_Prototype/INDEX.HTM
Musawwarat Graffiti Archive (2011) http://musawwaratgraffiti.mpiwg-berlin.mpg.de
digilib client-side point annotation (since 2002)
HyperImage annotated presentation (2009?) of James Gillray: „Political-Dreamings!” (1801)
Status quo
• lots of digital editions and projects
• nice and diverse features and tools
• re-inventing lots of wheels
• data silos
• solution: annotations and standards
Our vision• "Weaving a web of knowledge" (Jürgen Renn, 2003?)
• close and distant reading of all kinds of sources (works best with Open Access)
• create comments, relations, narratives
• share the results during the process or the result with collaborators or everybody
• bi-directional links (Xanadu) - collect research results related to sources creating a "semantic network" of sources
references to networks
Annotations as bi-directional links
• standardized (open, stable) access to source documents (image, text,…)
• standardized annotation data linking comment to source
• relevant segment needs to be referenced
• annotation server needs to be shared
• relation can be reversed
Annotation standards• Open Annotation Data Model (http://openannotation.org)
• general annotation data model
• last spec 1.0 of 2013
• W3C Web Annotation Working Group (since 2014)
• Abstract Annotation Data Model (spec)
• Data Model Vocabulary
• Data Model Serializations
• HTTP API (spec)
• Client-side API (FindText API spec)
Image annotation in W3C web annotation
• Target of annotation can be an image
• Segment of image can be specified by
• FragmentSelector with W3C media fragment (rectangle specified in pixel or percent)
• SVGSelector with SVG
Annotation standards• SharedCanvas (http://shared-canvas.org)
• data model for ”virtual books“
• uses Open Annotation
• last spec 1.0 in 2013
• IIIF (http://iiif.io)
• Image API standard
• Metadata API standard
• spec 2.1 released May 12, 2016
IIIF (International Image Interoperability Framework)
• ARTstor • Bayerische
Staatsbibliothek (Bavarian State Library)
• La Bibliothèque nationale de France
• Biblissima • British Library • British Museum • Brown University • Centre de
Recherche et de Restauration des Musées de France (C2RMF)
• Cogapp • Columbia
University • e-codices – Virtual
Manuscript Library of Switzerland
• Cornell University • DPLA • Digital Image
Archive of Medieval Music (DIAMM)
• Digirati Ltd • Europeana • The J. Paul Getty
Trust • Ghent University • Gottingen State
and University Library
• Harvard University • Indiana University • Internet Archive • Johns Hopkins
University • Klokan
Technologies • Leiden University
• MIT Libraries • National Gallery of
Art • National Library of
Austria • Nasjonalbiblioteke
t (National Library of Norway)
• National Library of Denmark
• National Library of Egypt
• National Library of Israel
• National Library of New Zealand
• National Library of Poland
• National Library of Scotland
• National Library of Serbia
• National Library of Wales
• New York University Libraries
• Ohio State University
• Oxford University (Bodleian Library)
• Princeton University Library
• Qatar National Library
• Sirma Group • St. Louis
University • Stanford University • TextGrid • text & bytes • University College
Dublin • University of
Edinburgh
• University of Hong Kong
• University of Illinois at Urbana-Champaign
• University of Notre Dame
• University of Pennsylvania
• University of Toronto
• Vatican Library • The Walters Art
Museum • Wellcome Trust • Wikipedia
(Wikimedia Foundation)
• World Digital Library
• Yale Center for British Art
• Yale University
{scheme}://{server}{/prefix}/{identifier}/{region}/{size}/{rotation}/{quality}.{format}
Image API Presentation API
SoftwareIIIF Shapes
Annotorious - (+)
HyperImage / Yenda - / + + / +
digilib + / - +
SemToNotes - +
Mirador + (+)
Diva + -
T-PEN (+) -
AnnoTate - -
Annotorious
Annotorious• http://annotorious.github.io
• Javascript client image annotation toolkit
• annotates images in HTML pages
• can integrate with Annotator.js
• can use scaling image server (OpenSeadragon)
• can do polygons
HyperImage
HyperImage / Yenda• http://hyperimage.ws
• “like hypertext for images”
• Java editor and server, Flash/HTML5 client
• free shapes in layers, linking between images, texts and layers
• used in a number of digital humanities projects
• Yenda (http://yenda.tools)
• new fully web architecture using IIIF
• under development
digilib
digilib• http://digilib.sourceforge.net
• scaling image server (Java)
• HTML + Javascript client
• view state and client-side annotations encoded in URL (since 2002)
• server-side annotations using Annotator.js (since 2012)
• polygon shapes (since 2015)
• IIIF API server
digilib
SemToNotes
SemToNotes
• https://hkikoeln.github.io/SemToNotes
• JavaScript API to develop browser-based image annotation tools
• topological queries on eXist and EXPath Geo Module backend
• under development
Mirador
Mirador• http://projectmirador.org/
• IIIF reference viewer
• Javascript client, can use any IIIF Presentation API feed
• deep zoom, metadata display, book reading, bookmarking, comparison view
• shape annotations (under development)
Diva.js
Diva.js• http://ddmal.github.io/diva.js
• Javascript client
• uses IIIF Image API
• deep zoom, fast scrolling through documents, synchronized information views
• annotation overlays
T-PEN
• http://t-pen.org
• Transcription for Paleographical and Editorial Notation
• Java server, online version free to use
• new version 3 with IIIF support under development
AnnoTate
AnnoTate
• https://anno.tate.org.uk , https://github.com/zooniverse/AnnoTate
• Zooniverse crowdsourcing project for Tate Archive holdings
• transcription of artist’s personal papers
• zoomable view, simple annotations
where are we now• IIIF image server standard incredibly successful
• Annotation use cases growing
• Annotation standards on the way…
• Annotation software development scattered
• Discovering and sharing annotations not yet there
ceterum censeo…
Standards for annotating image areas
• something between very simple (W3C Media Fragments)
"selector": { "type": „FragmentSelector", "conformsTo": "http://www.w3.org/TR/media-frags/", "value": "xywh=135,16,25,53" }
• and very complex (SVG)
"selector": { "type": ["SvgSelector", "Content"], "text": "<svg:svg> … <polygon xmlns="http://www.w3.org/2000/svg" points="5315,5639 5444,5529 5505,5593 5346,5691 5315,5639" /> … </svg:svg>“ }
Standards for annotating image areas
• Proposition
• use resolution independent coordinates (e.g. 0≤x≤1, fraction of image width)
• use GeoJSON / WKT
• points, lines, polygons
• only point coordinates
Annotations as "micro-publications"
• stable URLs for annotations
• provenance information
• copyright and license information
• versioning?
• authority / reputation?