General Framework for the Rapid Development of Interactive Paper Applications

November 4, 2006

General Framework for the Rapid Development

of Interactive Paper Applications

Moira C. Norrie, Beat Signer and Nadir Weibel

Institute for Information Systems, ETH Zurich

8092 Zurich, Switzerland

http://vub.academia.edu/BeatSigner

November 4, 2006 CoPADD 2006 2

Applications

Enhanced Reading

interactive paper maps

interactive festival brochures

educational materials

auction brochures

scientific publications

…

Enhanced Writing

photo album

researcher's notebook

mammography

recommender system

…

Paper-Based Interfaces

PaperPoint

educational games

digital libraries

video analysis tool

…

Artists' Installations

Lost Cosmonaut

Generosa Enterprise

Interactive TableTops

November 4, 2006 CoPADD 2006 3

Input Devices

November 4, 2006 CoPADD 2006 4

Interactive Paper Architecture

iPaperAC

Encoded

Position

HTTP Request

ID, Page, (x,y)

Response

iPaper Client and

Output Device

Interactive

Document

Link DB

XIM

A

Print-n-Link: Weaving the Paper Web

Moira C. NorrieDept. of Computer Science

ETH Zurich8092 Zurich, Switzerland

[email protected]

Beat SignerDept. of Computer Science

ETH Zurich8092 Zurich, Switzerland

[email protected]

Nadir WeibelDept. of Computer Science

ETH Zurich8092 Zurich, Switzerland

[email protected]

ABSTRACT

Citations form the basis for a web of scientific publications.Search engines, embedded hyperlinks and digital librariesall simplify the task of finding publications of interest onthe web and navigating to cited publications or web sites.

However the actual reading of publications often takes placeon paper and frequently on the move. We present a systemPrint-n-Link that uses technologies for interactive paper toenhance the reading process by enabling users to access dig-ital information and/or searches for cited documents from a

printed version of a publication using a digital pen for inter-action. A special virtual printer driver automatically gener-ates links from paper to digital services during the printingprocess based on an analysis of PDF documents. Depend-ing on the user setting and interaction gesture, the systemmay retrieve metadata about the citation and inform the

user through an audio channel or directly display the citeddocument on the user’s screen.

Categories and Subject Descriptors

D.2.11 [ Software Engineering ]: Software Architectures;H.4.m [ Information Systems Applications ]: Miscella-neous; H.5.m [ Information Interfaces and Presenta-tion ]: Miscellaneous

General Terms

Design, Algorithms, Experimentation

Keywords

Interactive paper, document integration, digital library, ci-tation management

1. INTRODUCTIONNowadays we are all familiar with the notion of hyperlinks

as a means of creating interconnected document spaces thatcan be easily searched and navigated with the help of search

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.DocEng’06, October 10–13, 2006, Amsterdam, The Netherlands.Copyright 2006 ACM 1-59593-515-0/06/0010 ...$5.00.

engines and web browsers. The ease with which we cannot only find documents on the web, but also follow linksto related documents is considered to be one of the mainadvantages of working with digital documents as opposedto paper equivalents. Yet many studies have shown thatpaper is still the preferred med ium when it comes to reading

documents for reasons of comfort, mobility and also ease ofannotation.

In this paper, we show how emerging technologies for in-teractive paper enable a bridge to be built between paperand digital document spaces that allows users to travel backand forth between the two using the medium most appropri-

ate to the current task and sit uation. Users can print doc-uments for reading and, by activating hyperlinks on paperwith a digital pen, they can not only access digital informa-tion about related documents, but also initiate web searchesto retrieve copies of those documents. The retrieved docu-

ments may in turn be printed for reading with automaticallygenerated embedded hyperlinks on the paper. Mobility issupported by providing disconnected users on the move withaudio information about related documents and the optionof adding them to a request list to be processed when recon-nected.

As an application, we take the everyday academic task ofreading scientific publications. Whether the reading activ-ity is for the purpose of knowledge discovery or reviewing,readers regularly look up citations to other publications byflicking to the reference list at the back of the paper. In somecases, a search for the cited publication may follow and, if

found, often it is printed and added to a pile of documents tobe read. Citations are the established meansof providing thelinks that form the basis of the scientific web of publicationsthat spans the digital and paper worlds. Just as citationswithin digital documents can be used to automatically gen-erate hyperlinks to cited documents, we can automatically

generate links within paper documents through an analysisof the PDF file when the document is printed.

Section 2 presents a more detailed look at the activityof reading scientific publications and a discussion of relatedwork. Section 3 describes the digital pen and paper tech-

nologies and how these can be used to support interactionbetween paper and digital services. We then present thegeneral functionality of our enhanced reading system in Sec-tion 4 and the overall architecture in Section 5. Details ofthe virtual printer component used to analyse PDF docu-ments for automatic link generation as well as printing are

given in Section 6. Concluding remarks follow in Section 7.

November 4, 2006 CoPADD 2006 5

Input Devices

+addInputDeviceEventListener()

+removeInputDeviceEventListener()

<<interface>>

InputDevice

AbstractInputDevice

InductivePen NokiaPen

+inputDeviceEvent()

+addInputHandler()

+removeInputHandler()

<<interface>>

InputDeviceEventListener

AbstractInputDeviceEventListener

InductivePenEventListener NokiaPenEventListener

+getTimestamp() : long

<<interface>>

InputDeviceEvent

AbstractInputDeviceEvent

NokiaPenEventInductivePenEvent

0..*1Notifies

......

...

0..*

1

Creates

+id: String

+page: int

+x: double

+y: double

Location

+timestamp: long

TimestampedLocation

+handle()

<<interface>>

InputHandler

DefaultInputHandler ...

0..*

1

Creates

1

0..*

Notifies

November 4, 2006 CoPADD 2006 6

iServer Cross-Media Platform

iPaper

Shape Page

Application Database

DB ObjectData/Metadata

Query

Movie Clips

Time Span /

Spatial InfoMovie Clip

HTML Documents

XPointer XHTML

Document

Selectors

Entities

Resources

link

November 4, 2006 CoPADD 2006 7

Interactive Paper Authoring

iServer/iPaper interfaces

Java API

XML-based specification

Web Service interface

P2P interface

Manual link authoring

Content-driven publishing

Automatic link authoring

November 4, 2006 CoPADD 2006 8

iDoc Publishing Framework

Printer

Driver

Document DB

iDOC

Document

Metadata

Semantic Mapper

Plug-ins

iSERVER

Positions and

Active Content

Print

CMS

iPAPER

November 4, 2006 CoPADD 2006 9

Active Components

Button

Slider

Slider2D

DateSelector

Browser

MapSelector

Rating

Capture

CaptureAndSpeak

Image

Sound

Movie

SQLClient

XCMRequest

ClientCtrlRequest

PaperPointControl

PaperPointAnnot

...

November 4, 2006 CoPADD 2006 10

Slider / Slider2D Active Component

November 4, 2006 CoPADD 2006 11

Conclusions

General framework for interactive paper

Cross-media information server

extensibility (resource plug-ins)

link service acessible through various APIs

active content

information sharing

Constantly growing set of resource plug-ins and

active components