1 Lifelogging to Lifebraries (L2L) Outline proposal for a Network of Excellence Objective ICT-2009.4.1 Digital Libraries and Digital Preservation Target.

1

Lifelogging to Lifebraries(L2L)

Outline proposal for a Network of Excellence

Objective ICT-2009.4.1 Digital Libraries and Digital Preservation

Target Outcome (e) Interdisciplinary Research Networks

2

1. C

ult

ura

l Her

itag

e What is Cultural Heritage • Culture is based on what people do, on

what everyone does• Recording what everyone is doing allows

us to preserve cultural heritage at a hitherto unseen breadth and depth

• Lifelogging – a term with baggage, Lifebraries is our new community – Definition is recording, digitally, aspects of

one's life.– Captures one's life experiences and includes

explicit User Generated Content (UGC) as well as ambient logging

3

2. L

ifel

og

gin

g h

app

ens 2. Explicit Lifelogging

• There are several, and a growing number, of lifelogging projects in Europe and beyond

• MemoryLane, SenseCam, LiveMemories, LivingKnowledge, GLOCAL, Memories for Life Grand Challenge, HAPTIMAP, Oldenburg wellness trials

• Based on wearable cameras (SenseCam/Vicon), GPS recording tracks, accelerometers (independent/phones to record walk, sit, drive, etc.), Bluetooth sensing other BT devices (phones) we encounter, etc.

4

3. L

ifel

og

gin

g h

app

ens

3. Implicit Lifelogging• Record indirectly or infer activities –

from FaceBook, Twitter, phone/Skype calls, SMS, email and web, IPTV, LinkedIn, Flickr (etc.) photos, YouTube videos, blogs, online calendars, financial transactions.

• Smart meters on domestic electricity, gas, water infer use of different appliances, infer domestic activities.

• Sousveillance vs. Surveillance

5

4. W

hy

Lif

elo

gg

ing Why ?

• Find incidents from our past• Personal journal of one's history • Discover changes in our lifestyles, short

term or longer• Re-live the past as a memory prosthesis

- Alzheimer's/dementia & healthy aging• Share our past with others• Build a community memory of some

event• ... all these are library uses

6

5. W

hat

’s t

he

Pro

ble

m ? Problem ?

• All happening with a technological drive, with no input from: information science, memory science, neurospychology, sociology, law, ethics, governance, social acceptance, curation

• Also, projects focus on lifelogging data – no leverage of additional User Generated

Content to socially link different people's views/lives/experiences

• Need for a new community, drawn from all of the above.

7

Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

6. C

ON

SO

RT

IUM

8

• Guardian of a photographic ‘corpus’ of >5.5M pictures, historical and contemporary

• Responsible for the management of an ongoing program of exhibitions and publishing

• Relevant Expertise: watermarking, digital rights management, content provider, preservation and image restoration, multimedia content supply

• Relevant EU Projects: GLOCAL

• People: Andrea de Polo

CO

NS

OR

TIU

M AlinariAlinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

9

• Harvesting & harnessing of large volumes of sensed information, from both the physical world in which we live, and the digital world of modern communications & computing

• >50% of all SenseCam publications have CLARITY authors

• Relevant Expertise: Information Management in Ambient Lifelogs, human digital memories

• People: Alan F. Smeaton, Cathal Gurrin, Aiden Doherty, Noel E. O’Connor

Dublin City University - CLARITYC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

10

• Studies in Ethics, Law and Technology

• Play a leading role in raising public awareness of, and stimulating debate about, ethical issues

• Relevant Expertise: ethics of emerging technologies, ethical issues in life sciences, healthcare, technology and innovation, and business and society

• People: Bert Gordijn

DCU - Institute of EthicsC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

11

• International reputation in research on information retrieval methods

• Focus on Multimedia information management

• Relevant Expertise: Developing advanced retrieval models, studying the role of emotion in search, personalization and adaptive retrieval

• People: Joemon M. Jose, C. J. (Keith) van Rijsbergen

University of GlasgowC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

12

• World-class reputation in memory science & neuropsychology, with links to local and national clinicians in the National Health Service

• Memory impairment research at Leeds work includes some of the first work with the SenseCam wearable camera in memory impaired groups

• Relevant Expertise: Autobiographical memory, memory rehabilitation, memory dysfunction in dementia & healthy aging, the self & memory

• People: Chris Moulin, Martin Conway

University of LeedsC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

13

• Past projects geared towards health-care (e.g. FP5 MobiHealth), digital preservation and context aware social networking

• Relevant Expertise: advanced applications for eHealth, support for people with disabilities using enabling technologies, context discovery in social networks

• Relevant EU Projects: COGKNOW

• Relevant Regional Projects: MemoryLane

• People: Arkady Zaslavsky, Kåre Synnes, Josef Hallberg, Johan E. Bengtsson

LuleåC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

14

• Research in the field of new technologies with a special emphasis on privacy issues, individual and public freedom in the Information Society and Internet Governance

• Member of the Belgian Commission on Data Protection

• Relevant Expertise: social science & philosophy, ethical, political and legal challenges raised by the new information, communication and surveillance technologies

• People: Yves Poullet, Claire Lobet-Maris, Antoinette Rouvroy, Nathalie Grandjean

NamurC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

15

• Identify, enhance, and evaluate new techniques of information and communication technology for design of environments for aging

• Detection of daily activities by monitoring

• Relevant Expertise: Information summarisation, person activity classification

• Relevant Industry Projects: CeWe Color

• Relevant EU Projects: HaptiMap

• People: Susanne Boll, Jochen Meyer, Wilko Heuten

OldenburgC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

16

• Provides services such as technology research, organisation of business and cooperation events, EU project proposal support and eLearning consultations

• Relevant Expertise: Project Administration & Management

• People: Craig Stewart

PraepositC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

17

• Semantic multimedia analysis

• Leveraging user generated content for enhancing representation

• Knowledge extraction from socially driven content generation

• Relevant Expertise: multimedia content analysis, implicit & explicit user relevance feedback

• People: Ebroul Izquierdo, Krishna Chandramouli, Alan Pearman, Qianni Zhang, Tomas Piatrik

Queen Mary University LondonC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

18

• Experimental research on cognitive aspects of user-system interaction

• Graduate school with focus on theoretical and practical information science

• Relevant Expertise: library science storage and retrieval

• People: Peter Ingwersen, Pia Borlund, Birger Larsen

Royal School of Library and Information Science

CO

NS

OR

TIU

M Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

19

• Investigations into social structure; inequalities & collective actions; social norms, political & ethical values; social policies in EU

• Social construction of technology; knowing and learning as a collective

• Relevant Expertise: management of cultural memories, shared cultural experiences, sociology

• Relevant Regional Projects: LiveMemories

• Relevant EU Projects: LivingKnowledge, GLOCAL

• People: Fausto Giunchiglia, Silvia Gherardi, Pierre Andrews

TrentoC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

20

• Develop advanced techniques to understand and manage multimedia assets

• Relevant Expertise: semantic enrichment of multimedia, ontology construction

• People: Julián Flórez, Jorge Posada

VicomTechC

ON

SO

RT

IUM Alinari

DCU/CLARITY

DCU/Ethics

Glasgow

Leeds

Luleå

Namur

Oldenburg

Praeposit

QMUL

RSLIS

Trento

VicomTech

HUMANITIES

SCIENCES

21

CO

NS

OR

TIU

M

HUMANITIES &USE CASES

LAW & PHILOSOPHY

COGNITIVE SCIENCE

COMPUTER SCIENCE

22

CO

NS

OR

TIU

M

HUMANITIES &USE CASES

LAW & PHILOSOPHY

COGNITIVE SCIENCE

COMPUTER SCIENCE

23

CO

NS

OR

TIU

M QMUL

Alinari

VicomTech

Glasgow

RSLIS

LeedsDCU-Ethics

Namur

Trento

Oldenburg

LuleaDCU-CLARITY

Praeposit

Information Search

LifeLog Interpretation

Image ProcessingLifeLog Technology

Social Interaction

Health & Wellness

Shared Culture

Content Rights

Social Acceptance Memory Science

Library Science

Curation (Industry)

Management

24

CO

NS

OR

TIU

M7. The L2L Work Programme structure • WP1- Management (Praeposit)• WP2: Integration activities (Glasgow)• WP3: Technological Challenges in

Lifebraries (DCU) • WP4: Cognition, Memory, Emotion in

Lifebraries (Leeds)• WP5: Societal issues in Lifebraries (Namur)• WP6: Use cases and Demonstrators

(QMUL)• WP7: Dissemination, exploitation and

standardisation activities (DCU)

25

CO

NS

OR

TIU

M7. The L2L Work Programme structure • WP1- Management (Praeposit)

– Activity 1.1: Network Administration– Activity 1.2: Network Communications and Reporting to the Commission– Activity 1.3: Financial Management– Activity 1.4: Quality Control– Activity 1.5: Risk Management– Activity 1.6: Legal and Ethical Reporting– Activity 1.7: Objective tree and assessment metrics

• WP2: Integration activities (Glasgow)– Activity 2.1: Towards a Virtual Competence Centre– Activity 2.2: Researcher Mobility Program– Activity 2.3: Cooperation in Education, teaching materials and PhD formation– Activity 2.4: Lifebrary Festival

• Summer Schools• Cross-disciplinary exploration

– Activity 2.5: Pan-European Integration

• WP3: Technological Challenges in Lifebraries (DCU) – Activity 3.1: Data Preparation and Pre-Processing– Activity 3.2: Evidence by Combination Techniques– Activity 3.3: Context Modeling and User/Community Profiling– Activity 3.4: Event Detection & Summarisation– Activity 3.5: Event Visualisation Schemes– Activity 3.6: Semantic Modeling /Ontology– Activity 3.7: Community Profiling using Social Networks

• WP4: Cognition, Memory, Emotion in Lifebraries (Leeds)– Activity 4.1: Cognitive Issues in archiving and interacting with lifebraries– Activity 4.2: Memory impairment and lifebraries– Activity 4.3: Emotional issues in preserving lifebraries– Activity 4.4: Lifebrary selection and preservation models– Activity 4.5: Lifebrary curation model– Activity 4.6: Browsing, search and retrieval models in lifebraries– Activity 4.7: The self and other: What is remembered about who in lifebraries

• WP5: Societal issues in Lifebraries (Namur)– Activity 5.1: Stakeholders cartography– Activity 5.2: Case Studies– Activity 5.3: Legal Issues in Lifebrary Construction and Archiving including Privacy & Security Issues in Lifebraries– Activity 5.4: Social Impact and societal Issues– Activity 5.5: Ethical Issues– Activity 5.6: Dissemination

• WP6: Use cases and Demonstrators (QMUL)– Activity 6.1: Virtual Laboratory– Activity 6.2: Evaluation Methodology– Activity 6.3: Data Set Creation and Ground Truth Generation– Activity 6.4: Case Studies

• Autobiography-based Lifebraries• Societal Biographies-based Lifebraries• Preserving cultural heritage – Digital curation to Lifebrary

– Activity 6.5: Distributed research environment and L2L Integration Platform– Activity 6.6: Lifebrary Archive

• WP7: Dissemination, exploitation and standardisation activities (DCU)– Activity 7.1: Web portal & Promotional materials– Activity 7.2: Joint Publications– Activity 7.3: Standards and Technology Transfer– Activity 7.4: Societal Impacts– Activity 7.5: L2L Platform Challenge

26

8. A

dvi

sors

External Advisory ChamberSCIENTIFIC ADVISORY BOARD

USER GROUP

• Emma Berry, MSR (Co-chair)• Frank Nack, CWI (Co-chair)

INDUSTRIAL ADVISORY BOARD

• Jim Gemmel (Microsoft)• Peter Scott (KMI, Open U)• Steve Mann (Toronto U)• Ramesh Jain (UC Irvine)• Hari Sundaram (ASU)• Gary Marchionini (UNC)• Nick Belkin (Rutgers)• Keith Oatley (Toronto U)• Rafael Capurro (Stuttgart

Media U)

• Eric Moore (FAST/Microsoft)• John Herlihy (Google)• Geoff Cross (Vicon)• Ron Yang (FaceBook)• Ricardo Baeza Yates (Yahoo!

Research, Barcelona)• John Tait (IRF)

27

9. A

Dig

ital

Lib

rary

? Why is this a Digital Library, why this call ?

• L2L matches perfectly with the aims of the call - interdisciplinarity in composition.

• More than bridges, integrates – technological, information and archival science & practice– social and cognitive sciences– law and ethics

• Creates a new research community where non-technological partners are more than just cheerleaders.

• Lifelogging is happening, building personal lifelogs, explicitly, or implicitly, and 5 years from now …

• These lifelogs are our personal Digital Libraries. • As Libraries, they are in need of inputs from all the

different L2L communities.• We need to move from Lifelogging to Lifebraries.

28

CO

NS

OR

TIU

MQuestions:

• Collaborative research/integration vs. community building

• 13 partners belonging to 4 areas, 42 months duration, €5M Commission contribution