MyLifeBits Jim Gemmell & Gordon Bell SDForum Distinguished Speaker Series February 19, 2004.

MyLifeBitsJim Gemmell & Gordon BellJim Gemmell & Gordon Bell

SDForum Distinguished Speaker SeriesSDForum Distinguished Speaker SeriesFebruary 19, 2004February 19, 2004

Outline

What is MyLifeBitsWhat is MyLifeBits History/MotivationHistory/Motivation MyLifeBits system outlineMyLifeBits system outline DemoDemo Future workFuture work

MyLifeBits is:An experiment in lifetime storageAn experiment in lifetime storage

Digitizing Gordon Bell’s pastDigitizing Gordon Bell’s pastCapturing more of his futureCapturing more of his future

A software systemA software systemCaptureCaptureStorage & retrievalStorage & retrievalOrganization & annotationOrganization & annotation

Minimum requirement: fulfill Vannevar Minimum requirement: fulfill Vannevar Bush’s 1945 “Memex” visionBush’s 1945 “Memex” vision

MemexAs We May Think, Vannevar Bush, 1945

““A memex is a device in which an individual stores all A memex is a device in which an individual stores all his books, records, and communications, and which his books, records, and communications, and which is mechanized so that it may be consulted with is mechanized so that it may be consulted with exceeding speed and flexibility”exceeding speed and flexibility”

Full-text search, text & audio annotations, and Full-text search, text & audio annotations, and hyperlinkshyperlinks

I am data

History

Started in telepresence & teleworkStarted in telepresence & teleworkRemote worker needs everything in Remote worker needs everything in

notebooknotebookRediscovery of the PCRediscovery of the PCStarted scanning…Started scanning…

Our guinea pig,Gordon Bell, has digitized a lifetime’s worth of:

Books written (and read when possible)Books written (and read when possible) Personal documents (letters, bills, legal, Personal documents (letters, bills, legal,

papers written, memos, email…)papers written, memos, email…) PhotosPhotos CD collectionCD collection Posters, paintingsPosters, paintings Home moviesHome movies And, of course, all files on his PCAnd, of course, all files on his PC Now recording: telephone calls, more video, Now recording: telephone calls, more video,

all web pages visited, usage logging, radio, all web pages visited, usage logging, radio, TV…TV…

Lifetime storage Suppose you began keeping:Suppose you began keeping:

100 email messages a day (5KB each)100 email messages a day (5KB each) 100 web pages day (50KB each)100 web pages day (50KB each) 5 scanned pages a day (100KB each)5 scanned pages a day (100KB each) 1 book every 10 days (1 MB each)1 book every 10 days (1 MB each) 10 photos per day (400 KB JPEG each)10 photos per day (400 KB JPEG each) 8 hours per day of sound - e.g. telephone,8 hours per day of sound - e.g. telephone,

voice annotations, and meeting recordings (8 Kb/s)voice annotations, and meeting recordings (8 Kb/s) 1 new music CD every 10 days (45 min each at 128 Kb/s)1 new music CD every 10 days (45 min each at 128 Kb/s)

It will take you 5 years to fill up your 80 GB driveIt will take you 5 years to fill up your 80 GB drive By that time, you can buy a terabyte hard drive – which will By that time, you can buy a terabyte hard drive – which will

take more than 60 years to filltake more than 60 years to fill

0

0.2

0.4

0.6

0.8

1

2002 2003 2004 2005 2006 2007

Dis

k ca

pac

ity

(TB

)

Trying to fill a terabyte in a year Gordon’s lifetime collection < 30 GB Gordon’s lifetime collection < 30 GB

(12 GB is music CDs)(12 GB is music CDs)

ItemItem Per TBPer TB Per dayPer day

Photo Photo (400 KB JPEG)(400 KB JPEG) 2.7M photos2.7M photos 7254 photos7254 photos

1 MB document1 MB document 1.0M docs1.0M docs 2872 docs2872 docs

128 kb/s audio128 kb/s audio 18.6K hours18.6K hours 51 hours51 hours

256 kb/s video256 kb/s video 9.3K hours9.3K hours 26 hours26 hours

1.5 Mb/s video1.5 Mb/s video 290 hours290 hours 4 hours4 hours

“yet if the user inserted 5000 pages of material a day it

would take him hundreds of years to fill the repository, so that he can be profligate and

enter material freely”-Vannevar Bush, 1945

Why bother? An existence proof.The following exist in abundance:

Shoeboxes full of photosShoeboxes full of photos Photo albums & framed photosPhoto albums & framed photos

Creative Memories is a thriving business selling resources for Creative Memories is a thriving business selling resources for created high-end photo albums that are well laid out and highly created high-end photo albums that are well laid out and highly annotated, using long-lasting materials.annotated, using long-lasting materials.

Home videosHome videos Bookshelves and filing cabinetsBookshelves and filing cabinets Old bundles of lettersOld bundles of letters Professional video/photo companies do capture at kids’ Professional video/photo companies do capture at kids’

sports events and sell content like hotcakessports events and sell content like hotcakes Probably not accessed very often but Probably not accessed very often but TREASURED TREASURED

(what’s the one thing you would save in a fire?)(what’s the one thing you would save in a fire?)

Why bother? ..more reasons To eliminate physical storage (paper, CDs…)To eliminate physical storage (paper, CDs…) It costs more (in time) to delete than the cost the It costs more (in time) to delete than the cost the

storagestorage You may only want to retrieve one of many items You may only want to retrieve one of many items

in the future, but cannot predict which one in the future, but cannot predict which one (which is why you file many things now)(which is why you file many things now)

For posterity and nostalgiaFor posterity and nostalgia For memory enhancement & faster searchFor memory enhancement & faster search

(search your LifeBits rather than the web … a single (search your LifeBits rather than the web … a single source to look for anything you have ever seen)source to look for anything you have ever seen)

Let content analysis and data mining discover Let content analysis and data mining discover trends and correlations in your lifetrends and correlations in your life

So you’ve got it – now what do you do with it?

Can you find anything?Can you find anything?Can you organize that many objects?Can you organize that many objects?Once you find it will you know what it is?Once you find it will you know what it is?Once you’ve found it once, could you find Once you’ve found it once, could you find

it again?it again?

““A record if it is to be useful … must be A record if it is to be useful … must be continuously extended, it must be stored, continuously extended, it must be stored, and above all it must be consulted” and above all it must be consulted”

““The difficulty seems to be, not so much that The difficulty seems to be, not so much that we publish unduly … but rather that we publish unduly … but rather that publication has been extended far beyond publication has been extended far beyond our present ability to make real use of the our present ability to make real use of the record” record”

- Vannevar Bush- Vannevar Bush

MyLifeBits Software

MyLifeBits store

database

Voice Voice annotation annotation tooltool

Text Text annotation annotation tooltool

Telephone Telephone capture toolcapture tool

TV capture TV capture tooltool

TV EPG TV EPG download download tooltool

Radio Radio capture toolcapture tool

Radio EPG Radio EPG tooltool

PocketPC PocketPC transfer transfer tooltool

PocketRadio PocketRadio playerplayer

Import filesImport files

MyLifeBits MyLifeBits ShellShell

files

Legacy Legacy applicationsapplications

Browser Browser tooltool

InternetInternet

IM captureIM capture

MAPI MAPI interfaceinterface

Legacy Legacy email clientemail client

Making the store useful Encourage annotationsEncourage annotations Alternatives to hierarchy for organizationAlternatives to hierarchy for organization Exploiting database featuresExploiting database features

Everything goes in a database

You need all the features of a databaseYou need all the features of a database(Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, (Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication)replication)

If you don’t use one, you will find yourself creating one!If you don’t use one, you will find yourself creating one! Files as blobs, also sync with file system for legacy appsFiles as blobs, also sync with file system for legacy apps

SQLSQL

Entities & Links

AnnotatesAnnotates

Caller in Phone CallCaller in Phone Call

Photo of EventPhoto of Event

TranscludesTranscludes

MyLifeBits Schema(simplified)

ImagesImages

MusicMusic

Phone callsPhone calls

ResourcesResources

Relation-Relation-shipsships

Relation-Relation-ship typesship types

Entity typesEntity types

Resource Resource entitiesentities

Event Event typestypes

Event logEvent log

EventsEvents

TasksTasks

PeoplePeople

NotesNotes

Email Email MessagesMessages

Saved searchesSaved searches

DEMO

Summary

We have entered an era of virtually We have entered an era of virtually unlimited storage, enabling the lifetime unlimited storage, enabling the lifetime store (“Memex”)store (“Memex”)

To make the store useful we need To make the store useful we need annotation, typed links, and database annotation, typed links, and database featuresfeatures

Future work:Content analysis & Data Mining

“Creative thought and essentially repetitive thought are very different things. For the

latter there are, and may be, powerful mechanical aids” – Vannevar Bush

Is MyLifeBits just enough rope to hang yourself with?Is MyLifeBits just enough rope to hang yourself with? MyLifeBits must become MyPersonalAssistantMyLifeBits must become MyPersonalAssistant Content analysis and data miningContent analysis and data mining Doc similarity & “clean living” Doc similarity & “clean living” Document meta-data extractionDocument meta-data extraction

Future work: UI

Focus to date on Focus to date on functionality – now we functionality – now we need to make it need to make it friendlierfriendlier

User studies with User studies with SFSUSFSU

Future work: Visualizations Don't give me a little card Don't give me a little card

image and say, "That's all image and say, "That's all you've got, because that's you've got, because that's what I thought you should what I thought you should want for your virtual want for your virtual shoebox." There have got shoebox." There have got to be multiple modalities to be multiple modalities and the designers have to and the designers have to be able to deal with that. be able to deal with that. … don't metaphor me in, … don't metaphor me in, don't give me only one don't give me only one way of looking at things.way of looking at things.

-Andy van Dam, Hypertext '87 Keynote -Andy van Dam, Hypertext '87 Keynote AddressAddress

Next MediaNext Media

Web ScoutWeb Scout

U. MarylandU. Maryland IN-SPIRE

Future work: new capture modes/devices

www.MyLifeBits.com

BONUS SLIDES

Dear Appy, How committed are you? Signed, Lost and Forgotten Data

Dear Appy,Dear Appy,I'm having trouble with long-term commitment -- not on my end, I'm having trouble with long-term commitment -- not on my end, heaven knows, but from the apps that created me and with whom I heaven knows, but from the apps that created me and with whom I like to associate. Over time, these pesky apps evolve and they like to associate. Over time, these pesky apps evolve and they simply don't recognize the data that they once helped create! But, simply don't recognize the data that they once helped create! But, we data progeny -- and there are lots of us -- feel that as our we data progeny -- and there are lots of us -- feel that as our creators, these apps should be responsible for eternal support. creators, these apps should be responsible for eternal support. But the little problem with recognition isn't the worst of it – But the little problem with recognition isn't the worst of it – sometimes the apps even disappear altogether. I ask you, is it sometimes the apps even disappear altogether. I ask you, is it expecting too much for 20-something year old data like me to be expecting too much for 20-something year old data like me to be interpretable by my app (e.g. Acrobat, DB2, Draw, Eudora, Office, interpretable by my app (e.g. Acrobat, DB2, Draw, Eudora, Office, Quicken, or RealNetworks), or am I just associating with Quicken, or RealNetworks), or am I just associating with irresponsible apps? irresponsible apps? If things continue on their current path, it seems I will be completely If things continue on their current path, it seems I will be completely un-interpretable within 20 to 50 years! My apps will move to other un-interpretable within 20 to 50 years! My apps will move to other platforms, or evolve to be more Internet- or Next-Big-Thing-centric...platforms, or evolve to be more Internet- or Next-Big-Thing-centric...

By Gordon Bell http://research.microsoft.com/~gbell