Computing - The Next 10 Years Universal Access to Information Raj Reddy Carnegie Mellon University Pittsburgh, USA April 6, 2001 Talk presented at Georgia.

Computing - The Next 10 YearsComputing - The Next 10 YearsUniversal Access to InformationUniversal Access to Information

Raj ReddyRaj Reddy

Carnegie Mellon UniversityCarnegie Mellon University

Pittsburgh, USAPittsburgh, USA

April 6, 2001April 6, 2001

Talk presented at Georgia Tech 10Talk presented at Georgia Tech 10thth Anniversary Convocation Anniversary Convocation

Future TechnologyFuture Technology

Computational power doubles every 18 months Computational power doubles every 18 months (Moore’s Law)(Moore’s Law) 100-fold improvement every 10 years100-fold improvement every 10 years

Disk Densities double every 12 monthsDisk Densities double every 12 months 1000-fold improvement every 10 years1000-fold improvement every 10 years

Optical bandwidth doubling every 9 monthsOptical bandwidth doubling every 9 months 10000-fold improvement every 10 years 10000-fold improvement every 10 years

Infinite Bandwidth and Memory before Infinite Bandwidth and Memory before ComputationComputation Cost decreasing, density increasingCost decreasing, density increasing

What does the future hold?What does the future hold?

We can see some glimpses of the future We can see some glimpses of the future Universities without walls,Universities without walls, Computers that never fail and self healing softwareComputers that never fail and self healing software Every home with giga PCs connected by gigabit Every home with giga PCs connected by gigabit

networksnetworks Access to all the published creative works of the world Access to all the published creative works of the world

anytime anywhere anyoneanytime anywhere anyone

Emergence of the World Bank of, not money, but Emergence of the World Bank of, not money, but KnowledgeKnowledge

Systems, so-called geriatric robotics, that help the Systems, so-called geriatric robotics, that help the disabled lead normal lives, and disabled lead normal lives, and

Systems that give the rest of us superhuman Systems that give the rest of us superhuman capabilities, like getting a month’s work done in a daycapabilities, like getting a month’s work done in a day

Universal Access to Universal Access to InformationInformation

Information at your fingertipsInformation at your fingertips

Access to all human knowledge:Access to all human knowledge:AnyoneAnyoneAnywhereAnywhereAnytimeAnytime

All Human Knowledge All Human Knowledge Recorded InformationRecorded Information

BooksBooksPeriodicals (journals, newspapers)Periodicals (journals, newspapers)Music, opera, danceMusic, opera, dancePaintings, Sculptures and MonumentsPaintings, Sculptures and MonumentsMovies, videoMovies, videoDatabases, softwareDatabases, software

Suppose all of this were on the WebSuppose all of this were on the Web

Examples from www.ulib.orgExamples from www.ulib.org

Lecture: Lecture: Michael Shamos on ULBooks: Books: A Child’s History of EnglandArt: Art: Greek Art

Collection of static contentCollection of static content Collection of dynamic multimedia contentCollection of dynamic multimedia content

Linearly organisedLinearly organised Browsable, navigableBrowsable, navigable

Selected by an Author as relatedSelected by an Author as related Selected by User as relatedSelected by User as related

Occupying a single physical locationOccupying a single physical location No physical existenceNo physical existence

Physically bound between coverPhysically bound between cover Instantly TransmittableInstantly Transmittable

What is a book? What is a book? What is a digital book ?What is a digital book ?

What is a Library?What is a Library?

Collection of itemsCollection of itemsLinearly organized (shelves)Linearly organized (shelves)Chosen by budget constraintsChosen by budget constraintsOccupying physical spaceOccupying physical spaceCataloged for accessCataloged for access

What is a Digital Library?What is a Digital Library?

Collection of digital itemsCollection of digital items(potentially huge(potentially huge))

Encompassing everything (someday) Encompassing everything (someday) Organized arbitrarilyOrganized arbitrarilyOccupying no physical spaceOccupying no physical spaceFully content-searchableFully content-searchable

Universal Library ImplicationsUniversal Library Implications

Elimination of time, space, cost constraintsElimination of time, space, cost constraints Democratization of informationDemocratization of information

““Knowledge is power”Knowledge is power” Hyperlinks to related informationHyperlinks to related information Preservation and Dissemination of Preservation and Dissemination of

KnowledgeKnowledge faster and widerfaster and wider Backup preservationBackup preservation

Preservation of culturePreservation of culture

Universal Library ImplicationsUniversal Library Implications

ResearchResearchWeb of scholarly information, reviewsWeb of scholarly information, reviews

TeachingTeachingSupport for distance educationSupport for distance educationAcademic publishingAcademic publishingVirtual museumsVirtual museums

InteractivityInteractivity

Universal Library ApplicationsUniversal Library Applications

Acess to “Born Digital” InformationAcess to “Born Digital” InformationWorld produces a Billion Billion(10World produces a Billion Billion(101818) )

bytes of information every year(Lyman bytes of information every year(Lyman and Varian)and Varian)

90% is stored digitally90% is stored digitallyDigital museumDigital museumDigital tour guideDigital tour guide

What’s in the Taj Mahal?What’s in the Taj Mahal?

Universal Library ApplicationsUniversal Library Applications

Research assistantResearch assistantWhat did Newton write about color?What did Newton write about color?What are Moslem views on race?What are Moslem views on race?

Teaching resourceTeaching resource““Act out” books in virtual realityAct out” books in virtual realityReal-time explanationsReal-time explanations

Business informationBusiness informationData miningData mining

We Can Store EverythingWe Can Store Everything

1 book = 500 pp. 1 book = 500 pp. 1MB uncompressed – 300KB compressed1MB uncompressed – 300KB compressed101088 to 3x 10 to 3x 1088 books = ~10 books = ~101414 bytes = 100 bytes = 100

terabytesterabytes

Over 100 million computers on the Over 100 million computers on the InternetInternetAt 1 GB each, >100 petabytes At 1 GB each, >100 petabytes nownow

1 GB of disk costs ~$31 GB of disk costs ~$3100 terabytes < $300 thousand to $1 million100 terabytes < $300 thousand to $1 million

Non-textual MaterialNon-textual Material

1 Movie = 10 GB1 Movie = 10 GB 1 petabyte = 100,000 movies1 petabyte = 100,000 movies All the movies ever made!All the movies ever made!

AudioAudio 1 petabyte = 3000 1 petabyte = 3000 yearsyears of music of music All music ever performed or recordedAll music ever performed or recorded

Paintings and Photos @ 1 MBPaintings and Photos @ 1 MB 1 petabyte = 1 billion painting or photos1 petabyte = 1 billion painting or photos

Non-textual MaterialNon-textual Material

Gore’s Digital EarthGore’s Digital Earth ““A multi-resolution, three-dimensional A multi-resolution, three-dimensional

representation of the planet, into which we can representation of the planet, into which we can embed vast quantities of geo-referenced data.”embed vast quantities of geo-referenced data.”

Area of Earth Area of Earth 1/21/2 peta m peta m22 1000 bytes/m1000 bytes/m22 feasible feasible2 MB/m2 MB/m22 not practical yet not practical yet 10102121 bytes bytes

= 1 zettabyte = 1 zettabyte {peta-, exa-, zetta-, yotta-}{peta-, exa-, zetta-, yotta-}

Technological ChallengesTechnological Challenges

Input (scanning, digitizing, OCR)Input (scanning, digitizing, OCR)Data representationData representation

text, notations, images, web pagestext, notations, images, web pagesNavigation and SearchNavigation and SearchMultilingual IssuesMultilingual IssuesOutput (voice, pictures, virtual Output (voice, pictures, virtual

reality)reality)Synthetic DocumentsSynthetic Documents

Universal Library DesignUniversal Library Design

ModularModularTechnology plug-ins (e.g. machine Technology plug-ins (e.g. machine

translation)translation)DistributedDistributed

Mirror sitesMirror sitesMultiple interfacesMultiple interfaces

Human (languages, cultures, literacy)Human (languages, cultures, literacy)MachineMachine

Universal Library DesignUniversal Library Design

Speech input/outputSpeech input/outputPictorial outputPictorial outputLanguage supportLanguage support

Translation assistantsTranslation assistantsSummarization toolsSummarization toolsSynthetic documentsSynthetic documents

Encyclopedia-on-demandEncyclopedia-on-demand

Input IssuesInput Issues

Non-digital mediaNon-digital mediaConversion, scanning, correctionConversion, scanning, correctionTriple keyboard, uncorrected OCRTriple keyboard, uncorrected OCR

Digital mediaDigital mediaFormats, conversions, color Formats, conversions, color

representationrepresentationASCII, HTML, SGML, XML, PDF, PS, TEXASCII, HTML, SGML, XML, PDF, PS, TEXJPEG, TIFF, GIF? JPEG, TIFF, GIF?

Input IssuesInput Issues

Structured matterStructured matterMusical notation, LabanMusical notation, LabanChemistryChemistry

3D Items3D ItemsResource allocation (what’s Resource allocation (what’s

first?)first?)Duplication of effort (no registry)Duplication of effort (no registry)

MetadataMetadata

Data Data aboutabout an item not part of the an item not part of the itemitemBibliographicBibliographicFormat, medium, encoding, resolutionFormat, medium, encoding, resolutionProvenanceProvenanceReliability, integrityReliability, integrityPermissionsPermissions

Who generates metadata?Who generates metadata?

NavigationNavigation

Browsing, finding, searching, flyingBrowsing, finding, searching, flying

Fractal viewFractal viewKeys are Keys are granularitygranularity and and

connectivityconnectivityView whole collections or one glyphView whole collections or one glyph

UnderstandingUnderstanding structurestructure of of informationinformation

Making Sense Of The World’s Knowledge

Searching MathematicsSearching Mathematics

0

2sin2

dxxe x

4/92

22

Searching MathematicsSearching Mathematics

0

2sin2

dxxe x

MATHEMATICA Canonical Form:

Integrate[

Times[Power[E,Times[-1,Power[V1,2]]],

Sin[Power[V1,2]]],

{V1,0,Infinity}]

Multilingual IssuesMultilingual Issues

Character setsCharacter setsRepresentationsRepresentations

Íîäà ôèçè÷åñêè íàõîäèòñÿ â çäàíèè ÈçâåñòèéÍîäà ôèçè÷åñêè íàõîäèòñÿ â çäàíèè Èçâåñòèé

Нода физически находится в здании ИзвестийНода физически находится в здании Известий

Multilingual navigationMultilingual navigationTranslation assistanceTranslation assistance

Synthetic DocumentsSynthetic Documents

Documents derived Documents derived automatically from retrieved automatically from retrieved informationinformationMultilingual translationMultilingual translation

Abstracts, summaries, glossariesAbstracts, summaries, glossariesEncyclopedia-on-demandEncyclopedia-on-demand

Information ReliabilityInformation Reliability

Existence Existence validity validityUniversal Library PhilosophyUniversal Library Philosophy

Avoid value judgments Avoid value judgments Provide information from which Provide information from which

usersusers(and programs) can assess validity(and programs) can assess validity

Source, reputation, recency, Source, reputation, recency, reviews, consistencyreviews, consistency

Scaling ProblemsScaling Problems

Search services (e.g. Altavista) Search services (e.g. Altavista) index >10index >1088 documents documentsSuppose there were 10Suppose there were 101212 ? ?

How can a billion users access How can a billion users access the same item at once?the same item at once?

Policy ChallengesPolicy Challenges

Use of copyrighted materialUse of copyrighted materialEconomics (Who pays? Who Economics (Who pays? Who

gets?)gets?)PrivacyPrivacyReliability of informationReliability of informationChange in the nature of teachingChange in the nature of teaching

Use Of ©Use Of © ContentContent

Philosophy: must pay for usePhilosophy: must pay for useAuthors, publishers will not sufferAuthors, publishers will not suffer

Implied licenseImplied licenseAutomated permissionsAutomated permissionsBulk licensing Bulk licensing Compulsory licensingCompulsory licensing

Owner CAN’T refuse; user MUST Owner CAN’T refuse; user MUST paypay

EconomicsEconomics

Flat-fee subscriptions (e.g. HBO)Flat-fee subscriptions (e.g. HBO)Metered use (electric company)Metered use (electric company)Microcharge (Tobias “clickl”)Microcharge (Tobias “clickl”)Free (paid by government)Free (paid by government)Automated permissionsAutomated permissionsUse measured by technologyUse measured by technology

Operating ModelOperating Model

Single portal for access to Single portal for access to allall informationinformation

Universal Library provides input, Universal Library provides input, access, multilingual, output and access, multilingual, output and synthesis toolssynthesis tools

Universal Library will be a model Universal Library will be a model scanning operationscanning operation

Registry of digitized worksRegistry of digitized works

Operating ModelOperating Model

Specialized collections curated Specialized collections curated by specialists, provided to by specialists, provided to Universal LibraryUniversal Library

Foreign collection performed in Foreign collection performed in foreign countriesforeign countries

Universal Library will be mirrored Universal Library will be mirrored in ~12 sites around the worldin ~12 sites around the world

Universal Library StatusUniversal Library Status>13,000 digital volumes>13,000 digital volumesArtArtNewspapersNewspapersMusic, videoMusic, videoPortal to hundreds of other collectionsPortal to hundreds of other collections

Visit Visit http://www.ulib.orghttp://www.ulib.org

ProjectsProjects

NavigatorNavigatorAcademic electronic publishingAcademic electronic publishingElectronic Union CatalogElectronic Union CatalogBooks out of copyrightBooks out of copyright

books out of print books out of printSoftware distributionSoftware distribution

Conclusions and Conclusions and RecommendationsRecommendations

ConclusionsConclusions Barely 10% of all public information is available on the Barely 10% of all public information is available on the

InternetInternet Government needs to play a leadership role in developing Government needs to play a leadership role in developing

digital librariesdigital libraries Significant technical and operational challenges in Significant technical and operational challenges in

migrating and maintaining holdings in digital formmigrating and maintaining holdings in digital form Intellectual Property rights need to be addressed to Intellectual Property rights need to be addressed to

facilitate creation and access digital librariesfacilitate creation and access digital libraries

RecommendationsRecommendations Support research: meta data, scalability, multiple Support research: meta data, scalability, multiple

languages, security, and usabilitylanguages, security, and usability Create testbeds: million book projectCreate testbeds: million book project Place all public governmental information onlinePlace all public governmental information online Preserve IP rights of creators by creating tax incentives Preserve IP rights of creators by creating tax incentives

for public use of online copyrighted informationfor public use of online copyrighted information