Improving Graduate Education with Digital Libraries
Eröffnung des DFG-Graduiertenkollegs 1223“Qualitätsverbesserung im E-Learning durch
rückgekoppelte Prozesse”Darmstadt, GE, September 14, 2006
Edward A. [email protected] http://fox.cs.vt.edu
Outline
AcknowledgementsETDs and NDLTDDL curriculum and resource developmentPrior work on quality and success models DL success model
From end user perspective
Summary and Conclusions
Acknowledgements (selected)
• Colleagues: Lillian Cassel, Debra Dudley, Weiguo Fan, Marcos Gonçalves, Doug Gorton, Rohit Kelapure, Neill Kipp, Aaron Krowne, Ming Luo, Uma Murthy, Manuel Perez, Ananth Raghavan, Rao Shen, Hussein Suleman, Srinivas Vemuri, Layne Watson, …
• Sponsors: ACM, AOL, CAPES, DFG, IBM, Microsoft, NSF (IIS-9986089, 0086227, 0080748, 0325579, 0535057, 0535060; ITR-0325579; DUE-0121679, 0136690, 0121741, 0333601), SUN, …
Acknowledgements: ETD Mtgs• 1987 mtg in Ann Arbor: UMI, VT, …• 1992 mtg in Washington: CNI, CGS, UMI, VT and 10
universities with 3 reps each• 1993 mtg in Atlanta to start Monticello Electronic Library
(regional, US Southeast): SURA, SOLINET• 1994 mtg at VT: std: PDF + SGML + multimedia objects• 1996 funding by SURA, US Dept. of Education (FIPSE)• 1997 meetings in UK, Germany, ...• 1998 – 1st symposium – Memphis (20)• 1999 – 2nd symposium – Blacksburg (70)• 2000 – 3rd symposium – St. Petersburg (225)• 2001 – 4th symposium – Caltech (200)• 2002 – 5th syposium – BYU, Provo, Utah• 2003 – 6th syposium – Berlin (215) • 2004 – 7th syposium – U. Kentucky• 2005 – 8th syposium – Sydney, Australia
Acknowledgements:Future ETD Conferences
• 2007 – 10th symposium– Uppsala University, Sweden– 13-16 June
• 2008 – 11th symposium– Dartington College of Arts, Devon, UK– 29 June – 2 July (tentative)
Outline
AcknowledgementsETDs and NDLTDDL curriculum and resource developmentPrior work on quality and success models DL success model
From end user perspective
Summary and Conclusions
The Networked Digital Library of Theses and Dissertations
www.NDLTD.org
Leader of the Worldwide ETD(Electronic Thesis and Dissertation) Initiative
Training AuthorsExpanding Access
Preserving KnowledgeImproving Graduate Education
Enhancing Scholarly CommunicationEmpowering Students & Universities
NDLTD Incorporation
• Networked Digital Library of Theses and Dissertations incorporated May 20, 2003 in Virginia, USA
• Charitable and educational purposes (501 c 3)
• Officers– Executive Director (Ed Fox)– Secretary (Gail McMillan)– Treasurer (Scott Eldredge)
Board of Directors• Suzie Allard (ETD 2004, U. Kentucky)• Denise A. D. Bedford (World Bank)• Julia C. Blixrud (ARL, SPARC)• José Luis Borbinha (Natl Lib Portugal)• Tony Cargnelutti (Ex Libris)• Vinod Chachra (VTLS)• William Clark (Ohio State U.)• Susan Copeland (RGU, UK)• Jude Edminster (Bowling Green St. U.)• Scott Eldredge (Treasurer, ETD 2002,
BYU)• Edward A. Fox (Exec Director,Virginia
Tech)• John H. Hagen (West Virginia U.)• Thomas B. Hickey (OCLC)• Christine Jewell (U. Waterloo, Canada)• Joan K. Lippincott (CNI)
• Austin McLean (Proquest)• Gail McMillan (Secretary, Virginia Tech)• Joseph Moxley (ETD 2000, USF)• Eva Müller (U. Uppsala, Sweden)• Ana Pavani (PUC Rio, Brazil)• Janice Rickards (chair of ADT)• Sharon Reeves (National Library Canada)• Peter Schirmbacher (ETD 2003, Humboldt)• Samson Soong (Hong Kong U. Science &
Technology)• Hussein Suleman (U.Cape Town, S. Africa)• Shalini R. Urs (U. Mysore, India)• Eric F. Van de Velde (ETD 2001, Caltech)• Ellen Wagner (Adobe)
Some Countries• Argentina• Australia• Belgium• Brazil• Canada• Chile• China, Hong Kong• Columbia• Finland• France• Germany• Greece• India• Italy• Jamaica• Korea• Lithuania• Malaysia• Mexico• Namibia• Netherlands
• Namibia• Netherlands• Norway• Peru• Poland• Russia• Singapore• S. Africa• S. Korea• Spain• Sudan• Sweden• Switzerland• Taiwan• Thailand• Turkey• UK• Ukraine• United Arab Emirates• USA• Venezuela• Yugoslavia
OCLC SRU Interface
ETD Union Search Mirror Site in China (CALIS)(http://ndltd.calis.edu.cn – popular site!)
Outline
AcknowledgementsETDs and NDLTDDL curriculum and resource developmentPrior work on quality and success models DL success model
From end user perspective
Summary and Conclusions
DL Curriculum Development Project
• Collaborative Research launched by:- Department of Computer Science, Virginia Tech
- School of Information and Library Science, University of North Carolina, Chapel Hill
• Three year (2006 - 2008) funded project
Project Teams/NSF Grant
• Project Team at VT (IIS-0535057): – PI: Dr. Edward A. Fox ([email protected]) – GRA: Seungwon Yang ([email protected])
• Project Team at UNC-CH (IIS-0535060): – Co-PI: Dr. Barbara Wildemuth ([email protected]) – Co-PI: Dr. Jeffrey Pomerantz ([email protected]) – GRA: Sanghee Oh ([email protected])
Development / Evaluation PlanJanuary 2006 – December 2008
· From research team (VT & UNC)
· From current courses at VT & UNC
· From Advisory Board· From CC 2001
Vision/plan
Design
· Modules· Lessons
· Inspection by Advisory Board
· Inspection by external experts
· Inspection by Doctoral Consortium participants
Evaluate
· CC 2001 context· Curricular needs· Student background
Analyze· Specific strengths· Specific weaknesses
Feedback
· At UNC & VT· At additional universities
(in CS & LIS programs)
Revise & Implement
· Modules ready for use
· Lessons ready for use
Products
· Teacher perceptions· Student perceptions· Student outcomes
Evaluatein the field
DL Topics in 19 Modules (original)
Outline
AcknowledgementsETDs and NDLTDDL curriculum and resource developmentPrior work on quality and success
models DL success model
From end user perspective
Summary and Conclusions
has an exampleIS success model
Prior Work on Measuring DL Success
DeLone et al.
Seddon
has an exampleInformation seekingbehavior model
Ellis
Kuhlthau
has an exampleDL quality model Gonçalves
has an exampleUsability of DLsinspection of NCSTRL
evaluation of ACM, IEEE-CS,NCSTRL, and NDLTD
evaluation of ADL
evaluation of ADEPT
has an indicator
system usage
has an exampleTechnology
acceptance model Venkateshpredict Intention to re/use
Behavioral Attitude & Intension to Use — Venkatesh Model of IT Adoption
intention to use system
performance expectancy
system usage
Unified Theory of Acceptance and Use of Technology
effort expectancy
social influence
facilitating conditions
• Performance expectancy: – perceived usefulness, extrinsic motivation, job-fit, relative advantage,
and outcome expectations
• Effort expectancy– the degree of ease associated with the use of system
• Social influence– Subjective norms, social factors, and image
• Facilitating conditions– the degree to which an individual believes that an organizational and
technical infrastructure exist to support the system
Venkatesh Model of IT Adoption
DeLone and McLean Model of IS Success
System Quality
Information Quality
UserSatisfaction
IndividualImpact
OrganizationImpact
Use
Seddon Model of IS Success
System Quality
Information Quality
UserSatisfactions
Net Benefits to:
IndividualsOrganizations
Society
PerceivedUsefulness
Outline
AcknowledgementsETDs and NDLTDDL curriculum and resource developmentPrior work on quality and success models DL success model
From end user perspective
Summary and Conclusions
DL Success Model
Synthesize IS success and adoption models DL quality model (Gonçalves) Information life cycle model Information-seeking behavior models (Ellis’ and
Kuhlthau’s)
From end user perspective
crea
tion
distributionsee
kingutilization
search
ing,
browsing,
recommending
storing, archiving,
networkingau
thor
ing,
mod
ifyin
g,
desc
ribin
g org
anizi
ng, i
ndex
ing
pres
erva
bilit
y,
sim
ilari
ty,
tim
elin
ess,
accuracy, completeness,
conformance accessibility, preservability
pertinence,significance,
timelinessre
levan
ce
Activ
e Semi-active
Inactive
accessibility,accessing,
filteringGonçalves et al.
crea
tion
distributionsee
kingutilization
E1:starting
E2: chaining
E3: browsin
g
E4: diffe
rentia
ting
E5: monito
ring
E6: extra
cting
storing, archiving,
networking
K1:
in
itia
tion
K2:
se
lectio
n
K3:
explorationK4:
formulation
K5: collection
K6:
presentation
auth
orin
g, m
odify
ing,
desc
ribin
g org
anizi
ng, i
ndex
ing
pres
erva
bilit
y,
sim
ilari
ty,
tim
elin
ess,
accuracy, completeness,
conformance accessibility, preservability
DL Success Constructs
Activ
e Semi-active
Inactive
E: Ellis’ modelK: Kuhlthau’s model
satisfaction
performanceexpectancy
(PE)
behavioralIntention to
(re)use
relevance adequacy timeliness
reliability understandability scope
accessibilityease of use
joy of use reliability
userinterface
social influence (SI)
information quality(IQ)
system quality(SQ)
DL Success Model
information quality(IQ)
DL Concepts Regarding Information
collection metadata catalog
digital object metadata specification
repository
consist of
associated with
associated with
consist of consist of
accessibilitypreservability
pertinencerelevancesimilarity
significancetimeliness
digital object
accuracycompletenessconformance
metadata specification
impact factorcompletenesscollection
consistencycompletenesscatalog
consistencycompletenessrepository
understandabilityrelevancetimelinessreliabilityadequacy
digital objectmetadata specification
adequacy
catalogcollection
scope
repository
Information Quality (IQ)
Digital Library IQ
satisfaction
performanceexpectancy
(PE)
behavioralIntention to
(re)use
relevance adequacy timeliness
reliability understandability scope
accessibilityease of use
joy of use reliability
userinterface
social influence (SI)
information quality(IQ)
system quality(SQ)
DL Success Model
system quality(SQ)
screen designnavigation
user interface
composabilityreusability
extensibilityefficiency
effectivenessreliability
serviceaccessibility
reliabilityease of usejoy of use
System Quality(SQ)
performanceexpectancy
Digital Library SQ
DL success
Construct
seeking phase utilization phase
starting (E1/K1)
selection exploration
(E2-E6)/(K2-K3)
formulation (K4)
collection (K5)
presentation (K6)
social influence DL visibility
information quality adequacy, scope accuracy
system quality ease of use
joy of use (interface)
accessibility accessibility accessibility
performance
expectancy
usefulness
(interface)
DL Success Constructs Associated with Seeking and Utilization Phases
system quality (SQ)
performance expectancy (PE)
society, scenario,
space
service
accessibility
reliability
ease of use
joy of use
composabilityefficiency
effectiveness
extensibility
reusability
reliability
accessibility accuracy
completeness consistence conformance
pertinence preservability
relevance significance
similarity
timeliness
DL quality
dimension
information quality (IQ)
stream, structure
digital object
metadata
collection
catalog
repository
adequacy
relevance
reliability
scope
timeliness
understandability
social influence (SI)
DL success construct
society
5S and
DL concept
DL visibility
DL success
manifest variable
system quality (SQ)
performance expectancy (PE)
society, scenario,
space
service
accessibility
reliability
ease of use
joy of use
composabilityefficiency
effectiveness
extensibility
reusability
reliability
accessibility accuracy
completeness consistence conformance
pertinence preservability
relevance significance
similarity
timeliness
DL quality
dimension
information quality (IQ)
stream, structure
digital object
metadata
collection
catalog
repository
adequacy
relevance
reliability
scope
timeliness
understandability
social influence (SI)
DL success construct
society
5S and
DL concept
DL visibility
DL success
manifest variable
Outline
AcknowledgementsETDs and NDLTDDL curriculum and resource developmentPrior work on quality and success models DL success model
From end user perspective
Summary and Conclusions
Summary and Conclusions
ETDs and NDLTD progressing wellDL curriculum and resource development
with iterative refinementFoundation for defining success of DLs from
the view of DL end users Assumes a multi-theoretical perspectiveSynthesizes many related research areas in
terms of theory and empirical work
Selected Links - http://fox.cs.vt.edu
• DL curriculum - http://curric.dlib.vt.edu/wiki, http://curric.dlib.vt.edu/DLcurric.html
• NDLTD (electronic theses and dissertations worldwide)– www.ndltd.org and etdguide.org
• Virginia Tech Digital Library Research Laboratory (DLRL, www.dlib.vt.edu)