Assessment Cycle or Circular File: Do Academic Librarians Use Information Literacy Assessment Data? Megan Oakleaf & Lisa Hinchliffe Library Assessment Conference, August 2008
Mar 29, 2015
Assessment Cycle or Circular File:
Do Academic Librarians Use Information Literacy
Assessment Data?
Megan Oakleaf & Lisa HinchliffeLibrary Assessment Conference, August 2008
Oakleaf & Hinchliffe, LAC 2008
Methods - IL Assessment Data
• Surveys• Tests• Performance assessments
– Ex: evaluation of student bibliographies, research papers, or portfolios
• Classroom Assessment Techniques (CAT)– Ex: muddiest points or minute paper
• Etc.• Etc.
Oakleaf & Hinchliffe, LAC 2008
From the literature: (Bresciani 2009)
• Too little time/resources
• Lack of knowledge or skills
• Lack of process coordination
• Lack of conceptual framework for assessment
• Lack of collaboration with faculty
• Lack of trust
• Difficulties managing expectations
Barriers to Assessment
Oakleaf & Hinchliffe, LAC 2008
Research Questions
• Do academic librarians use information literacy assessment data? – If so, how do they use the data? – If not, why not?
• What can be done to encourage the use of information literacy assessment data in academic libraries?
Oakleaf & Hinchliffe, LAC 2008
Survey & Participant Details
• 12-question survey using SurveyMonkey
• Distributed to Information Literacy Instruction Listserv (ILI-L) and the ACRL Immersion Alumni listserv
• 437 participants – 98% academic librarians– 96% conduct IL instruction– 35% supervise people who conduct IL instruction– 31% have been a librarian 1-5 years– 21% have been a librarian 6-10 years– 24% have been a librarian 20+ years
Oakleaf & Hinchliffe, LAC 2008
Do academic librarians assess information literacy skills?
76% (332)
assess IL skills
24% (105)
do not
Barriers to Assessing IL Skills
Oakleaf & Hinchliffe, LAC 2008
Too Little Time
26% I spend my time on other work responsibilities.
21% I’m unsure how to re-allocate my responsibilities to make time for assessment.
Oakleaf & Hinchliffe, LAC 2008
Too Few Resources
29% I don’t have the necessary staff.
19% Other library services and priorities monopolize available resources.
Oakleaf & Hinchliffe, LAC 2008
Lack of Knowledge & Skills
21% I have limited understanding of or expertise in assessment.
17% I’m concerned that I am not competent in conducting assessment.
15% I don’t know how to select & implement assessment tools.
14% I don’t know how to analyze assessment data.
Oakleaf & Hinchliffe, LAC 2008
Lack of Process Coordination
29% There is no centralized support/commitment for assessment.
22% There is no assessment coordinator to assist me.
18% Assessment is not valued in my library.
15% There is no assessment committee to assist me.
Oakleaf & Hinchliffe, LAC 2008
Lack of Conceptual Framework
18% It’s hard to integrate with college-wide assessment efforts.
17% It’s hard to grasp librarians’ role in assessing student learning.
Oakleaf & Hinchliffe, LAC 2008
Fear, Anxiety, Lack of Trust
17% I’m concerned that I am not competent in conducting assessment.
Oakleaf & Hinchliffe, LAC 2008
Lack of Collaboration with Faculty
21% Faculty involvement is limited.
19% Faculty consider the library an auxiliary service.
11% Faculty do not collaborate with librarians or vice versa.
Oakleaf & Hinchliffe, LAC 2008
Difficulty Managing Expectations
17% Many assessment tools don’t adequately measure information literacy.
Oakleaf & Hinchliffe, LAC 2008
To Assess IL Skills, Librarians Need:
• More time
• Help re-allocating their time
• More staff
• Greater understanding of IL assessment
• Centralized support and/or a coordinator of assessment
• Increased faculty involvement
Oakleaf & Hinchliffe, LAC 2008
Do academic librarians who assess IL skills
have results from these assessments?
86% (276)
have results
14% (46)
do not
Barriers to Producing Assessment Results
Oakleaf & Hinchliffe, LAC 2008
Too Little Time
31% I spend my time on other work responsibilities besides producing assessment results.
24% I don’t have time for the added responsibility of producing assessment results.
21% I’m unsure how to re-allocate my responsibilities to make time for producing assessment results.
Oakleaf & Hinchliffe, LAC 2008
Too Few Resources
24% I don’t have the necessary staff to produce assessment results.
10% I won’t be rewarded for producing assessment results.
Oakleaf & Hinchliffe, LAC 2008
Lack of Knowledge & Skills
31% I have limited understanding of or expertise in producing assessment results.
14% I don’t know how to select & implement tools for producing assessment results.
Oakleaf & Hinchliffe, LAC 2008
Lack of Process Coordination
24% There is no centralized support/commitment for producing assessment results.
10% There is no assessment coordinator to assist me in producing assessment results.
10% There is no assessment committee to assist me in producing assessment results.
10% Assessment results are not valued on my campus.
Oakleaf & Hinchliffe, LAC 2008
Lack of Conceptual Framework for Assessment
17% It’s hard to integrate assessment results with college-wide assessment efforts.
Oakleaf & Hinchliffe, LAC 2008
Fear, Anxiety, Lack of Trust
10% I’m concerned that I am not competent in producing assessment results.
Oakleaf & Hinchliffe, LAC 2008
Lack of Collaboration with Faculty
24% Faculty do not collaborate with librarians or vice versa to produce assessment results.
17% Faculty involvement in producing assessment results is limited.
Oakleaf & Hinchliffe, LAC 2008
Difficulty Managing Expectations
31% Many assessment results don’t adequately measure information literacy.
17% Many assessment results don’t provide an adequate level of detail.
Oakleaf & Hinchliffe, LAC 2008
To Produce Results, Librarians Need:• More time
• Help re-allocating their time or responsibilities
• More staff
• Greater understanding of how to produce assessment results
• Centralized support for producing assessment results
• Increased faculty collaboration in producing assessment results
• Assessment results that adequately measure IL skills
Oakleaf & Hinchliffe, LAC 2008
Do academic librarians who have IL assessment results
use them?
83% (228)
use assessment results
17% (48)
do not
Barriers to Using Assessment Results
Oakleaf & Hinchliffe, LAC 2008
Too Little Time
45% I spend my time on other work responsibilities besides using assessment results.
25% I don’t have time for the added responsibility of using assessment results.
18% I’m unsure how to re-allocate my responsibilities to make time for using assessment results.
Oakleaf & Hinchliffe, LAC 2008
Too Few Resources15% I don’t have the necessary budget to use assessment
results.
15% I don’t have the necessary professional development opportunities to use the assessment results.
15% I don’t have the necessary staff to use assessment results.
15% I won’t be rewarded for using assessment results.
15% Other library services and priorities monopolize available resources for using assessment results.
Oakleaf & Hinchliffe, LAC 2008
Lack of Knowledge & Skills
40% I have limited understanding of or expertise in using assessment results.
33% I don’t know how to select & implement tools for using assessment results.
Oakleaf & Hinchliffe, LAC 2008
Lack of Process Coordination
28% There is no centralized support/commitment for using assessment results.
15% The use of assessment results is not valued on my campus.
13% The use of assessment results is not valued in my library.
13% There is no assessment committee to assist me in using assessment results.
10% There is no assessment coordinator to assist me in using assessment results.
Oakleaf & Hinchliffe, LAC 2008
Lack of Conceptual Framework
35% It’s hard to grasp expectations for using assessment results.
33% It’s hard to integrate the use of assessment results with college-wide assessment efforts.
28% It’s hard to grasp librarians’ role in using assessment results.
13% It’s hard to grasp librarians’ role in using assessment results for accreditation purposes.
Oakleaf & Hinchliffe, LAC 2008
Fear, Anxiety, Lack of Trust
25% I’m concerned that I am not competent in using assessment results.
Oakleaf & Hinchliffe, LAC 2008
Lack of Collaboration with Faculty
30% Faculty do not collaborate with librarians or vice versa to use assessment results.
28% Faculty involvement in using assessment results is limited.
Oakleaf & Hinchliffe, LAC 2008
Difficulty Managing Expectations
48% The assessment results can’t be used because they don’t adequately measure information literacy.
40% The assessment results can’t be used because they don’t provide an adequate level of detail.
Oakleaf & Hinchliffe, LAC 2008
To Use Results, Librarians Need:• More time• Greater understanding of
how to use assessment results
• Training in how to use assessment results
• Training in how to select & implement tools for using assessment results
• Centralized support for using assessment results
• Clearer expectations for using assessment results
• Greater understanding of librarians’ role in using assessment results
• Increased faculty collaboration/involvement in using assessment results
• Methods for integrating assessment results with college-wide efforts
• Assessments results that adequately measure IL skills
• Assessment results that provide an adequate level of detail about IL skills
Oakleaf & Hinchliffe, LAC 2008
What do academic librarians use IL assessment results for?
Oakleaf & Hinchliffe, LAC 2008
Ways Academic Librarians Use IL Assessment Results
96% To improve instruction
73% To increase student learning
58% To inform future assessment efforts
52% To provide data for accountability measures
50% To collaborate with faculty
36% To collaborate with other librarians
20% To argue for increased resources
Oakleaf & Hinchliffe, LAC 2008
Results Summary (n=437)
Yes No
Assess? 332 105
Analyze Results?
276 46
Use Results? 228 48
42
Closing the Loop…
Oakleaf & Hinchliffe, LAC 2008
So, what can be done to encourage academic librarians to assess IL, produce assessment
results, & use the results to achieve the purposes of
assessment?
Oakleaf & Hinchliffe, LAC 2008
What can be done to encourage collection & use of assessment data?
From the literature: (Bresciani 2009)
• Educate• Clarify• Collaborate• Coordinate
• Celebrate• Be flexible• Keep it simple
Oakleaf & Hinchliffe, LAC 2008
EducateTrain librarians about
– IL assessment in general, – tools for assessing IL skills
adequately (and in detail), – tools for producing data, and – tools for facilitating the use of
assessment data
Inform library administration about – the time and – resources required to assess IL
skills, as well as produce and use assessment results
Oakleaf & Hinchliffe, LAC 2008
Clarify
Clarify the role of librarians in assessing information literacy as well as producing & using assessment data
Oakleaf & Hinchliffe, LAC 2008
Collaborate
Collaborate with faculty, centralized campus assessment support processes and personnel.
Participate in college-wide efforts to act on assessment results.
Oakleaf & Hinchliffe, LAC 2008
Coordinate
Coordinate assessment efforts within the library by creating structures to support IL assessment.
Oakleaf & Hinchliffe, LAC 2008
Be Flexible
Re-allocate job responsibilities of those tasked with assessment duties.