W.E.B. Du Bois Library
Combining Quantitative and Qualitative Assessment of an Information Commons
orTwo Heads are Better than One – How Qualitative and Quantitative Assessment are Better Together
Rachel Lewellen and Gordon FretwellSeptember 26, 2006
Library Assessment ConferenceBuilding Effective, Sustainable,
Practical Assessment
2W.E.B. Du Bois Library
Overview
About the Learning Commons (LC)
About the observational study
About the focus groups
Conclusions based on combined data
Need for future assessment
3W.E.B. Du Bois Library
About the Learning Commons
Facilities• Located in the Library, 23,000+ square feet, 17 group
study rooms, 250+ seats, 169+ computers
Services• Reference, Writing Center, Advising, Career Services,
Technology Assistance, Peer Tutoring, Coffee/Snack Café
Access to • Software, hardware, library print and electronic
resources, wired and wireless network access, multimedia tools, etc.
4W.E.B. Du Bois Library
The Tower and the Courtyard
5W.E.B. Du Bois Library
Lounge Seating
6W.E.B. Du Bois Library
Public Computer Area
7W.E.B. Du Bois Library
Study Pod Area
8W.E.B. Du Bois Library
Group Study Room
9W.E.B. Du Bois Library
Tall Café Style Table with View of Courtyard
10W.E.B. Du Bois Library
Study Tables each with a Computer
11
Learning Commons Floor Plan
12W.E.B. Du Bois Library
The Observational Study - Questions
What services are being used? When are services being used? Under what conditions are services used? Do we have enough workstations during peak
periods of use? How does use of the same space compare before
and after the implementation of the Learning Commons?
13W.E.B. Du Bois Library
Observational Study
Revision of a similar counting survey done in 2001 For each survey period
• Once every hour that the LC was open a staff member walked around counting the individuals and activities in each of the predefined locations.
• 61 locations, service points or activities were defined and counted (group study rooms, tables, reference desk, computers used, cell phone users, sleepers etc.).
• Data was entered into Excel from tally sheets and analyzed. Six one-week surveys were completed (Oct 05-May 06)
• One shortly after opening (before 24/5 hours)• One during exam period
14
Learning Commons Usage Survey Tally SheetSpring Semester 2006
Survey Date : WEDNESDAY 5/17 Nominal Time____________Collector Time started countTime finished count
# Fixed UserSeats Count
Elevator LobbyElevator Lobby Phone & queue
Assistive Technologies services workstations - 6 total6High tables (beneath east stairs)(2@2 each) - 4 total4Tables wired for elec. but not data (12@ 2 each)24Main lounge seating area - 20 total20"Dark" seating (beneath west stairs) - 10 total10Dictionary Stands 2 total 2
Reference Workstations - 58 total58Reference Wkstn queueRef. Wkstns lounge seating along windows - 6 total6Reference Consultation OfficeReference Desk 2Reference Desk queueTunnel corridor/NW corner tables - 42 total42Reference Book Stacks
ILL Service WindowILL services window - high table - 3 total2ILL services window - low table (3@4 seats) - 12 total12
Microform/CPR entrance tables (3@4 seats) - 12 total12CPR Browsing area 2Microform storage area tables (15@6 seats) - 90 total90Atlas & Consultation casesInformation table at wall
Duplicated Sleepers tally (entire floor)
Duplicated Lap Top & PDA Users (entire floor)
15W.E.B. Du Bois Library
Observational Study Results
Building traffic increased 53% between 2001 and 2006. Use of the main floor (Learning Commons floor) increased
190% between 2001 and 2006. Overnight hours are popular all semester and especially so
at exam time. Students use of a variety of seating configurations and
locations. High-use times across a day and throughout the week were
identified with implications for staffing service points. Laptop use is substantial (10% of LC users). Cell phone use is ubiquitous but not overwhelming. 1.2%
of LC clientele were using cell phones. Use was distributed proportionally during the open hours.
16W.E.B. Du Bois Library
Daily Use of the Library BuildingTypical Week in March
0
500
1,000
1,500
2,000
2,500
3,000
3,500
Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Users FY2001
FY2006
Use Increased
17W.E.B. Du Bois Library
New Questions Raised
We wanted to know more about the areas that were most heavily used
Were the areas that were used at equal levels liked equally?
What attributes made areas heavily/lightly used?• Was noise a factor?
We wanted to know if users cared about the requirement to log in to some of the computers
We wanted information about future development• What changes would students like to see?• What would users like in a separate quiet study area?
18W.E.B. Du Bois Library
About the Focus Group Study
Follow-up to observational study Times and locations for invitations to participate
were randomly generated LC staff members/assessment librarian issued
invitations Free beverage coupons offered as incentive Three focus group meetings and an individual
interview – April 2006 Diverse mix of students
19W.E.B. Du Bois Library
Discussion Topics
Favorite and least favorite areas of the LC and their attributes
Discussion of specific areas including: study pods, study rooms, public stations, tables, and lounge chairs
Development of quiet areas Development of the courtyard
20W.E.B. Du Bois Library
Focus Group Results
Students liked the study pods area best
They liked the public station area least
They want more electrical and Ethernet connectivity
Additional feedback about remaining locations
21W.E.B. Du Bois Library
Study Pod Area
22W.E.B. Du Bois Library
Another Study Pod Picture
23W.E.B. Du Bois Library
Public Station Area
24W.E.B. Du Bois Library
Findings from Combined Data
The public stations and study pod areas were used equally but the public stations were the least favorite of all LC locations and the study pods were the favorite location for students.
Students wanted more computers with larger work surfaces.
Choice of seating location was at times driven by proximity to electrical or wired Ethernet access.
Cell phone use is common and fairly steady but students generally feel that individuals are attuned to noise and courtesy issues themselves
One set of lounge chairs had extremely low use - the location was too sunny and/or chairs were too close to computer stations –some students didn’t even know lounge chairs were there at all.
25W.E.B. Du Bois Library
Need for Future Assessment
How does the space contribute to learning? Does the LC foster a sense of community and support
diversity? What are students doing in the space - is it many different
things? Are the right services provided? Are more/others needed? Do the students feel their needs and/or expectations are
being met? What is lacking? Are faculty teaching differently or seeing changes they
attribute to the LC – what impact has this had on teaching and faculty perception of the impact on student learning?
What kind of innovation is born here? How do we know?
26W.E.B. Du Bois Library
Complete Observational Study & Focus Group Information Learning Commons Assessment
http://www.library.umass.edu/assessment/learningcommons.html
Observational Study Data• Summary data• Raw data by week• Comparative charts
Focus Group Report
Other LC assessment activities and data• Reference transactions, Café sales, Writing Center consultations,
LC survey etc.
27W.E.B. Du Bois Library
Contact Us
Gordon Fretwell Statistics ConsultantUniversity of Massachusetts [email protected]
Rachel LewellenAssessment LibrarianUniversity of Massachusetts Amherst413 545-3343 [email protected]