LIRG SCAN AWARD 2010 A REVIEW OF EXISTING QUANTITATIVE AND QUALITATIVE EVALUATION METHODOLOGIES FOR DEMONSTRATING THE VALUE OF PUBLIC LIBRARIES Christine Rooney-Browne PhD Researcher, University of Strathclyde, Glasgow Umbrella Conference 2011
Nov 15, 2014
LIRG SCAN AWARD 2010
A REVIEW OF EXISTING QUANTITATIVE AND QUALITATIVE
EVALUATION METHODOLOGIES FOR DEMONSTRATING THE VALUE OF
PUBLIC LIBRARIES
Christine Rooney-BrownePhD Researcher, University of Strathclyde, Glasgow
Umbrella Conference 2011
To define value To present an overview of current methods
for measuring performance To discuss quantitative and qualitative
evaluations methodologies To identify examples of successful library
valuation projects To introduce potential methods for
measuring value from the non-profit sector
Objectives...
How do we define value?
© Christine Rooney-Browne (2009).
“...defining value in the context of libraries is complex, individual
stakeholders are unique, performance measurement is
essentially spatial, and operating in an environment that is neither causal nor predictive creates
complications” (Cram, 1999, p. 1).
Value is not fixed...
Personal, educational, professional circumstances
Seven Ages of Library Use (Bohme and Spiller, 1999):
Times of crisis: War, Recession, Natural
Disasters (New Orleans)
“Libraries can be valued in many different ways, from hard dollars to intangibles like
community goodwill and historical significance”
(Elliott, 2005).
Quantitative Methodologies
Audits...
CIPFA Public Library Statistic Actuals Report Library managers complete a
report about expenditure and income, staff levels, service points, stock levels, issues, enquiries, visits, inter-library loans
Library & Information Statistics Unit (LISU) summarises key statistics from this report in their Libraries, Archives, Museums, Publishing, Online Statistics Table (LAMPOST)
(LISU, 2010)
Pros & Cons...
“...quantity of use and quality of performance do not yet prove that users benefitted from their interaction with a
library. Measuring impact or outcome means going a step further and trying to assess the affect of services on users
and on society” (Poll and Boekhorst, 2007, p. 31).
How well the library is performing statistically
Statistics can be invaluable (budgets, staffing, stock)
Enables comparisons (league tables)
Disparity in funding Geographical location Overlook contribution
to lives of individuals, local community and economy
Return on Investment Studies (ROIs)
Cost-Benefit Analysis (CBA) Enables quantifiable values, such as cost or purchase price,
to be applied to variables that are difficult to measure; used to measure direct benefits
Consumer Surplus the value that consumers place on the consumption of a good
or service in excess of what they must pay to get it Cost of Time and Effort
measures time and effort expended by users Contingent Valuation (CV)
measures the value of use and non-use of non-priced goods and services (e.g. public libraries)
Willingness to Pay (WTP) Willingness to Accept (WTA)
Input-Output Models (IOMs)
Evaluating indirect benefits (e.g. impact that library has on the local economy)
Benefits measured using mathematical software models which looks at cause and effect relationships (e.g. Regional Input-Output Modelling System II and the Regional Economic Models, Inc (REMI)).
Have been used in the past to measure the value of beaches and parks.
“...if we can demonstrate our worth, with numbers, our budget numbers will
be all the more justifiable”
(Finch and Warner, 1998, p.158).
Combining CBAs and IOMs...
Suffolk Cooperative Library System (SCLS)
• Benefit-to-cost-ratio methods to measure direct benefits:
Total Value of library services ÷ Tax dollars supporting service:$509,415,038 ÷ $131,647,566 = $3.87 : 1 benefit/cost ratio
• For every $1 invested the library returned $3.87 in direct benefits
• Using input-output model called RIMS II to measure indirect benefits:
• Library generated $26 million in goods and services• Library enabled local earnings to increase by more than $50 million• Created more than 1,200 jobs for the local economy• Total multiplier effect of SCLS spending = $232 million.
(Imholz and Arns, 2007, p. 19).
Library Use Valuation Calculator...
(Chelmsford Public Library, 2010)
But what about social value...?
(Rooney-Browne, 2009b).
Qualitative Methodologies
Social Impact Audit...
Ethnography...
“Ethnography is the art and science of describing a group or culture......the ethnographer is both storyteller and
scientist; the closer the reader of an ethnography comes to understanding the native’s point of view, the better the
story and the better the science” (Fetterman, 1998, pp. 1-2).
Tracking Value...The Engaged Library: Chicago Stories of Community Building
•Prove that public libraries build social capital
•Identify & connect the library’s assets to the community
•Assess & strengthen the library’s connections with and use of community assets
•Produce a toolkit for other libraries to adopt to
•Mapping tools to perform an inventory services, identify areas for improvement and highlight library’s contribution to the community’s wider social, educational, cultural and economic goals.
Tracking Value...
(UKOC, 2008, p.4).
People behind the Numbers: Janet
(UKOK, 2010, p.38).
Profiling...
• Profiling has enabled UKOC to translate faceless data into outcomes that are “tangible, practicable and workable” (UKOC, 2010, p.5).
• Stakeholders and funders might be able to relate easier and have empathy for ‘Janet’ rather than a ‘C2DE female’.
Multiple Method Approaches...
Seattle Central Library Economics Benefits Assessment...
(SCL, 2005, p.10).
Methodology & Results... CBA methods to estimate economic impact Case studies, visitor and user surveys Interviews with local businesses, developers and
representatives from tourism Analysis of visitor and usage statistics.
Generated $16 million in net new economic activity in its first year of operation
Projections next twenty years: $80m (5yrs), $155m (10 years) or $310m (20 yrs)
improves desirability of downtown area & Seattle as tourist destination
An icon Findings used as an advocacy tool
Social Return on Investment...the future?
“...whether the capital provided is generating meaningful, real returns--returns for the manager, the investor and society as a whole” (Emerson, 2004).
Roberts Enterprise Development Fund (REDF) venture philanthropists working with non profits
to create job opportunities for disenfranchised Created SROI Model (2000):
Measure success...achieving goals...inform decisions...convince others of our impact
(Javitis, 2008, p.1).
Measuring SROI...
Compares net benefits of a project to investment (financial & time) Social Return Ratio (SRR): by combining net social benefits with
cash flow of the business then dividing by the total value of the philanthropic investment
SROI Rate: by carrying out an Internal Rate of Return (IRR) derived from total socio-economic value and total costs (Emerson and Cabaj, 2000, p. 11)
The New Economics Foundation, London Business School & Small Business Service produced a SROI Primer: Helps organisations identify and measure social outputs, outcomes
and impacts Provides advice to help organisations use SROI Model to translate
impacts into values (NEF, 2004).
Conclusion...
Perfect methodology does not exist but there are many possibilities
No general consensus as to the ideal model for measuring value
Competition for funding is fierce Cuts are inevitable Find the methodology that best fits your project and your
objectives “In today’s climate of accountability, a better understanding of the value of
public libraries is becoming essential to preserving and encouraging public and
private investment” (Imholz and Arns, 2007,
p.12).
CONTACT ME:
E-MAIL: [email protected]
BLOG: www.libraryofdigress.co.uk
Twitter: www.twitter.com/librarydigress
Voices for the Library: http://www.voicesforthelibrary.org.uk/wordpress/
THANK YOU