Top Banner
Measuring Quality in the Cultural Sector Standardised Metrics + Big Data = New Data Culture? ‘What Does Data Have To Do With Me?’ British Museum, June 5 th John Knell, Culture Counts
28

John Knell, Culture Counts: Measuring Quality In The Culture Sector

Aug 15, 2015

Download

Education

BethBate
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Measuring Quality in the Cultural Sector Standardised Metrics + Big Data = New Data Culture?

‘What Does Data Have To Do With Me?’ British Museum, June 5th

John Knell, Culture Counts

Page 2: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 3: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Our Mission

To help the cultural sector measure the quality of their work on their own terms,

and to communicate the value they create through big data

Page 4: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Anybody else feel lost?

What We Found:

• No Common Language• Struggling to define ‘intrinsic’ elements of value• Very little info sharing or benchmarking• No rigorous assessment of quality of outcomes post event• Manual, old-fashioned research• No Data Culture – ‘the plural of anecdote is not data’

Page 5: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 6: John Knell, Culture Counts: Measuring Quality In The Culture Sector

How do we measure?

• Triangulation of evaluation by self, peers and public

• Online data capture through the Culture Counts system via a web app or via post event surveys (urls posted via email)

• Current quality of experience metrics have 9 dimensions evaluated by all 3 groups

• 5 extra dimensions evaluated by self and peers

Page 7: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Quality dimensions (self, peer, public)Dimension Metric Statement

Rigour ‘It was well thought through and put together’

Distinctiveness ‘It was different from things I’ve experienced before’

Captivation ‘It was absorbing and held my attention’

Relevance ‘It has something to say about the world in which we live’

Meaning ‘It meant something to me personally’

Challenge ‘It was thought provoking’

Enthusiasm ‘I would come to something like this again’

Presentation ‘It was well produced and presented’

Local impact ‘It is important that it’s happening here’

Page 8: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Quality dimensions (self & peer only)

Dimension Metric Statement

Concept ‘It was an interesting idea / programme’

Originality ‘It was ground-breaking’

Risk ‘The artists / curators really challenged themselves with this work’

Excellence (global)

‘It is amongst the best of its type in the world’

Excellence (national)

‘It is amongst the best of its type in the UK’

Page 9: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Simple interface for respondents

Page 10: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 11: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 12: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 13: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 14: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Average self, peer and public scores for ‘In My Father’s Words’ (awarded after the event)

Page 15: John Knell, Culture Counts: Measuring Quality In The Culture Sector

What types of insights are generated?

• The triangulation element is vital to the insight and reflection process

• All responses are interesting in their own right, but cultural organisations tell us that it is the triangulation that generates new insight for them on the extent to which they are achieving their creative objectives

• Being able to anticipate how audience members and peers will respond to a work is a reflection of the quality / maturity of creative and cultural leadership in an organisation

Page 16: John Knell, Culture Counts: Measuring Quality In The Culture Sector

What types of insights are generated?

• Integrating quality metrics with audience size and profile and financial data allows for a sophisticated assessment of– How successfully organisations balance creative,

commercial and audience objectives– How different kinds of cultural experience create

different kinds of value• Developing prompts to help organisations

understand their evaluation results and use them to reflect on their creative practice

Page 17: John Knell, Culture Counts: Measuring Quality In The Culture Sector

The data is only a starting point for a richer

conversation about your creative practice and

audience relationships

Page 18: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 19: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 20: John Knell, Culture Counts: Measuring Quality In The Culture Sector

Elongate and Enhance the Cultural Experience

• There is an untapped demand for audiences to give feedback and enter into a richer dialogue about their cultural experiences

• Culture Counts and this type of approach offers a potentially powerful way of elongating and enhancing the cultural experience

• Integrate with social media to increase the sense of shared experience

Page 21: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 22: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 23: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 24: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 25: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 26: John Knell, Culture Counts: Measuring Quality In The Culture Sector
Page 27: John Knell, Culture Counts: Measuring Quality In The Culture Sector