A centre of expertise in digital information management UKOLN is supported by: Pitfalls and blind alleys: perspectives on.

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

UKOLN is supported by:

Pitfalls and blind alleys: perspectives on quality measures

Ann Chapman

Interoperability Focus

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Quality measurement

• Accuracy – is the data correct?• Consistency – is the data

consistent?• Functional – does the data support

users?• Currency – is data available when

needed

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Measuring tape or string?

• Surveys– Sampling frame, sample sizes– What is being surveyed?

• Questionnaires– Leading questions, response scales– User perceptions

• Toolkits– Limitations

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Survey samples

• Sample sizes– Minimum 100 (BNB 1440 over a year)– Split samples, bigger overall sample

• Sampling frame– Public and academic libraries– Staff and students– Subject areas

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Quality = type of record

• BNB Currency Survey 1980 – 2005

• Hit split into record types– Full record from BL– Formerly CIP– CIP

• Assume records found are correct

• Sometimes spot error

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Errors in data

• LDLSCP Audit 1994 –

• 100 records BL, 9 each LD library

• Checklist of error types– 008/260 don’t match, spelling, indicators,

added entries, consistent data in related records

• Limited by not seeing items

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Edits = errors

• BLCMP (Talis) study 1992-1993– % of records edited– Analysis of 1310 paired ‘before/after’

records – but no verification of need to edit

– Member libraries submitted a small set of records edited and gave their reasons for editing

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Quality = user understanding

• Cambridge University Study 1992• 300 interviews

– Random (91 UG, 209 PG/staff)– Users not grouped by area of study

• Main problems found– Misunderstanding data (jargon)– Difficulty with OPAC labels– Misinterpreting abbreviations

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Decide on parameters

• Cat-Assess Tool (1999-2001)– Record cataloguing policies first– Random sample ‘book/record’ check

• BNB Currency Survey– BNB Exclusions policy

• LDLSCP Audit– Assessment limited by not seeing item– To assess consistency, some related records

must be in sample

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

Re-using data

• BNB Currency Survey– Acquisitions trends over time– Retention of stock– Comparison with other record suppliers

(1996-1999)– Monitor effect of BL policies

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

So remember …

1. What do you want to measure?

2. Do you already collect data you can use?

3. If not, choose an appropriate method of data collection

4. Write up methodology and results, especially for repeated studies

A centre of expertise in digital information management

www.ukoln.ac.uk www.bath.ac.uk

UKOLN

Visit the UKOLN website at:

http://www.ukoln.ac.uk/

Bibliographic Management

http://www.ukoln.ac.uk/bib-man/

Collection Description Focus

http://www.ukoln.ac.uk/cd-focus/