Institutional Repository Usage Statistics IRUS-UK: the story so far and what’s next 17 July 2013 Balviar Notay, Jisc Ross Macintyre, Mimas Paul Needham,

Institutional Repository Usage Statistics

IRUS-UK: the story so far and what’s next17 July 2013

Balviar Notay, JiscRoss Macintyre, Mimas

Paul Needham, Cranfield UniversityAngela Conyers, Evidence Base, BCU

irus.mimas.ac.uk

The Jisc view (1)

Jisc began work in repository usage statistics area in 2009 – with requirements gathering and feasibility testing (PIRUS)

Now we are building and in process of delivering a national shared service for usage stats.

Recognised a growing need to measure usage - as the repository infrastructure grew. (now 200 UK repositories approx)

Parallel work with OpenAIRE. Also library usage statistics– under the Jisc Activity Data programme).

irus.mimas.ac.uk

The Jisc view (2)

Usage is seen as important factor in capturing impact. Usage part of the growing metrics infrastructure - which includes citation and Altmetrics.

Usage statistics supports – management reporting and wider business intelligence gathering. National aggregation of usage statistics for allows for functions such as benchmarking

IRUS-UK gives JISC, other infrastructure providers and funders, a nation-wide picture of the overall use of UK repositories (demonstrate their value and impact).

IRUS-UK could also potentially act as an intermediary between UK repositories and other agencies. E.g. The EU funded OpenAIRE project are interested in an API to the service to monitor usage of UK FP7 funded research.

irus.mimas.ac.uk

The Jisc view (3)

Jisc is planning the sustainability of IRUS-UK in the context of a number of co-ordinated repository shared services.

–The UK RepNet initiative (scoping, co-ordinating and developing technical infrastructure) coming to an end in July 2013.

–From August 2013 (for 2 years) Jisc will be building on the work of UK RepNet to support sustainable repository shared services.

–New name and branding will be developed for infrastructure.

– Jisc coordination and management of services (transition to service)

–Developing the appropriate business models

–Build scalable infrastructure - interoperate with RIM and RDM

irus.mimas.ac.uk

Jisc services (1)

irus.mimas.ac.uk

Jisc services (2)

irus.mimas.ac.uk

Jisc services (3)

irus.mimas.ac.uk

Jisc programmes Academy/JISC Open Educational

Resources Programme Phase 3

Assessment & Feedback Programme

Business Intelligence

Content Programme 2011-2013

Developing Digital Literacies

Digital Infrastructure: Directions

Digital Infrastructure: Information and Library Infrastructure Programme

Digitisation and Content

Directions: Strategic Directions

e-Learning Programme

Enhancing DMPonline Projects

Greening ICT Programme

Information and Library Infrastructure: Emerging

Opportunities

Information and Library Infrastructure: Resource Discovery

Managing Research Data Programme 2011-13

Relationship Management

Research Data Management Infrastructure Projects

Research Data Management Planning Projects

Research Management: Repositories and Curation Shared Infrastructure

Strategic Content Alliance

Transformations Programme

UMF Shared Services and the Cloud Programme

World War One Commemoration

irus.mimas.ac.uk

IRUS-UK: the story so far...

irus.mimas.ac.uk

IRUS-UK

Funded by Jisc as part of UK RepositoryNet+

Project Team Members: Mimas Cranfield University EvidenceBase, BCU

IRUS-UK: ‘Institutional Repository Usage Statistics – UK’ Enable UK IRs to share/expose usage statistics based on a

global standard – COUNTER

Emerged from work done in ‘PIRUS2’ Project Publisher and Institution Repository Usage Statistics project http://www.cranfieldlibrary.cranfield.ac.uk/pirus2/

irus.mimas.ac.uk

IRUS-UK: Current status

Production-strength service infrastructure

Tracker code: DSpace & Eprints , Functional specification for Fedora.

Collecting raw usage data from UK IRs for all item types within (33) repositories Downloads not record views

Processing those raw data into COUNTER-compliant statistics Making available to the originating repositories for their own use

Providing an aggregated picture of the use of items in UK repositories

Published ‘Item Types’ Report classifying types of items downloaded

COUNTER PIRUS Code of Practice published

Evaluation, dissemination and community engagement

irus.mimas.ac.uk

The IRUS-UK portal

Live demonstration

irus.mimas.ac.uk

IRUS-UK: Repository Totals

irus.mimas.ac.uk

IRUS-UK: Item Types Totals

irus.mimas.ac.uk

IRUS-UK: Item Type <->IR: Item Type

irus.mimas.ac.uk

IRUS-UK: DOI Summary Stats

irus.mimas.ac.uk

IRUS-UK: Title/Author Search

irus.mimas.ac.uk

IRUS-UK: Ingest Summary Stats

irus.mimas.ac.uk

IRUS-UK: IR1 Report LSE Jan-Feb 2013

irus.mimas.ac.uk

IRUS-UK: CAR1 Report Jan-Feb 2013

irus.mimas.ac.uk

IRUS-UK: what’s next...

irus.mimas.ac.uk

IRUS-UK: the old ingest process

The existing ingest process has been described in detail in previous webinars and presentations - http://www.irus.mimas.ac.uk/news/

The key point is to apply the COUNTER Code of Practice to filter out robots and double clicks

However the COUNTER Robot Exclusion list is specified only as a minimum requirement – more can be done

We’ve added additional filters to Remove more user agents Apply a simple threshold for ‘overactive’ IP addresses

Substantially better, but we’re still not satisfied - we need a more sophisticated filtering system!

irus.mimas.ac.uk

IRUS-UK: the new ingest process (1)

We commissioned Information Power to: Analyse raw data we’ve collected since July 2012 Test the feasibility of devising a set of algorithms that would

‘dynamically’ identify and filter out unusual usage/robot activity

A report on that work is available from http://www.irus.mimas.ac.uk/news/

Key findings from the work are Suspicious behaviour can’t necessarily be judged on the basis

of one day’s usage records or a month’s. At certain levels of activity machine/non-genuine usage is

practically indistinguishable from genuine human activity.

 Going forward, we will test out and experiment with the new dynamic filtering

irus.mimas.ac.uk

IRUS-UK: the new ingest process (2)

As a service, we have to be pragmatic so we will go for a ‘best result for least effort’ approach.

In each calendar month we will process logs daily eliminate as much as we can with a quick, minimalist

approach insert statistics into a ‘Provisional Daily Stats’ table

At the end of each month we will reprocess those provisional stats Apply more comprehensive, sophisticated filtering load the restated stats into the permanent daily stats table empty the provisional table ready for the next month

We can’t ever get to perfection in open web environment but, by the time we’re done, we will be producing ‘the best wrong stats in town’

irus.mimas.ac.uk

IRUS-UK: Tracker patches and add-ons

Available for DSpace and Eprints

Eprints Add-on for 3.2.x and 3.3.x Not feasible to back port to earlier versions

DSpace Patches for 1.8.x and 3.x But there are a lot of older DSpace instances out there We’ll commission 1.6.x and 1.7.x versions

irus.mimas.ac.uk

IRUS-UK: Tracker for other IR software

We will have to look at other repository software platforms on a case by case basis

Fedora Every Fedora repository is a one-off. - but some general

guidelines are available in Appendix O in the PIRUS2 Final Report, http://www.projectcounter.org/News/Pirus2_oct2011.pdf

University of Hull

PURE Portals We’ve opened discussions with Atira and we’re hopeful that

IRUS-UK Tracker functionality will be available for PURE portals …

Other platforms We would welcome dialogue with interested vendors &

developers - Contact us!

irus.mimas.ac.uk

IRUS-UK: Exposing statistics

We will be expanding The Portal Adding new views and reports Delving deeper into individual repository statistics Improving DOI based views and reporting Incorporating more metadata – Funder and Grant number

SUSHI Server New SUSHI Service to meet COUNTER Release 4 requirements

API/Web Service Expanding and enhancing the existing version Usage statistics for incorporation into Repositories Determining further requirements

irus.mimas.ac.uk

IRUS-UK: Community engagement

Growing number of repositories sending data to IRUS-UK

Currently 33 participants: Eprints: Bath Spa, Bournemouth, City, UEA, Glasgow

School of Art, Goldsmiths. Greenwich, Huddersfield, Kent, Kingston, Lancaster, LSE, Middlesex, NERC, Northampton, Northumbria, Open, Reading, Salford, Sussex, UEL, UWE, Warwick

DSpace: Aberdeen, Abertay, Aberystwyth, Cranfield, Edinburgh, Exeter, Heriot-Watt Imperial, RGU, St Andrews

Others in the pipeline

irus.mimas.ac.uk

IRUS-UK new users’ survey

Survey sent to new users 2-3 weeks after joining – 19 replies so far

Aim to get initial impressions from users:

Best features of IRUS-UK

Benefits of using IRUS-UK

Ways in which IRUS-UK might be used

Challenges to using IRUS-UK.

irus.mimas.ac.uk

Best feature of IRUS-UK

Reliable, authoritative statistics

Ability to benchmark against others

Demonstrating value

Speed and ease of set-up

irus.mimas.ac.uk

Institutional benefits

Reliable figures to demonstrate usage and for benchmarking

Improving repository by increasing deposit and demonstrating usage to researchers

Enhancing work flow and saving time

irus.mimas.ac.uk

Improving statistical reporting

Respondents described their current methods of collecting and using repository statistics and any challenges they faced.

16 out of 19 (84%) said they expected IRUS-UK to improve their statistical reporting (3 said don’t know/too early to tell)

irus.mimas.ac.uk

Open data

During this initial period all data in IRUS-UK are open to all UK HEIs via Shibboleth/Open Access

Asked if they were happy with data being open:

16 out of 19 (84%) said yes, 3 not sure

Asked if their institution would be happy:

11 out of 19 (58%) said yes, 8 not sure

irus.mimas.ac.uk

Some views from the IRUS-UK community

I do really look forward to using the download figures from your portal. They are reliable and very useful.

 It is a helpful service so far - and we are interested to follow future developments

Very useful service, would like to see further long term development.

Impressed so far, and looking forward to investigating further!

Great work and hope that other IRs join up asap!

irus.mimas.ac.uk

IRUS-UK: how to join

If you are a UK repository: Contact us at irus.mimas.ac.uk to register your interest Answer a few questions on the type of repository you

have and the version you are running Get advice from us on what work will be involved

depending on your repository type and version Implement any changes advised and then see your

usage data instantly in IRUS-UK with no more work from you

“The set up was quick and painless, which is always a delight!”

“Consistent collection of statistics without me having to do it!”

irus.mimas.ac.uk

Contacts and information

If you are a UK repository wishing to participate in IRUS-UK, please contact [email protected]

Project web site:http://www.irus.mimas.ac.uk/

Thank you!