OSTRICH Project - le.ac.uk · PDF fileOSTRICH Project External Evaluation Report Page 3 of 20 1 Summary Peter Chatterton has been commissioned by the Beyond Distance Research Alliance

OSTRICH Project University of Leicester External Evaluation Report

Date: 7 October 2011

Ref: OSTRICH - External Evaluation Report V2.doc

Author: Peter Chatterton

Daedalus e-World www.daedalus-e-world.com

[email protected]

OSTRICH Project External Evaluation Report

Page 2 of 20

Contents

1 SUMMARY ............................................................................................................... 3

2 BACKGROUND .......................................................................................................... 4

2.1 INTRODUCTION .......................................................................................................................................... 4 2.2 OSTRICH PROJECT AIMS AND OBJECTIVES ...................................................................................................... 4 2.3 EVALUATION OBJECTIVES ............................................................................................................................. 5

3 EVALUATION METHODOLOGY ................................................................................... 6

3.1 OVERVIEW ................................................................................................................................................ 6 3.2 INTERVIEWS .............................................................................................................................................. 6 3.3 FOCUS GROUP ........................................................................................................................................... 6 3.4 EVALUATION FRAMEWORK ........................................................................................................................... 7

4 EVALUATION ............................................................................................................ 8

APPENDIX 1. EVALUATION FINDINGS .......................................................................... 9


Page 3 of 20

1 Summary Peter Chatterton has been commissioned by the Beyond Distance Research Alliance (BDRA) at the University of Leicester to conduct an external evaluation of the OSTRICH project (Open Educational Resources Sustainability through Teaching and Research Innovation: Cascading across HEIs). The primary aim of the OSTRICH project is to transfer and cascade, in usable formats, the key outcomes of Leicester’s institutional OER pilot project (OTTER, http://www.le.ac.uk/otter) to two partner institutions, the universities of Bath and Derby. The objectives of the external evaluation are to determine (a) the overall impact and success of the project, (b) viewpoints of the value of the project deliverables and outcomes and (c) identify key findings and lessons learnt to feed forward to the HE Academy, JISC’s Open Educational Resources Programme (Phase 2) Evaluation Team. In view of the limited budget available for the external evaluation, the approach adopted included desk research, telephone/Skype interviews with key members of the project team, facilitation of a face-to-face focus group at the University of Leicester and review of the internal evaluations at the Universities of Bath and Derby. The project has met (& in areas exceeded) its objectives and the project team must be congratulated for achieving this in a short time-scale project, where there were significant changes in project staff early on in the project. The detailed evaluation findings are presented in Appendix 1 and recommendations have been suggested to support sustaining and embedding of the project outcomes.

http://www.le.ac.uk/otter


Page 4 of 20

2 Background

2.1 Introduction

Peter Chatterton has been commissioned by the Beyond Distance Research Alliance (BDRA) at the University of Leicester to conduct an external evaluation of the OSTRICH project (Open Educational Resources Sustainability through Teaching and Research Innovation: Cascading across HEIs). The project runs from the 1 September 2010 until the 31 August 2011 and is part of the HEFCE-funded Open Educational Resources (OER) programme (phase 2), managed by the Higher Education Academy and JISC. The project is led by the BDRA with the Universities of Bath and Derby as partners. Further information about the project is available on the project web-site – www.le.ac.uk/ostrich and in the project plan1.

2.2 OSTRICH project aims and objectives

The primary aim of the OSTRICH project is to transfer and cascade, in usable formats, the key outcomes of Leicester’s institutional OER pilot project (OTTER, http://www.le.ac.uk/otter) to two partner institutions, the universities of Bath and Derby. OSTRICH will enable both partners to contextualise key OER lessons learnt, test and transfer OER models, leading to a sustainable approach to OER development and release. OSTRICH will:

a) Enable the partners to implement sustainable OER development processes by transferring the lessons learned and deliverables from OTTER and adapting them as appropriate to their own institutional approaches and cultures;

b) Brief stakeholders at partner institutions on a range of key issues associated with OER development, licensing and release to inform choices and policy;

c) Enable the application and refinement of OTTER’s CORRE quality and evaluation framework for OERs (http://bit.ly/adVruA) by the two cascade partners

d) Release a minimum of 100 credits’ worth of materials in the chosen priority areas at each institution within the 12 months of the project, deposited into the project repository and JorumOpen;

e) Provide a cascade model with associated guidance to enable partners to sustain the production and release of OERs beyond the duration of OSTRICH;

f) Offer practical and highly reusable guidance on designing for openness;

g) Facilitate the elaboration of draft policy documents for consideration by partner institutions;

h) Evaluate the interventions;

i) Generate opportunities for the dissemination of the OERs and associated findings within and beyond the partnership;

j) Promote further collaboration opportunities on OER issues across the sector.

1 OSTRICH project plan: http://www2.le.ac.uk/departments/beyond-distance-research-

alliance/projects/ostrich/media/OSTRICH-Project-Plan-Submitted.pdf

http://www.le.ac.uk/ostrich

http://www.le.ac.uk/otter

http://bit.ly/adVruA

http://www2.le.ac.uk/departments/beyond-distance-research-alliance/projects/ostrich/media/OSTRICH-Project-Plan-Submitted.pdf

http://www2.le.ac.uk/departments/beyond-distance-research-alliance/projects/ostrich/media/OSTRICH-Project-Plan-Submitted.pdf


Page 5 of 20

2.3 Evaluation objectives

The objectives of the external evaluation are to:

Determine the overall impact and success of the project and the extent to which the project’s aims and objectives have been met.

Determine viewpoints of the value of the deliverables and outcomes from the project from key stakeholder and the extent to which they can be adopted and/or further developed by other institutions.

Identify key findings, lessons learnt, outcomes, outputs to feed forward to the HE Academy, JISC’s Open Educational Resources Programme (Phase 2) Evaluation Team.

… with the following three areas to be assessed and reported on:

What has the project done/build/achieved, to what quality and how efficiently?

What has been learned or confirmed through development activities?

How has the learning been acted on or fed back in?


Page 6 of 20

3 Evaluation methodology

3.1 Overview

In view of the limited budget available for the external evaluation, the approach adopted was as follows:

Desk research.

Telephone/Skype interviews with key members of the project team.

Facilitation of a face-to-face focus group at the University of Leicester.

Review of the internal evaluations at the Universities of Bath and Derby.

Preparation of draft report for comments.

Preparation of final report.

3.2 Interviews

Details of the telephone/Skype interviews were as follows:

Date Meeting/interview Institution Project Role

7 June 2011 Jaideep Mukherjee University of Leicester Project Manager

7 June 2011 Phillip Gagen University of Derby OER Coordinator

23 June 2011 Vic Jenkins

Kyriaki Anagnostopoulou

University of Bath Learning Technologist

OER Coordinator

26 June 2011 Alejandro Armellini University of Leicester Principal Investigator

27 July 2011 Gabi Witthaus University of Leicester OER Knowledge Transfer Fellow

3.3 Focus group

The focus group was run on the 3 August at the BDRA (University of Leicester) with the following attending:

University of Leicester: Alejandro Armellini, Gabi Witthaus

University of Bath: Vic Jenkins, Kyriaki Anagnostopoulou

University of Derby: Phill Gagen The objectives of the focus group were to:

Present and discuss the initial evaluation findings and to further refine these as a group.

Brainstorm ideas for sustaining and embedding the project in each institution and in the sector as a whole.


Page 7 of 20

3.4 Evaluation framework

The following evaluation framework was used in the interviews and focus group:

Theme Topic

Overall evaluation Meeting of project objectives and outputs

Key challenges encountered and how they were overcome

Project management

Stakeholder engagement and buy-in to the project

Internal – within the project team members

Within each institution e.g. to

Practitioners

Support services

Middle management

Senior management

The sector

Cascade process Quality

Barriers/issues

Critical success factors

CORRE model

Contextualising/adapting to institutional approaches and cultures.

Sustainability/embedding of the model.

OERs developed Quality

Repository

Measuring impact

Take up by institutions

Take up be sector

How communicated?

Development processes

Sustainability

Embedding of CORRE 2.0 Existing embedding

Barriers to embedding

Opportunities for embedding

Business models Cost/benefit analysis

Resources Quality

Future Future work identified


Page 8 of 20

4 Evaluation The evaluation findings are presented in Appendix 1 and these are based on the following data collection techniques:

Desk research.

Telephone/Skype interviews with key members of the project team.

Facilitation of a face-to-face focus group at the University of Leicester.

Review of the internal evaluations at the Universities of Bath and Derby. Both Bath and Derby have produced internal evaluation reports and these have been important to this report as the budget allowed for the external evaluation has not permitted the external evaluator to interview different stakeholders in each partner institution. The Bath internal evaluation provides some very useful feedback of different stakeholder perspectives, however the Derby evaluation was more limited in its evaluation of different stakeholder perspectives (though Derby’s manager of technology-enhanced learning commented “OSTRICH has been enormously valuable to Derby and is starting to inform future thinking with regard to content creation, sharing and reuse”). It is recommended that Derby allocates time to distilling the lessons learnt so that it can capitalize on its considerable learning from this project and build on its successes with implementing the CORRE 2.0 model.


Page 9 of 20

Appendix 1. Evaluation findings

Theme Topic Feedback Lessons learnt or confirmed, effective practice & recommendations

Overall evaluation

Meeting of project objectives and outputs

Lessons learnt, key issues & deliverables from the Leicester OTTER project have been effectively transferred to the project partners, enabling them to implement OER development processes based around the CORRE quality and evaluation framework for OERs - which has been adapted to their own contexts, “readiness”, cultures, ways of working and needs and encompasses both new content creation and conversion of existing materials.

The project has over-delivered on the target of providing 100 credit’s worth of materials, deposited in the project repository (at the University of Bath) and JorumOpen, though the internal evaluations of stakeholder perspectives of the materials could be improved upon, particularly in relation to evaluating quality, fitness-for-purpose and usability/re-usability. It is recommended that this evaluation is sustained beyond the project life-time in order to provide valuable data on the take-up of the OERs.

Lessons learnt from the cascading process have enabled the project to refine the cascade model and resources to encompass different institutional contexts, cultures, needs, ways of working and “readiness” for use by other institutions. The revised model includes a scaffold to support institutional adoption and implementation of OERs and the key elements of this revised model could potentially be applied to other cascade/benefits realisation projects (outside of the OER sphere).

The cascading process has helped accelerate the project partners in respect of developing “OER maturity” which is reflected in active support from middle management, establishment of a “vocabulary” and dialogue and raising of the profile of the partner project teams.

Practical and reusable resources, including policy documents and guidance materials, have been developed in respect of designing for openness that can be adopted and adapted by other institutions.

The project has over-delivered in respect of sector dissemination of the OERs and associated findings and has identified a number of further collaboration opportunities on OER developments across the sector.

Though there is significant evidence of grass-roots adoption of open practices and active middle-management support, it is not clear the degree to which senior management will drive forward the sustaining and embedding of the project outcomes in order to scale up progress and overcome the significant challenges to widespread adoption that the project has identified. The project itself has identified a very effective method for embedding by integrating open design into programme/module design/review processes (& support services), however this will not in itself overcome all the challenges that programme teams will face (e.g. insufficient time allocation for implementing open design approaches). A recommendation has therefore been given that each partner institution


Page 10 of 20


sustains the already effective dialogue with their senior management team with a view to producing a sustainability and embedding plan.

Key challenges encountered and how they were overcome

Significant staff changes & academic input occurred during the project, which presented challenges for team working, continuity and communications, particularly as the project duration is only one year. However, the project managed to get successfully back on track after some initial delay and attract new enthusiastic project team members and academic practitioners.

The project highlights the key cultural/ways of working barriers to truly embedding open design approaches and the need to continuously improve stakeholder communications and engagement at all levels.

Project plans should “scenario plan” to take account of potential changes in staff.

Project sponsors/funders should recognise the difficulties that some institutions encounter with staff changes on short-duration projects, particularly those institutions which do not have a large resource of potential project team members to draw on.

Project management The project has been well managed and the Leicester team provided an excellent level of support to the partner institutions.

Stakeholder engagement and buy-in to the project

Internal – within the project team members.

The project team communications and ways of working were slow to gel which can present problems in short-term projects, though this was largely due to significant staff changes, particularly in the early stages of the project. The project team did, however, recover well from the initial set-backs and developed a strong project team culture.

The project team took time to develop shared terminology and understanding of terms, concepts and nature of OERs (including with practitioners).

The project team experience highlights the need for projects to place strong emphasis on stakeholder communications and engagement including within the project team itself. It is important that project teams take time to better understand the perspectives of ALL stakeholders e.g. senior managers - and communicate and engage with them in ways to make an effective case for open design approaches that will meet their needs. Guidance on developing communications and stakeholder engagement strategies and plans is provided in the JISC Guide to Sustaining and Embedding Innovations

2.

Attention to informal as well as formal means of communications is important, particularly where it is

2 JISC Guide to Sustaining and Embedding Innovations: https://sustainembed.pbworks.com

https://sustainembed.pbworks.com/


Page 11 of 20


necessary to quickly develop positive working relationships between team members.

Within each institution e.g. to

Practitioners

Support services

Middle management

Senior management

The project team has overall successfully engaged with practitioners and support services at Derby and Bath and created significant dialogue, engagement and a “shared vocabulary” in respect of open approaches.

In each partner institution, middle management has been highly supportive and engaged with the project.

It is very encouraging to see senior managers on the project steering group which demonstrates support at that level. However, there is minimal evidence so far that senior managers are (or will be) actively involved in sustaining and embedding the outcomes of the project, though it is likely that the senior managers will require further evidence of e.g. affordances, demand and resourcing implications for open approaches, before instituting sustainability and embedding measures.

Whilst some senior managers see the marketing benefits of OERs, this perhaps detracts from the true holistic affordances of open approaches.

Middle-management support and drive is essential to support grass-roots activities in open design approaches, including OER development.

The project has highlighted tensions in relation to IPR ownership by academics and this was reflected by different models of IPR ownership at Bath and Derby.

Practitioners typically have misconceptions about copyright and ownership issues in respect of use of third-party materials, therefore clear guidance materials, as produced by the project, are essential.

Where academic permission is required for use of materials as OERs, this can lead to complex scenarios where materials have been developed over long periods with multiple authors.

Open design is a complex area with many pre-conceived prejudices and beliefs therefore stakeholder engagement and communications in this area needs to become more sophisticated and address opportunities, needs and concerns of ALL stakeholders. Project teams should take steps to better understand the needs & concerns of stakeholders and to allocate time and resources to address these. To aid this process, it is useful to develop a stakeholder communications and engagement plan – further details are available in the JISC Guide to Sustaining and Embedding Innovations

3. A key

element of this, for instance, would be to ensure senior

3 JISC Guide to Sustaining and Embedding Innovations: http://sustainembed.pbworks.com

http://sustainembed.pbworks.com/


Page 12 of 20


managers/leaders are well-informed about open approaches – e.g. they will want to understand about business cases, risks (& managing them), the degree of “maturity” of open approaches, QA, work-load planning, resourcing and support, evidence of demand, staff CPD requirements, change management – as well as the affordances of open design approaches.

Motivations of practitioners for engaging with OERs vary significantly – ranging from personal learning, quality enhancement, & show-casing to meeting institutional priorities and wishing to adopt an open ethos.

The cascade process has enhanced Leicester’s "internal cascading" processes through refining the OTTER outputs, sustaining the internal dialogue and incorporating open design into the Leicester Carpe Diem model

4. The partner

institutions have learned from this and intend to adopt a similar approach. However, there will still be challenges to address with this approach e.g. providing sufficient time for academics to work on open design and providing them with the appropriate skills for e.g. DIY approaches (incorporated within CORRE 2.0).

The sector The project has “over-delivered” on its sector dissemination activities.

The project has developed outputs that will further enhance the JISC OER infoKit

5 - not just the updated CORRE model (CORRE 2.0),

but the incorporation of a scaffold for supporting institutional adoption and implementation of OERs.

It is recommended that the CORRE 2.0 framework (including the scaffold to support institutional adoption and implementation of OERs) becomes a key feature of the JISC OER infoKit. It would also help to develop a narrated presentation about using the framework to include on the infoKit site (this can be achieved by using TechSmith

4 University of Leicester BDRA Carpe Diem process www.le.ac.uk/carpediem

5 JISC Open Educational Resources infoKit: https://openeducationalresources.pbworks.com

http://www.le.ac.uk/carpediem

https://openeducationalresources.pbworks.com/


Page 13 of 20


Camtasia).

Cascade process Quality Overall, the cascade process has been highly successful and the project team evolved the process to deal with issues as they arose.

The cascade approach has been refined as the project progressed and is now available for other institutions to adopt and adapt.

The project has developed useful guidance for other institutions – including a scaffold to support institutional adoption and implementation of OERs.

The cascade process has helped the partner institutions in the following ways:

o Raising key issues to address (e.g. policies, IPR ownership).

o Providing momentum and impetus.

o Providing the opportunity to develop an evidence base.

o Helping to identify deficiencies in internal expertise and resources (e.g. lack of a copyright/IPR officer at Bath).

The project team debated the optimum working model between the cascade “host” and “adopter” institutions. The actual model adopted was more one of “project manager” which reflected the need to deliver the project outcomes, within a relatively short time-scale. However, future cascade projects might like to consider a working model more based on the “host” institution acting as Critical Friend to the adopter institutions.

The refined cascade process could potentially be adopted and adapted for use with other cascade/benefits realisation projects outside of the OER sphere.

Barriers/issues The cascade process highlighted:

Deficiencies in terms of institutional support and expertise.

Various issues surrounding IPR ownership and academic attitudes towards ownership (e.g. IP with a history of multiple authors).

Cascading complex concepts to other institutions presents challenges that are not immediately apparent in terms of institutional, cultural and staff "readiness" issues.

Critical success factors Core to a successful cascade process is all parties developing a mutual understanding of their respective cultures, drivers, motivations & working practices – this links back to the


Page 14 of 20


comments in the “stakeholder communications and engagement” section.

CORRE model

Contextualising/adapting to institutional approaches and cultures.

The CORRE model has been refined (CORRE 2.0) to incorporate the different institutional contexts, cultures, approaches, support mechanisms and ways of working at Derby and Bath and incorporate an institutional adoption model, covering both conversion of existing materials and development of new materials.

Open design can be a catalyst for good practice in programme design in general (many elements of open design reflect effective learning design in general).

Sustainability/embedding of the model.

Though engagement and dialogue with senior management has taken place throughout the project, it is not clear how each partner institution will institutionally sustain and embed the CORRE 2.0 model and gain senior management buy-in to scaling up and overcoming the significant challenges to widespread adoption that the UK OER projects have identified.

The partner institutions have learnt from Leicester that one approach to sustainability and embedding of the model is to embed the approach into a support process and services for programme/module design/review (see Leicester’s Carpe Diem process

6). Bath, in particular, has recognised that

effective open design represents effective learning design in general and aims to embed open approaches in the work of the e-learning team.

However, institutions need to recognise that there will still be challenges for programme teams even with such a process. For example, they will require sufficient time to allocate to open design and will need to acquire the appropriate skills for e.g. DIY approaches (CORRE 2.0 should provide support for this - Bath went a long way towards to enabling these skills amongst their academics who were developing OERs from scratch). In addition, support services will need to plan how they can adequately support programme teams throughout their institutions with limited resources themselves.

6 University of Leicester BDRA Carpe Diem process: http://www2.le.ac.uk/departments/beyond-distance-research-alliance/carpe-diem-folder

http://www2.le.ac.uk/departments/beyond-distance-research-alliance/carpe-diem-folder


Page 15 of 20


OERs developed

Quality The external evaluator did not have the resource to extensively evaluate the quality of the OERs developed – this was more the responsibility of the partner institutions and there are validation/evaluation processes built-in to the CORRE process e.g. at Bath, this included peer reviews and student input/review at the design stage – all of which are now part of the updated DORRE process. However, there was not a great deal of focus on evaluating the quality of the OERs in the internal evaluations, nor was there a framework specified for evaluating quality. Anecdotally, it was reported that the “better” materials were those where time and effort had been expended in using different, but appropriate media to develop or convert them.

In hindsight, it may have been preferable for Derby to produce a blend of OERs in text-form and media-rich form (e.g. audio-enabled PowerPoint files) in order to allow evaluation of the pros and cons of each form.

There are QA processes built-in to the model e.g. in relation to quality, IPR and copyright, but more detailed QA or guidance could be given in terms of what constitutes "good" materials and appropriate formats.

OERs could possibly be made more usable by removing logos and context-specific info etc. – but this can make them bland and un-interesting.

The partners should consider specifying a framework for evaluating the quality of OER materials.

Repository Drupal was a good choice for a repository, seeing that it is an open-source high quality content management system that has the degree of flexibility to develop customised features and to align its design/functionality with JorumOpen.

It is understood that JISC placed a requirement for projects to create a repository. For sustainability purposes though, it might make more sense to utilise JorumOpen, unless it is planned to further develop Drupal to provide additional repository functionality e.g. to make access to the OERs more usable. Maintenance and support of two repositories is not realistic

Further development of the Drupal repository could create applications that address some of the practical, usability, efficiency and cost-effectiveness issues associated with practitioners developing, using and re-using OERs e.g. to support efficient work-flow, version control and sign-off processes linked to the CORRE 2.0 model and maybe to develop practical features that make it simple for academics to feedback on OERs to help others judge their potential usefulness.

The project has highlighted the lack of common metadata


Page 16 of 20


unless there are clear benefits. standards for repositories, which makes searching for OERs more difficult for end users.

Measuring impact The internal evaluations have not been able to fully address impact measurement as the project focus has mainly been on the cascade process, the OER quality and evaluation framework and development of OERs.

The partners should aim to undertake impact measurement beyond the project life-time which could provide useful data to support the case for wider institutional adoption.

Take up by institutions There are difficulties in terms of tracking the take-up and use/re-use of OERs though such data would prove valuable in terms of assessing the impact of open approaches. The partners could perhaps place a greater priority in this area as the OERs begin to be used.

There is feedback (from Bath stakeholders) that smaller, re-usable OER chunks might be more attractive to practitioners compared with credit-bearing units.

Each partner institution should put in place mechanisms to evaluate the use and re-use of the OERs developed. It is recognised that such activities will be beyond the time and scope of the OSTRICH project, however, this data will be invaluable for helping to shape future OER policies.

Institutions and the sector as a whole should research needs and demands for OERs in terms of type, subject, size, characteristics etc. and should aim to determine demands from multi-stakeholder groups e.g. within sectors (including HE, FE, sector skills councils, professional bodies) and subject areas.

How communicated? The OERs developed have mostly been demand-driven, so there has not been so much of a priority in respect of communications.

Embedding of CORRE 2.0

Existing embedding Leicester is the most "mature" institution in terms of embedding through e.g. integration of open design with its pre-existing Carpe Diem curriculum design process/service, whereas Derby/Bath are still in early stages of introducing similar approaches.

Embedding open approaches in institutional strategies e.g. LT&A strategies and allocation of resources requires SMT buy-in and this can present obstacles if they have not "bought-in" to open concepts.

Guidance to embedding innovations is provided in the JISC


Page 17 of 20


"Good Practice Guide to Embedding Innovations7" which

provides an embedding framework around culture change, staff development, enhancing strategies and policies, embedding in processes and support services and development of communities of practice.

Barriers to embedding Key barriers to embedding include:

Lack of time and resources for academics to develop quality OERs – whilst it could be argued that open design is just one element of overall good design, there are a range of activities specifically required for open design e.g. checking for re-usability, IPR conformance etc.

Cultural attitudes and & ways of working barriers.

Skills required to create quality OERs using mixed media and DIY approaches are high (though CORRE 2.0 addresses the skills issue).

IPR and ownership concerns – at staff and institutional levels.

Understanding costs –v- benefits.

Senior management being fully “informed” in order to aid effective decision-making.

There is a need to better understand demand and characteristics for use and re-use of OERs including small OER chunks-v- whole modules, formats, topics, discoverability, culture/ways of working issues, quality assessment etc.

Project teams need to make a better case to senior managers for adopting open approaches and to address the needs and concerns of senior managers.

Opportunities for embedding

Identified opportunities for embedding include:

Integration with programme/module design/review processes and support services.

Integration into LT&A strategies and plans.

Integration into work-based learning strategies and plans.

Partner institutions should consider developing sustainability & embedding plans.

7 JISC Guide to Sustaining and Embedding Innovations: https://sustainembed.pbworks.com

https://sustainembed.pbworks.com/


Page 18 of 20


Aligning with QA/QE processes.

Incorporating open approaches as part of change management initiatives.

Incorporation into staff CPD/digital literacies development programmes.

Collaborative initiatives – particularly those which are demand-led e.g. sector initiatives.

Developing communities of practice e.g. at institutional levels, sector levels and subject levels.

Resources to provide time/incentives for practitioners.

Business models Cost/benefit analysis Whilst the project plan does not include the production of a business model or guidance on costs/resources required for OER development, it would be useful for projects such as this to provide guidance to other institutions in this area, based on their lessons learnt. Such data will be invaluable to practitioners and decision-makers and could help to shape institutional and sector policies in this area.

The sector as a whole needs to gain a better understanding of costs and benefits associated with open approaches and to identify sustainable, positive cost/benefit models, approaches and tools.

Resources Quality The quality of guidance documents, resources and templates (e.g. draft OER policies and guidelines in relation to IPR, copyright, lecture capture and YouTube policies) made available on the project blogs and repository is high and these have drawn on the experience both within each institution and external to the institutions - and there was positive feedback from practitioners at Bath.

Some of the materials developed would need adapting by other institutions to align with their own institutional contexts, cultures

The project should ensure that the feedback from the internal evaluations is embedded, where appropriate, into the guidelines.

In both institutions, the production of policy documents is acting a as a catalyst for institutional dialogue.


Page 19 of 20


and working practices.

Future Future work identified The project has identified a range of further development work opportunities which could help to address some of the issues raised in this evaluation:

Of particular importance is to gain further evidence about demand, use and re-use of OERs as well as the barriers to institutional and staff adoption. The section above on cost/benefit analysis would also be an area to gain further evidence in.

The project team have also recommended a project to blend the lessons learned/outcomes with those from the JISC Curriculum Design/Delivery projects – which could be a very pragmatic step to take given Leicester’s success with integrating open approaches with its Carpe Diem process. Furthermore the Higher Education Academy and JISC may wish to consider aligning the project outputs with its Change and Enhancement Academy models. The Leadership Foundation may also be interested in learning more about the CORRE 2.0 framework and the scaffold to support institutional adoption.

The project has suggested further development of the Drupal repository to create applications that address some of the practical, usability, efficiency and cost-effectiveness issues associated with practitioners developing, using and re-using OERs e.g. to support efficient work-flow, version control and sign-off processes linked to the CORRE 2.0 model and maybe to develop practical features that make it simple for academics to feedback on OERs to help others judge their potential usefulness. This would be a useful project to undertake, provided a sustainable model is specified, which would ensure that the developed features make their way to the wider sector repositories.

At the focus group meeting, an idea was discussed to adapt the scaffold framework into a maturity toolkit with a defined process to use the toolkit in a similar way to the toolkits that have been used for e-learning benchmarking

8 and the work-based learning

maturity toolkit9. The project team have identified that such an idea already exists in the form of the OPAL

10 project which focuses

on supporting open educational practice and which has a process for evaluation of an institution’s maturity in open educational practice. It is recommended that the project team should further investigate how their work could be aligned with and support the OPAL activities.

It is recommended that each partner institution continues the dialogue with its senior management towards developing a sustainability and embedding plan in respect of open approaches.

8 National Higher Education Academy/JISC e-learning benchmarking programme: http://elearning.heacademy.ac.uk/wiki/index.php/Main_Page;

http://elearning.heacademy.ac.uk/weblogs/benchmarking/ 9 Work-based learning maturity toolkit: http://wbltoolkit.pbworks.com

10 OPAL project: http://www.oer-quality.org/

http://elearning.heacademy.ac.uk/wiki/index.php/Main_Page

http://elearning.heacademy.ac.uk/weblogs/benchmarking/

http://wbltoolkit.pbworks.com/

http://www.oer-quality.org/


Page 20 of 20


The project team should particularly focus on OER developments where there could be multi-stakeholder demand in “win-win” scenarios e.g. in sectors or subject areas involving organisations such as sector skills councils, professional bodies, HE and FE and focused on supporting new work-based learning approaches.

A sector SIG (Special Interest Group) could help to drive forward the OER concept in the sector in the same way that ELESIG has for the student learning experience. It could also help make the linkages with curriculum design/delivery transformation and work-based learning.

OSTRICH Project - le.ac.uk · PDF fileOSTRICH Project External Evaluation Report Page 3 of 20 1 Summary Peter Chatterton has been commissioned by the Beyond Distance Research Alliance

Documents