Sarah Chesney – ESC & Caroline Marcangelo

JISC DEPTA JULY 2011 1

Sarah Chesney – ESC & Caroline Marcangelo

Digitally-enhanced Patchwork Text Assessment :

a JISC LTI project

JISC DEPTA JULY 20112

• Clarify terminology - PTA

• Harnessing technologies to enhance learning through assessment

• Using work/practice for learning in our Pg Cert HE & CPD programmes

• the JISC-funded DePTA project

This presentation:

3

Individual students carry out regular short writing tasks known as “patches” throughout the ‘learning module’Each patch shared is with a small peer group for discussion and formative feedback Tutor might feedback on a sample of these ‘patches’ Typically the patches embody different genres of writing, and different elements of a broader topic – number variesConcluding and summative activity is to produce a reflective, synoptic, ‘stitching together’ of the patches

Original work undertaken by Winter et al in early 2000s

Clarifying terminology:Patchwork Text Assessment

JISC DEPTA JULY 2011

4

Continuous workflow in moduleOpportunity to try different forms of presenting workCummulative assessment enabling rapid formative feedbackOpportunity to see peer responses to the task – aid to self-assessment and understanding of learning outcomesIntegration of learning through reflectionFostering self-management of learning through meta-cognitive self-reflection on the learning journey

What advantages are there to PTA?

JISC DEPTA JULY 2011

JISC DEPTA JULY 2011 5

Technology to enhance learning

• The PTA process can be enhanced – even ‘enabled’ – through technology that has developed significantly since the initial work undertaken by Winter et al (2003)

• Facilitates more opportunity for the style and genre of patches• Can include patches that are non-text• Enables easier sharing and feedback with peers and tutors – and records different versions

JISC DEPTA JULY 2011 6

• UoC PgC for academic staff new to HE

• e-portfolios* as a personal learning system

• PTA to enable recognition of the range of workplace activities where learning occurs

• Peer- collaboration and formative feedback

• Modelling and developing good practice

• CPD scheme uses e-portfolios for UKPSF recognition process

* 3 year JISC-funded project – ‘Flourish’ www.flourishproject.org

Background to project

DEPTA JULY 2011 7

Patchwork Text Assessment

Predominantly viewed as a positive method of assessment – flexible, reflective, covered a broad range of interests, logistically possible, demonstrated metacognition

A very small number viewed doing small pieces of work more difficult than one straight essay

Acknowledgement that frequent sharing and feedback encouraged deeper self-assessment and critical thinking Chesney, S. & Marcangelo, M. 2010 'There was a lot of learning going on' Using an e-portfolio to support learning activities in a professional course for new HE lecturers Computers & Education 54 (2010), pp. 701-708

DEPTA JULY 2011 8

Direction of travel …..

Later iteration uses “patchwork assessment” – encouraging the use of non-text patches and reflexive commentary

Participants using audio, video, verbal reflective commentary, and conceptual mapping software

Using e-portfolios with staff continues to ‘normalise’ and there is evidence that they cascade their use with students

JISC DEPTA JULY 2011 9

Aims of DePTA …

This one-year project was funded through the JISC Learning and Teaching Innovation Grant (5) programme with the aim of evaluating the use and effectiveness of digitally supported patchwork text assessments in traditional subject disciplines

DePTA project webpage: http://dpta.wordpress.com/

DEPTA JULY 2011 /May'11 10

JISC DePTA project

5 participating universities in different disciplinesPsychology (Wolverhampton,

Bedfordshire)Law (Law)Photography (Cumbria)Physics (Liverpool)Archaeology (Liverpool)

Final year modules giving the students an opportunity to produce artefacts that showcase graduate capabilities and employability skills using ‘regular’ technology for that HEI/discipline

11

Evaluation focused on a range of areas including:

~ Practicability in terms of institutional technological resources

~ Capacity to enhance and evidence student learning at a high-stakes level

~ Sustainability in relation to staff workload

Project evaluation frame:

JISC DEPTA JULY 2011

12

Project evaluation:contexts

JISC DEPTA JULY 2011

13

• generally existing technologies were used often chosen on pragmatic grounds and that students and staff were familiar with them.

• The timeframe of a single year presented challenges for participants to make modifications to working practice that required university approval e.g. assessment regulations and module protocols. In some cases this had an impact upon the extent of student participation, as the amended assessment was offered as an optional activity.

Project evaluation:

JISC DEPTA JULY 2011

14

• Peer to peer sharing: – sharing work between students has been part of some but not all

partner projects. Where peer feedback was encouraged it still presented considerable challenges due to lack of familiarity with the collaborative nature of the tasks, particularly in the online environment.

– When in a more familiar setting such as face-2-face seminar peer discussion including challenge and feedback was not regarded as problematic.

– there appeared to be a discipline-related culture related to valuing individual achievement rather than co-operative development.

– Not all allowed online comments; some retained face-2-face meetings for such feedback/discussion, and one plans to move in this direction next year.

• Formative tutor feedback: Unsurprisingly students appreciated receiving tutor feedback

Project evaluation: Feedback:

JISC DEPTA JULY 2011

15

• Where students were geographically distributed with limited opportunities to meet (e.g. on work placement), the technologies were seen as valuable in maintaining connection/momentum.

• Whilst most of the patches were text based, other approaches including video and concept mapping were also employed

Project evaluation:

JISC DEPTA JULY 2011

16

Whilst one of the project aims anticipated the significance of subject benchmark and employability statements, there was evidence that this was not always realised in practice.

In most cases the emphasis was upon the discipline rather than generic institutional graduate attribute statements

Project evaluation:

JISC DEPTA JULY 2011

17

• the majority of tutors see considerable potential in extending/expanding the DePTA approach for a variety of reasons including being able engage and motivate learners for the full length of a module, developing a range of discipline specific skills

Project evaluation:

JISC DEPTA JULY 2011

18

Students are more confident in their learning through requirement to express it regularly and consider others’ development to the level of commenting on it. The technology provides a constant record so they can view there development over a period, where their progress is evident, thus further improving confidence. (Liverpool Physics case study)

Project evaluation:

JISC DEPTA JULY 2011

19

Results:

To be discussed