Improving University Science Teaching and Learning

Improving University ScienceTeaching and Learning

Pedagogical Projects 2019

Department of Science EducationUniversity of Copenhagen

Published by the Department of Science Education, University of Copenhagen,Denmark

E-version and search engine at http://www.ind.ku.dk/publikationer/up_projekter/Printed at www.lulu.comThe anthology can be bought through the marketplace at http://www.lulu.com

© by the authors 2020

Improving University Science Teaching and Learning – Pedagogical Projects 2019,vol. 14. ISSN: 1904-2000

http://www.ind.ku.dk/publikationer/up_projekter/

Contents

PrefaceFrederik Voetmann Christiansen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Part I Writing for learning

1 Refleksionsopgaver som pædagogisk værktøjMona Chor Bjørn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Skrivning som læringsstrategiIben Gjødsbøl og Sara Green . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3 Enabling PhD students to write their own manuscriptsDuc Ninh Nguyen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Part II Improving the lecture experience

4 Does a simple exercise of student-generated take homemessage activate and thereby improve the attention level andlearning outcome of the students during an academic lecture?Marie Pedersen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5 Student activation in theoretical physics lecturesJan Olaf Haerter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

iv Contents

6 Split Information in PowerPoint Assisted Scholastic SettingsDavid Jason Koskinen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

7 Kan simple ændringer i underviserens undervisning ogfremtoning ændre de studerendes engagement?David Møbjerg Kristensen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Part III Flipped Classroom and blended learning

8 Effect of using online active learning platforms on students’engagement and the teaching outcomesAbbas Jafari . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

9 Students’ perceived learning outcome and attitudes towardsnon-traditional lecturingPieter Jan Kerstens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

10 Transforming an online course to blended and studentcentred learningXi Jiao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

11 Enhancing student research projects with new technologiesDarach Watson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

12 Can the flipped classroom enhance student learning?Melanie Ganz-Benjaminsen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Part IV Course (re)design and constructive alignment

13 Course alignment: Application and revision of a method toalign a courseGabriela Alandia Robles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

14 Introducing elements of active learning in a small courseLaura Mancinska . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

15 Implementing a student-centered exercise format in a PhDcourse in applied biostatisticsSigne Marie Jensen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Contents v

Part V Strengthening peer feedback

16 Peer-feedback i holdundervisning og hvordan det påvirkerde studerendes opfattelse af egenlæring på kurset GlobaleGeosystemerMikkel Fruergaard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

17 Udvikling af et peer-2-peer feedback forløbSandra Gentin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

18 MSc students’ experience with peer feedback during awritten taskAndy Gordon Howe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Part VI Conceptual understanding and metacognition

19 Improving Mathematics Understanding in the PharmacyUndergraduate ProgramKathryn Browning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

20 Hvem har Ansvaret? - Om studerendes og underviseresforståelse af begrebet ‘vejledning’ på kurset AppliedEnvironmental and Natural Ressource EconomicsAyoe Hoff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

Del VII Improving practical work and exercise classes

21 Helping less to strengthen active student-centred learning inexercise classesMette Frahm Olsen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

22 The learning of veterinary students during practicalultrasound exercisesAnna Müller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

Preface

Frederik Voetmann Christiansen

Department of Science EducationUniversity of Copenhagen

This is the 14th volume of the Department of Science Education’s seriesof anthologies based on participant’s development projects made in rela-tion to the Teaching and Learning in Higher Education programme (“Uni-veristetspædgogikum” or UP) at the University of Copenhagen. The seriescan be downloaded from the webpage of the Department of Science Ed-ucation under publications, and can also be bought in paper version fromprint-on-demand. It is possible to search through previous projects in theanthology by using the local search engine found here: http://www.ind.ku.dk/publikationer/up_projekter/.

This volume consists mainly of project reports written by participantsfrom the January and August 2018 courses. Each participant is required toconduct a small-scale development project as part of UP. The projects showhow individual teachers have identified specific problems relating directlyto their teaching practices and includes their reflections on how to developtheir practice and the quality of teaching more generally.

Topics covered in the projects include course design and redesign, con-structive alignment, research based teaching, feedback to name a few. Thisanthology is organized into parts based on some of the recurring overarch-ing topics to give the reader a structured overview.

We would like to thank all the authors for their valuable contributionsto the promotion of Scholarship of Teaching and Learning at the Universityof Copenhagen.



Part I

Writing for learning

1

Refleksionsopgaver som pædagogisk værktøj

Mona Chor Bjørn

Institut for Miljø og PlantevidenskabKøbenhavns Universitet

I de seneste 5 år har botanikerne på Sektion for Organisme Biologi, Insti-tut for Plante og Miljøvidenskab, Københavns Universitet (KU) oplevet enstadigt stigende efterspørgsel på efteruddannelse i plantebestemmelse frade grønne erhverv (f.eks. landskabsarkitekter, hortonomer, byplanlæggere).Ofte med begrundelsen: ”vi har brug for at få genopfrisket vores botanik”eller ”jeg vil gerne vide mere om planter”. Denne øgede interesse for at op-gradere botaniske færdigheder, fra erhvervets side, har blandt undervisernemedført en interesse for hvordan vi kan øge motivationen blandt landskabs-arkitektur (LA) studerende på KU for at specialisere sig inden for botanik,plante- og vegetationsøkologi i deres studietid.

Motivation for læring (at opnå viden, færdigheder og kompetencer in-den for et emne) påvirkes ifølge Illeris (2017, s.45) af en samspilsprocesmellem tre overordene faktorer 1) den kognitive dimension (fagligt ind-hold), 2) den psykodynamiske dimension (den studerendes egen drivkraft/motivation) og 3) den sociale dimension (samspil med andre, herunder bådemedstuderende og underviseren).

For at få indsigt i de studerendes motivation for at specialisere sig indenfor kernefag (botanik, plante- og vegetationsøkologi) vil jeg anvende reflek-sionsopgaver som undersøgelses metode. Refleksionsopgaver er et interes-sant pædagogisk redskab fordi formatet muliggør 1) aktiv deltagelse fra alletyper af studerende uanset baggrund, køn, alder og læringsstil, 2) frigør denenkelte fra den sociale kontekst som vedkommende måtte befinde sig i samt3) muliggør at underviseren kan invitere de studerende til dialog i takt medat relationen opbygges.

4 Mona Chor Bjørn

Problemformulering

I denne udviklingsopgave undersøges: hvordan kan anvendelsen af reflek-sionsopgaver bidrage til, at øge den studerendes motivation for at opnå vi-den, færdigheder og kompetencer i kernefagområder set fra de studerendesperspektiv?

Opgavens relevans

Ved at belyse dette problem kan andre undervisere måske finde inspirationtil, hvordan man som underviser kan 1) undersøge de studerendes forvent-ninger til et undervisningsforløb, 2) italesætte den faglige og samfunds-mæssige relevans af kurser 3) sikre at formålet med undervisnings- ellerkursusforløbet er klart for den enkelte studerende såvel som hele holdetsamt 4) motivere de studerende til at fordybe og forfølge kernekompeten-cer inden for et fagområdet.

Introduktion

Udviklingsopgaven er gennemført på Bacheloruddannelsen (Figur 1.1) iLandskabsarkitektur på Københavns Universitet i 2018. Uddannelsen be-står af et naturvidenskabeligt fundament med kernefagene botanik, plante-og vegetationsøkologi som indgår i de obligatoriske kurser: ”Naturgrund-laget 2A (NG2A), Naturgrundlaget 2B samt ”By- og LandskabsplanternesBotanik” (Botanik). Kurserne har normalt ca. 50-60 tilmeldte og inkludereren hel årgang af LA studerende. Yderligere specialisering indenfor botanik,plante- og vegetationsøkologi kræver et aktivt tilvalg fra den enkelte stude-rende i resten af studietiden på bachelor- såvel som kandidatuddannelsen ilandskabsarkitektur.

1 Refleksionsopgaver som pædagogisk værktøj 5

inkluderer en hel årgang af LA studerende. Yderligere specialisering indenfor botanik, plante- og

vegetationsøkologi kræver et aktivt tilvalg fra den enkelte studerende i resten af studietiden på bachelor- såvel

som kandidatuddannelsen i landskabsarkitektur.

Figur 1. Obligatoriske kurser på Bachelor Uddannelsen i Landskabsarkitektur, Københavns Universitet

(https://studier.ku.dk/bachelor/landskabsarkitektur/undervisning-og-opbygning/)

Resumé af den udvalgte litteratur Til at undersøge problemet, vælger jeg at inddrage litteratur der belyser refleksion som pædagogisk redskab i

undervisningen samt didaktiske perspektiver på organisering (individuel eller gruppebaseret refleksion) af

undervisningsforløb med fokus på refleksion. Yderligere anvendes litteratur der belyser hvilke faktorer der

spiller ind på den studerendes motivation for læring. Desuden inddrages litteratur der belyser kvalitative

undersøgelsesmetoder, herunder interviewteknik og -metode.

Rienecker et al. (2015) behandler didaktiske og pædagogiske aspekter af

universitetsundervisning fra kursusplanlægning til undervisningsmiljø og individets læringsproces. I denne

opgave anvendes bogen som inspiration til organisering af undervisning samt perspektiver på

hvordan man skaber et godt undervisningsmiljø for de studerende som udgangspunkt for

planlægning af interventionen. Illeris (2017) sætter fokus på læring som en tilegnelsesproces der påvirkes af

tre dimensioner 1) fagligt indhold (færdigheder, viden, forståelse), 2) drivkraft (motivation, følelser, vilje) og

3) samfund (samspil med omgivelser, medstuderende, underviser). Herudover sætter Illeris (2017) fokus på

læringstyper, barrierer mod læring, samt individets forudsætninger, herunder læringsstil (f.eks. aktiv

eksperimenterende) eller intelligens (f.eks. logisk-matematisk). Rasmussen & Østergaard (2009) giver

grundlæggende introduktion til kvalitative undersøgelsesmetoder samt diskuterer fordele og ulemper ved

disse. Desuden anvendes Kvale & Brinkmann (2009) som inspiration til opbygning af interviewguide og

interviewteknik af grupper. Wahlgen et al. (2002) stiller spørgsmål ved refleksions- og læringsprocesser i

Figur 1.1. Obligatoriske kurser på Bachelor Uddannelsen i Landskabsarkitektur,Københavns Universitet.

Resumé af den udvalgte litteratur

Til at undersøge problemet, vælger jeg at inddrage litteratur der belyserrefleksion som pædagogisk redskab i undervisningen samt didaktiske per-spektiver på organisering (individuel eller gruppebaseret refleksion) af un-dervisningsforløb med fokus på refleksion. Yderligere anvendes litteraturder belyser hvilke faktorer der spiller ind på den studerendes motivation forlæring. Desuden inddrages litteratur der belyser kvalitative undersøgelses-metoder, herunder interviewteknik og -metode.

Rienecker et al. (2015) behandler didaktiske og pædagogiske aspek-ter af universitetsundervisning fra kursusplanlægning til undervisningsmil-

6 Mona Chor Bjørn

jø og individets læringsproces. I denne opgave anvendes bogen som in-spiration til organisering af undervisning samt perspektiver på hvordan manskaber et godt undervisningsmiljø for de studerende som udgangspunkt forplanlægning af interventionen. Illeris (2017) sætter fokus på læring som entilegnelsesproces der påvirkes af tre dimensioner 1) fagligt indhold (fær-digheder, viden, forståelse), 2) drivkraft (motivation, følelser, vilje) og 3)samfund (samspil med omgivelser, medstuderende, underviser). Herudoversætter Illeris (2017) fokus på læringstyper, barrierer mod læring, samt in-dividets forudsætninger, herunder læringsstil (f.eks. aktiv eksperimenteren-de) eller intelligens (f.eks. logisk-matematisk). Rasmussen og Østergaard(2009) giver grundlæggende introduktion til kvalitative undersøgelsesme-toder samt diskuterer fordele og ulemper ved disse. Desuden anvendes Kva-le og Brinkmann (2009) som inspiration til opbygning af interviewguideog interviewteknik af grupper. Wahlgen et al. (2002) stiller spørgsmål vedrefleksions- og læringsprocesser i erhvervsmæssig kontekst, men diskute-rer i Del II – Refleksion og læringsbegrebet refleksion ud fra 6 forskelligeteoretikere herunder Peter Javis (Wahlgren et al., 2002, s. 143-155).

Beskrivelse af interventionen og dokumentationen

I perioden februar til oktober 2018 gennemførte jeg to interventioner (ano-nyme refleksionsopgaver), før og under det samlede obligatoriske kursus-forløb i botanik-, plante- og vegetationsøkologi for LA studerende på Kø-benhavns Universitet. Herefter blev der gennemført et fokusgruppe inter-view for at undersøge potentialet i at bruge refleksionsopgaver som pæda-gogisk redskab til at øge den studerendes motivation for at opnå kompeten-cer i kernefag set fra de studerendes perspektiv.

Den første intervention blev gennemført i kurset NG2A og bestod af enskriftlig, anonym refleksionsopgave forud for den officielle introduktion tilkurset. De studerende fik 5-10 min. til at besvare spørgsmålet ”X”, hvoref-ter de blev bedt om at aflevere besvarelserne anonymt i en fælles kuvert.

Den anden intervention blev gennemført i kurset Botanik hvilket af-slutter det samlede obligatoriske kursusforløb i botanik-, plante- og vege-tationsøkologi. Denne intervention bestod en gruppebaseret refleksions op-gave. De studerende blev inddelt i grupper af 4-6 personer og bedt om atudarbejde et fælles mindmap med udgangspunkt i samme spørgsmål somblev anvendt i den anonyme refleksionsopgave tidligere på året. Mindmapser en kreativ, logisk tænkemåde der kan anvendes til at strukturere ideer,


skabe overblik, finde sammenhænge mellem emner og som kan anvendestil individuelle, gruppe- eller plenumopgaver (EMU). Efterfølgende blevden anonyme gruppe-baserede refleksionsopgave afleveret til underviseren.

Til at undersøge potentialet i at bruge anonym skriftlig refleksion sompædagogisk redskab, set fra de studerendes perspektiv blev der på baggrundaf de to interventioner gennemført et fokusgruppeinterview (ultimo oktober2018). Respondenterne inkluderede 2 kvindelige og 1 mandlig studerende,der alle havde deltaget i begge typer refleksionsopgaver i 2018. Interviewdeltagerne ønsker alle at være anonyme. Fokusgruppeinterviewet blev gen-nemført ved hjælp af en interviewguide (Bilag A) suppleret med uddybendeog opklarende spørgsmål undervejs. I denne opgave er citater fra fokus-gruppe interview gengivet i uredigeret form. Fordelen ved interviewmeto-den er, at den giver indsigt i respondenternes motivation og er en velegnetmetode til at generere nye ideer, give vurderinger, eller til at få dybere for-ståelse for respondenternes forståelse af et fænomen, men ulemperne er atudsagn opnået via fokusgruppeinterview er tæt knyttet til de personer somdeltager (Rasmussen et al., 2009, s. 92-93).

Analyse og diskussion

Dette udviklingsprojekt skal ses i et relativt langt tidsperspektiv fra førsteintervention (anonym skriftlig refleksion) i februar 2018 til den anden inter-vention (grupperefleksion) oktober 2018. I den periode formodes de stude-rende at have gennemgået en faglig progression (jf. kursernes læringsmål)inden for de kernefag som ligger til grund for projektet. Begge interven-tioner var en succes i forhold til gennemførsel. Besvarelserne fra første in-tervention er personlige og giver indtryk af hvad den enkelte studerendeforventer (tror/drømmer om) mens grupperefleksionen, godt et halvt år se-nere, for langt det fleste gruppers vedkommende er faglig og konkret (BilagB).

Motivation for læring kan også kaldes drivkraft eller vilje til at tilegnesig ny viden. Ifølge Illeris (2017) påvirkes motivationen af mange faktorer,herunder den sociale og den kognitive dimension. Den studerendes inter-esse for stoffet kan stimuleres af underviseren ved f.eks. at sætte stoffet irelation til den studerendes verden og lave en tydelig kobling mellem teoriog praksis. Denne pointe bekræftes af interviewdeltagerne. Én siger: ”ge-nerelt på NG2 - at I var gode til at samle lidt op på det og sige det er dether vi har lært i dag, hvorfor er det relevant, altså ”hvorfor er en strandeng

8 Mona Chor Bjørn

vigtig” og det mere samfundsorienterede, bredere kontekst end hvorfor erden her rosen-familie vigtig for samfundet?” en anden studerende supple-rer ”man går aldrig fra en systematisk / botanik time med fornyet viden,man kan bruge til en pind. Jo, måske – ah ha, den plante har en virkeligsjovt formet maske og nu skal du høre hvorfor”. . . . Det er ligesom bezzer-vizzer viden”. Der kan således opstå en barriere for læring, hvis stoffet ikkegiver mening for den enkelte, eller i værste fald for et større antal kursister.En studerende siger ”den kursus beskrivelse man tager med sig, er den frade ældre årgange, snakken går om hvad det er for nogle kurser man skalhave og hvad man kan forvente sig”. Som underviser har man flere hand-lemuligheder. F.eks. at finde ud af hvad de studerende interesserer sig for,synliggøre relevansen af kurset, eller undersøge hvad de studerende tænkerom indholdet (Reinecker et al., 2015, s.123-126). Refleksion i form af tæn-keskrivning, summe opgaver (enkeltvis eller i grupper) (Ulriksen, 2014),kan være metoder til at undersøge de studerendes tanker og forestillingerom stoffet, og samtidig en måde at evaluere om undervisningen virker efterhensigten. En studerende siger ”jeg føler, at jeg er ret godt inde i min faglig-hed, så for mig vil det ikke være noget problem at skulle lave en refleksionselv, (. . . ) men jeg tænker også at for andre gælder det sammen ikke (. . . ).For en underviser kan det være fint at have en individuel refleksion hvisman skal præge undervisningen. Hvis folk sidder og rent faktisk er i tvivlom hvad de får ud af kurset eller hvis halvdelen sidder og ikke kan få nogetpå papiret, så er det jo kritisk i forhold til undervisningen”.

Ifølge læringsteoretiker P. Javis kan læring opdeles i 9 processer her-under ”ræsonnement og læring” (Wahlgren et al., 2002). Refleksion kanunderinddeles i 3 former hvoraf den ene er reflekterende læring. Reflekte-rende læring kan være kontemplativ læring, hvor tænkningen skal udmundei en konklusion (Wahlgren et al., 2002, s. 149). At stoffet giver mening i enstørre sammenhæng kan være afgørende for den enkeltes læringsproces. Enstuderende kommenterer: ”vores uddannelse er naturvidenskabelig - vi bli-ver uddannet på Science, det er en svær kobling for vores fag, men hidtilhar det givet total god mening – lær om naturen før du kan arbejde medden som et værktøj. . . Hvis man kan få undervisningen til at give mening ien større sammen, så er den en sund erfaring at tage med sig videre, hvordet måske halter med vores botanik fag”. Sammenhængen mellem ræson-nement og refleksion bliver her tydelig, fordi den studerende italesætter atstoffet måske ikke er lige relevant for alle. En anden studerende siger: ”dethar slet ikke været et issue med de andre kurser vi har haft, det er megetdet her kursus (red. By og Landskabsplanternes Botanik), jeg tror der er


mange der stopper op nu og tænker hvorfor? ”. Det faglige indhold er idette tilfælde tilsyneladende en barriere for læring, hvilket ifølge interviewdeltagerene forklares ved følgende: ”jeg kan godt se hvad du mener . . . .fordi vores fag er så bredt, nogle vil højest se på et par træer og nogleder virkelig gerne nørde, også alt imellem”. To ting er interessante her 1)interviewdeltagerne bekræfter hinanden i at udsagnet er rigtigt, hvilket re-fererer tilbage til ulemperne ved at interviewe i grupper (Kvale et al., 2009,s. 170-171, Rasmussen et al., 2009, s. 93) samt 2) at de studerende sæt-ter fokus på kompleksiteten mellem den sociale og den kognitive dimen-sion i forhold til læring (Illeris, 2017). I den konkrete situation stiller fle-re studerende tilsyneladende kollektivt spørgsmålstegn ved meningen medsystematik-undervisningen, hvilket vil påvirke deres personlige drivkraftfor at tilegne sig de specifikke faglige færdigheder. En didaktisk handle-mulighed kan være løbende refleksion samt at organisere undervisningeni grupper. En fokusgruppeinterviewdeltager kommenterer: ”jeg kan megetgodt lide at få stillet spørgsmål, hvad forventer du af det? hvad kan vi brugedet til? Og man så bagefter samler op, eller løbende samler op. Nåh – nuforstår jeg bedre hvad I snakker om, nu er jeg lidt mere inde i jeres termi-nologi (. . . ). Det vil hjælpe til sidst, at man samler det op i grupper, det kansagtens være anonymt, men kan også sagtens være i grupper. I grupper fårman en lidt bredere forståelse af stoffet – nåh det var det du fik ud af det –det kan jeg godt se”. Den studerende opfatter således sine medstuderendesom en ressource, som kan udnyttes til at få nye perspektiver på det fagligeindhold i kurset.

Gruppearbejde i forbindelse med refleksion i grupper kan organiseressom kortvarige summe grupper eller længerevarende diskussionsfora hvorhovedformålet er, at de studerende skal ”tale med hinanden” for til slut atudforme et produkt (Reinecker et al., 2015 s.189). Interventionen (grup-perefleksion => mindmap) som ligger til grund for denne opgave kan be-skrives som en kortvarig summe opgave. Selv om forløbet var kortvarigtkan gruppedannelsen og sociale interaktioner blandt gruppens deltagerestadig have indflydelse på resultatet fordi de studerende vil påtage sig rollerfx ”den der noterer”, ”den der styrer diskussionen”, eller baseret på tidlige-re erfaringer med gruppearbejde; være mere eller mindre positive over forprocessen (Reinecker et al., 2015 s.192). Fordelen ved gruppearbejde er atde studerende kan optimere deres læring, få nye perspektiver eller minime-re misforståelser af stoffet gennem dialogen med medstuderende. En andenstuderende kommenterer: ”det betyder noget at det er i grupper (. . . ) detkan være at man sidder, som enkelt person svarer på noget - også ikke op-

10 Mona Chor Bjørn

dager før bagefter, når man så snakker med de andre, men gud jeg har joogså lært det og det og det. At man måske bare lige kan huske det man harlært inden for de sidste to uger, man kan huske noget man lige bed specieltmærke i, hvorimod i grupper, så er der nogen der kan huske noget og no-gen der kan huske noget andet. Nogle ting man lærer lægger man ikke heltmærke til at man lærer”.

At vi i de senere år oplever en stigende efterspørgsel på efteruddannelsei botanik (red. plantebestemmelse, botanik) er måske et udtryk for at reflek-sionsprocessen omkring ”meningen” med undervisningen i kernefagenebotanik, plante- og vegetationsøkologi på landskabsarkitekt-uddannelsenrækker langt ud over det normerede uddannelsesforløb. Motivationen forat tilegne sig de specifikke kompetencer som 1. års-studerende er såledesafhængig af den enkelte studerendes evne til at sætte indholdet i en fagligkontekst. Som underviser har man en lang række didaktiske handlemulig-heder for at organisere undervisningen således, at det faglige indhold sættesind i en større kontekst samt udnytte at langt størstedelen af de studerendeevaluerer kurset positivt jf. obligatorisk skriftlig evaluering (Bilag C).

Konklusion på erfaringerne fra projektet

Refleksionsopgaver kan anvendes til forventningsafstemning, faglig evalu-ering og perspektivering. På baggrund af interventionen som ligger til grundfor denne udviklingsopgave, den udvalgte litteratur samt empiri, udtrækkesfire væsentlige pointer a) hvis man som studerende er inde i stoffet er re-fleksionsopgaver unødvendige, b) refleksion der sætter det faglige indholdi et større perspektiv kan understøtte den studerendes motivation c) reflek-sionsopgaver i grupper kan aktivere mere viden og bredere perspektiv samtd) underviseren skal spørge sig selv: hvad er hensigten med refleksionsop-gaven, og hvem skal den gavne?

Perspektivering og eventuelle begrænsninger i projektet

Interventionen gennemført i denne udviklingsopgave har været et godt red-skab, for mig som underviser, til at få indsigt i de studerendes tanker, for-ståelse og holdning om kernefagsområder. Om interventionerne har gavnetde studerendes motivation er ikke dokumenteret, men det kvalitative inter-view afslørede at refleksionsopgaver kan være gode til at opsummere viden


samt sætte denne i relation andre kurser og faget. Som studerende kan manmåske først se sammenhængen mellem alle de brikker et uddannelsesforløbbyder på i slutningen af uddannelsen, hvorfor den løbende perspektivering(faglig og samfundsmæssige) er gavnlig for de studerende.

Den kvalitative undersøgelses metode kan forbedres i forhold til organi-sering og tidspunkt. Det lykkedes ikke at skaffe det anbefalede antal (6-10)frivillige studerende (jf. Rasmussen et al., 2009) til deltagelse i det kvali-tative fokusgruppeinterview, hvilket kan betyde at de studerendes udsagnikke er repræsentative for årgangen. Ulempen ved fokusgruppeinterviewetvar at deltagerene i løbet af interviewet endte med at bekræfte hinanden,eller forsøger at finde frem til en fælles holdning, fremfor at diskutere ogfremføre egne synspunkter. Til allersidst var det derfor nødvendigt at spør-ge ind til deltagernes personlige synspunkt hvilket 2 ud af de 3 deltagerekunne redegøre for. Dataindsamlingen kan derfor forbedres ved at få fle-re frivillige deltagere, eller alternativt suppleres med flere dybde gåendeindividuelle kvalitative interview. Undersøgelses metoden kunne udvidesved brug af online spørgeskemaer for at inddrage et større antal studerende.Fokusgruppeinterviewet blev gennemført umiddelbart efter en botanikfore-læsning, hvorfor de første 10 min. af interviewet blev brugt til en åben sam-tale blandt deltagerne for at belyse deres tanker om hvorvidt fagområdernebotanik, plante- og plantesamfundsøkologi er et kernefag i deres uddannel-se eller ej. Denne indledning vurderende jeg var afgørende for kvaliteten affokusgruppeinterviewet fordi de studerende var frustrerede over den dagsundervisningsforløb. En studerende kommenterede: ”kan man lave et ellerandet for at bruge det, fordi lige nu virker det utroligt uoverkommeligt. . .det er virkelige de færreste der kan magte lære det”. Kvaliteten af inter-viewet kan måske forbedres, hvis det afholdes efter kursusforløbet såledesat de studerende har gennemført hele kursus inden der spørges ind til pro-cessen.

Det kunne være interessant og relevant at sammenligne betydning afrefleksionsopgaver for obligatoriske kurser og valgfri kurser. Min formod-ning vil være, at refleksionsopgaver i valgfri kurser vil have størst effektsom løbende opsamling eller underviserens evaluering af læringsprocessenfordi de studerende må formodes, at have en del personlig drivkraft somindre ressource til at opnå viden, færdigheder og kompetencer i valgfag.

En anden ting som jeg gerne ville studere nærmere er sammenhængenmellem de studerendes læringsstil og betydningen af refleksionsopgaver.Ifølge Reinecker (2015) og Illeris (2017) findes der ikke en generel defini-tion på begrebet læringsstil, men som underviser skal man være opmærk-

12 Mona Chor Bjørn

som på at ikke alle studerende har en ens læringsproces. Undervisningenkan derfor med fordel tilrettelægges således, at der anvendes varierendeundervisningsmetoder. I den specifikke kontekst ift. kernefag på LA ud-dannelsen kunne det ligeledes være interessant, at undersøge hvilke styren-de intelligens typer de studerende har. Er de logisk-matematik tænkende,eller har de en styrende spatial intelligens og hvordan påvirker det deresmotivation til at tilegne sig færdigheder i naturvidenskabelige kernefags-områder. Dog er der blandt psykologer og læringsteoretikere enighed om aten endimensionel intelligens forståelse er alt for unuanceret (Illeris, 2017,s. 211-12).

Referencer

Illeris, K. (2017). Aktuel Læringsteori I Spændingsfeltet Mellem Piaget,Freud Og Marx. 3. udgave, 3. oplag, Frederiksberg C, Samfundslitteratur

Kvale, S., & Brinkmann, S. (2009). Interview: introduktion til et håndværk.Hans Reitzels Forlag.

Rasmussen, E. S., Østergaard, P., & Andersen, H. (2009). Samfundsviden-skabelige metoder-En introduktion.

Rienecker, L., Jørgensen, P. S., Dolin, J., & Ingerslev, G. H. (2015). Uni-versitetspædagogik. Samfundslitteratur.

Ulriksen, L. (2014). God undervisning på de videregående uddannelser.Frydenlund.

Wahlgren, B., Høyrup, S., Pedersen, K., & Rattleff, P. (2002). Refleksionog læring. Kompetenceudvikling i arbejdslivet, Samfundslitteratur.

EMU: https://www.emu.dk/modul/mindmap-som-et-undervisningsv\T1\ae rkt\T1\o j (15.11.2018, kl. 17.01)

Landskabsarkitekt uddannelses opbygning:https://studier.ku.dk/bachelor/landskabsarkitektur/undervisning-og-opbygning/(10.12.2018 kl. 08)

https://www.emu.dk/modul/mindmap-som-et-undervisningsv\T1\ae rkt\T1\o j

https://www.emu.dk/modul/mindmap-som-et-undervisningsv\T1\ae rkt\T1\o j


A Interviewguide

Bilag 1: Interviewguide

14 Mona Chor Bjørn

B Eksempel på besvarelse af refleksionsopgave

Figur 1.2. Øverst: Individuel besvarelse af refleksionsopgave fra projektets førsteintervention (februar 2018). Nederst: Eksempel på grupperefleksion fra projektetsanden intervention (oktober 2018).


C Kursusevaluering 2018Bilag 3: Kursus evaluering 2018

Uddrag af Kursus evaluering ”By og Landskabsplanternes Botanik” 2018

2

Skrivning som læringsstrategi

Iben Gjødsbøl og Sara Green

1Institut for Folkesundhedsvidenskab2Institut for Naturfagenes DidaktikKøbenhavns Universitet

1

SKRIVNING SOM LÆRINGSSTRATEGI

Iben Gjødsbøl, Institut for Folkesundhedsvidenskab, Københavns Universitet

Sara Green, Institut for Naturfagenes Didaktik, Københavns Universitet

Introduktion

I undervisningssammenhænge bruges skriftligt arbejde primært til at doku-mentere og evaluere, hvad den studerende har lært. Flere læringsteoretikerepeger imidlertid på, at der er et vigtigt læringspotentiale i selve skrivepro-cessen. Eksempelvis argumenterer psykologen Lev Vygotsky i bogen Mindin Society (Vygotsky, 1978) for at bruge skrivning som et medium for tan-ken. Skrivningens potentiale for at fremme indlæringen er også omdrej-ningspunktet for bogen Skrive for at lære – faglig skrivning i de videre-gående uddannelser af Dysthe, Hertzberg og Hoel (Dysthe et al., 2005).Forfatterne fremhæver at: ”Skrivning hjælper os både med at få øje pånye sammenhænge og til at afsløre mangel på sammenhæng og forståelse.Skrivning fører til dybdelæring i stedet for overfladisk læring og hjælper os

18 Iben Gjødsbøl og Sara Green

til at tilegne os stoffet. Skrivning kan føre til ny erkendelse og indsigt. Skriv-ning er altså en vigtig læringsstrategi” (Dysthe et al., 2005: 16). Skrivningkan med andre ord anskues som et erkendelsesredskab til at fastholde ogstrukturere tankerne og dermed gøre dem tilgængelige for revidering ogvidereudvikling.

Dette projekt fokuserer på skriveaktiviteter, der har til formål at trænestuderende til at skrive for at udvikle viden. Vi ønsker at sætte fokus påden kreative dimension af skrivning – et aspekt som ofte overses på devideregående uddannelser. Emnet for projektet er motiveret af vores egneerfaringer som undervisere, hvor vi oplever, at vores studerende har sværtved at komme i gang med at skrive eller kan få skriveblokader i forbindelsemed større opgaver som et speciale. Og det er en skam – for selv oplever vii vores arbejde med at skrive videnskabelige artikler, at erkendelserne ikkeankommer før men derimod i kraft af, at man skriver.

Den didaktiske faglitteratur rapporterer om udfordringer med, at mangestuderende investerer mere psykisk energi i ”blokeringer og vedligeholdel-se af gammel modstand end i de kreative processer, der åbner for læring ogfaglig indsigt” (Dysthe et al., 2005; forord af Krogh-Jespersen et al., p.11).Studerende bruger meget tid på at læse fagligt stof men langt mindre tid påat skrive. Konsekvensen bliver derfor, at mange studerende – når de skriver– ofte kun genfortæller eller parafraserer andre tekster snarere end at brugeskrivning til at udvikle deres eget perspektiv. Desuden har mange en ten-dens til at udsætte skriveprocessen, til de føler, at de ”ved nok” til at kunneproducere et skriftligt produkt. Det kan få den konsekvens, at der ikke ertid eller overskud til at modtage og bruge vejlederens feedback (Dysthe etal., 1999), og at det skriftlige arbejde bliver en frustrerende oplevelse (Pe-dersen, 2004).

Der kan være forskellige årsager til de oplevede udfordringer. Den af-ventende strategi for skrivningen kan bl.a. skyldes præstationsangst ellerperfektionisme hos den studerende men kan også være et resultat af mådender typisk undervises på. I undervisningssammenhænge møder de studeren-de ofte kun skriftlige produkter i form af afsluttede og gennemredigeredefagtekster, og de kan derfor få en opfattelse af, at god skrivning indebæreret næsten færdigt produkt med en specifik form og struktur. Det ufærdige,processuelle skriveprodukt er så at sige usynligt for de studerende. Deru-dover er det også sandsynligt, at mange studerende ganske enkelt ikke harerfaring med eller anerkender potentialet i at bruge skrivning som en støttefor tanken. For at fremme sådanne erfaringer og færdigheder har vi forsøgtat designe undervisningsaktiviteter, som kan få de studerende til ikke bare

2 Skrivning som læringsstrategi 19

at skrive mere men også at skrive på en måde, som understøtter udviklingaf deres ”egen stemme” og dermed en dyb læringsproces (Entwistle, 2009).

I projektet ønsker vi at lade to former for skrivning komplementere hin-anden – hvad Dysthe et al. (2005) kalder hhv. ”tænkeskrivning” og ”præsta-tionsskrivning” (se tabel 2.1). Hvor det primære formål med præstations-skrivning er at producere et godt tekstprodukt ud fra fagspecifikke krav,har tænkeskrivning fokus på udvikling af ideer. Begge processer kan væreforankrede i fagligt indhold og begge indeholder elementer af kreativitetog kritisk stillingtagen til det skrevne. Men en vigtig motivation for at fo-kusere på tænkeskrivning er at aktiviteter fritaget for formål om at leveop til formelle krav (præstationsskrivning) kan åbne op for kreative pro-cesser som ellers kan hæmmes hvis den kritiske stillingtagen tilkobles fortidligt. F.eks. kan fokus på sproglig korrekthed være hæmmende for udvik-ling af nye ideer eller af de studerendes ”egen stemme”. Tænkeskrivning ogpræstationsskrivning kan derfor tænkes som strategier, der har forskelligeformål, og som er nyttige på forskellige tidspunkter i et læringsforløb.

Tabel 2.1. Modificeret efter Dysthe et al. (2005), figur 6 s. 44)

4

også være et resultat af måden der typisk undervises på. I undervisningssammenhænge møder de

studerende ofte kun skriftlige produkter i form af afsluttede og gennemredigerede fagtekster, og de

kan derfor få en opfattelse af, at god skrivning indebærer et næsten færdigt produkt med en specifik

form og struktur. Det ufærdige, processuelle skriveprodukt er så at sige usynligt for de studerende.

Derudover er det også sandsynligt, at mange studerende ganske enkelt ikke har erfaring med eller

anerkender potentialet i at bruge skrivning som en støtte for tanken. For at fremme sådanne

erfaringer og færdigheder har vi forsøgt at designe undervisningsaktiviteter, som kan få de

studerende til ikke bare at skrive mere men også at skrive på en måde, som understøtter udvikling af

deres ”egen stemme” og dermed en dyb læringsproces (Entwistle 2009).

I projektet ønsker vi at lade to former for skrivning komplementere hinanden – hvad Dysthe

et al. (2005) kalder hhv. ”tænkeskrivning” og ”præstationsskrivning” (se tabel 1). Hvor det primære

formål med præstationsskrivning er at producere et godt tekstprodukt ud fra fagspecifikke krav, har

tænkeskrivning fokus på udvikling af ideer. Begge processer kan være forankrede i fagligt indhold

og begge indeholder elementer af kreativitet og kritisk stillingtagen til det skrevne. Men en vigtig

motivation for at fokusere på tænkeskrivning er at aktiviteter fritaget for formål om at leve op til

formelle krav (præstationsskrivning) kan åbne op for kreative processer som ellers kan hæmmes

hvis den kritiske stillingtagen tilkobles for tidligt. F.eks. kan fokus på sproglig korrekthed være

hæmmende for udvikling af nye ideer eller af de studerendes ”egen stemme”. Tænkeskrivning og

præstationsskrivning kan derfor tænkes som strategier, der har forskellige formål, og som er nyttige

på forskellige tidspunkter i et læringsforløb.

Tænkeskrivning Præstationsskrivning

Formål ”Tænke med pennen” Få og udvikle ideer Tydeliggøre vage tanker Forklare for dig selv

Kommunikere Præsentere Fremlægge Forklare for andre

Karakteristiske træk

Kreativ tænkning Teksten orienteret mod skribenten selv Procesorienteret

Kritisk analytisk tænkning Modtagerbevidsthed Fokus på produkt

Modtager Skribenten selv Medstuderende Vejledere som dialogpartnere

Eksterne læsere Censor eller underviser som evaluator

Sprog Personligt Ekspressivt Uformelt og udforskende/legende

Formelt og tilpasset fagspecifikke krav Vægt på sproglig korrekthed

Genre Tænketekster, notater Fagopgave, artikel osv.

Tabel 1. Modificeret efter Dysthe et al. (2005, figur 6 s. 44).

Som det fremgår af tabel 2.1, er tænkeskrivning mere udforskende oglegende, og modtageren er skribenten selv, medstuderende, eller vejlede-re/undervisere som dialogpartnere (snarere end som evaluator). At fremmetænkeskrivning stiller derfor ikke kun krav til den studerende men ogsåtil underviseren og medstuderende, som skal hjælpe med at udvikle ideersnarere end at evaluere disse. Vi ønsker derfor at undersøge, om man som


underviser og vejleder kan designe øvelser og feedback således, at under-visningen mere eksplicit fremmer brugen af skrivning som læringsstrategi.Vores fokus er her på hvordan de studerende oplever øvelser i tænkeskriv-ning, som har idéudvikling og læring som det primære formål.

Problemformulering

Hvordan oplever de studerende udbyttet af mindre skriveøvelser i undervis-ningen, tiltænkt som redskaber til at udvikle, strukturere og fastholde ideerom et fagligt emne?

Metode og empiri

For at besvare problemformuleringen har vi udviklet og afprøvet tre forskel-lige skriveøvelser i Gjødsbøls undervisning af studerende på uddannelsenMaster of Public Health (MPH) ved Institut for Folkesundhedsvidenskab,Københavns Universitet. Observationer af og refleksioner over disse øvelserudgør én form for empiri. Derudover har vi lavet kvalitative, semi-strukturerede interviews (Kvale et al., 2008) med i alt fire studerende. Tre af dissedeltog i skriveøvelserne i Gjødsbøls undervisning. For at undersøge po-tentialet for tænkeskrivning i forbindelse med større opgaver, og hvordanman som vejleder bedst kan understøtte den proces, har vi desuden in-terviewet en studerende som tidligere har skrevet speciale hos Green. In-terviewmaterialet udgør det væsentligste empiriske materiale for besvarel-sen af problemformuleringen. De studerende citeres under pseudonymerneKatja, Maria, Lars og Louise.

Skriveøvelser i undervisningen

Faget Kvalitative metoder er placeret på MPH-studiets obligatoriske fælles-del. De fleste studerende tager en masteruddannelse i forlængelse af deresprofessionsbachelor (eksempelvis sygepleje, fysioterapi, eller jordemoder-uddannelsen), og de har derfor stor praktisk erfaring fra sundhedsvæsenet.De fleste studerende er godt bekendt med den kvantitative forskningstra-dition, til gengæld finder mange det svært og udfordrende at undersøgefolkesundhedsvidenskabelige problemstillinger kvalitativt, herunder at ud-vikle deres egne analyser, fortolkninger og formuleringer.


Kurset er tilrettelagt i fire forskellige blokke med fokus på hhv. denkvalitative metodes erkendelsesteori (blok I), analytiske perspektiver (blokII: livsverden, magt og styring samt praksis); udvalgte kvalitative metoder(blok III: semi-strukturerede interviews, deltagerobservation og dokumen-tanalyse), samt kvalitet i og formidling af kvalitative undersøgelser (blokIV). Sideløbede med undervisningen skal de studerende tilegne sig praktiskerfaring med at bruge kvalitative metoder ved at udføre et miniforsknings-projekt, hvor de i grupper identificerer en problemstilling og laver kvalita-tive semi-strukturerede interviews og deltagerobservation.

De studerende evalueres til slut med en skriftlig eksamensopgave (in-dividuel eller gruppeopgave), hvori de beskriver selve forskningsprocessenog reflekterer både metodisk og analytisk over deres vidensproduktion vedat sætte de teoretiske begreber i spil, der introduceres i løbet af kurset. Vo-res formål med at integrere skriftlighed som en del af undervisningen erbåde at stimulere de studerendes erkendelser og at støtte deres faglige skri-vekompetencer.

Skriftlighed i idéudvikling

Den første øvelse fandt sted under den anden kursusgang, hvor de stude-rende skulle finde emne og ideer til deres miniforskningsprojekt. De fik igrupper af to studerende udleveret en kæmpe post-it. Først skulle den enestuderende på to minutter forklare sin interesse for et emne eller idé til etprojekt til den medstuderende, der udelukkende måtte lytte. Derefter hold-tes et minuts pause i fuldkommen stilhed, hvor begge studerende reflekte-rede over, hvad der var blevet sagt. Den samme studerende skulle derefterigen forklare sin idé men nu på kun ét minut efterfulgt af endnu et minutsrefleksionspause. Derefter fik grupperne fem minutter til at nedfælde deemner, ideer og tanker, der var kommet frem. Denne skriftlige del foregik ifælleskab, men det var den studerende, der havde lyttet, som førte pennen.Bagefter blev øvelsen gentaget ved at de studerende byttede roller som hhv.taler og lytter.

Samskrivning

Den anden skriveøvelse var tiltænkt som katalysator for fælles idéudviklingog faglig refleksion over, hvorvidt og hvordan de forskellige grupper kunnebruge Michel Foucaults begreber om magt og styring som analytisk rammei deres egne projekter. Efter en indledende forelæsning fra underviseren fik


de studerende 10 minutter til parvist at diskutere følgende to spørgsmål ommagt- og styringsperspektivets relevans for deres egen problemstilling: 1.Hvilke indsigter om jeres felt vil I kunne skabe med et magt- og styringsper-spektiv? 2. Hvordan kunne I kombinere et magt- og styringsperspektiv medet livsverdenperspektiv? Herefter fik de studerende 10 minutter til at svarepå hver deres spørgsmål (1 eller 2). Efter 10 minutter byttede de studeren-de tekst og skulle nu læse, redigere og skrive til i den tekst og svar, deresmedstuderende netop havde produceret. Den relativt korte tidsbegrænsningfor skrivningen blev sat for at fjerne noget af præstationspresset hos de stu-derende, fordi alle teksterne vil fremstå ufærdige efter så kort tid (Dysthe etal., 2005: 173). Grundet sygdom måtte Gjødsbøl overlade undervisningentil en kollega med det resultat, at de enkelte diskussions- og skriveinterval-ler uheldigvis blev væsentligt forkortet.

Refleksionsskrivning som peer-feedbackform

Den sidste skriveøvelse var en del af kursusgangen om det kvalitative in-terview som metode og skulle hjælpe de studerende i deres proces medat udarbejde en spørgeguide til brug i deres miniforskningsprojekter. Somforberedelse til undervisningen havde hver gruppe udviklet et udkast til eninterviewguide. Selve skriveøvelsen havde fokus på peer feedback (Moo-re et al., 2013; Pearce et al., 2009; Rienecker et al., 2015), og formåletmed den var at bruge skriftlig feedback til at stimulere refleksioner og fast-holde ideer til at forbedre og forfine gruppernes interviewguides. Øvelsenstartede med, at hver gruppe blev sat sammen med en anden gruppe. Denene gruppe fik nu 10 minutter til at afprøve deres interviewguide på en an-den gruppe ved at én person agerede interviewer, og en anden person fraden modsatte gruppe agerede interviewperson. Resten af de studerende (totil tre) agerede observatører og tog noter under interviewet. Herefter blevsamtlige studerende instrueret i at nedskrive deres umiddelbare refleksio-ner over, hvordan interviewet var forløbet ud fra følgende spørgsmål, dervar formuleret af underviseren og fremgik af en slide på projektoren: Hvadfungerede godt i interviewet, og hvad var svært? Var der spørgsmål, der varsvære at stille (og besvare), og som bør omformuleres? Blev I inspirerede tilat inkludere nye spørgsmål? Gav interviewet anledning til etiske refleksio-ner? Hvad hæftede I jer ved mht. kropssprog? Hvordan var interaktionenog samtaledynamikken? Har I lært noget om jeres egne forforståelser ogderes rolle i interviewet? Efter 10 minutters skrivning fik de studerende tilsidst fem minutter til mundtligt at samle op og udveksle oplevelser af in-


terviewet, inden de gentog øvelsen ved at bytte roller. De studerende var påforhånd instrueret i, at de nedskrevne refleksionstekster skulle deles medderes medstuderende til brug i det videre arbejde med interviewguiden.

De studerendes oplevelse af skriveøvelserne

For at belyse både potentialer for og udfordringer med at integrere skrivningi undervisningen interviewede vi tre studerende efter gennemførelsen afden sidste skriveøvelse. Vi spurgte bl.a., hvad de oplevede som givtigt ogsvært ved øvelserne; hvad der fungerede særligt godt eller dårligt; hvordande oplever det at skrive i forhold til at diskutere mundtligt; i hvor høj gradde var bevidste om formålet med skriveøvelserne, samt hvorvidt og hvordandisse kunne forbedres. Vi fokuserer i det følgende primært på øvelse 2 og3, fordi skriftlighed stod mere centralt i disse end i øvelse 1.

Skriften som katalysator for erkendelse, bevidstgørelse ogsamarbejdsrelationer

Hovedformålet med samskrivningen (skriveøvelse 2) var, at den skullehjælpe de studerende til at se potentialet i at bruge Foucault’s begreber ommagt og styring som analytisk perspektiv i deres egne forskningsprojek-ter. Her gav alle tre studerende udtryk for, at den tidsmæssige ramme forøvelsen satte begrænsninger for udbyttet.

Katja og Maria have begge haft svært ved at sætte det teoretiske ind-hold fra forelæsningen i spil under øvelsen. Lars oplevede imidlertid en godsammenhæng mellem skriveøvelsen og den synopsis for projektet, som destuderende arbejdede på sideløbende med undervisningen: ”Jeg tror egent-lig bare, det var fordi, vi ikke havde tiden til at samle ordentligt op bagefter.For det [skriveøvelse 2] var jo egentlig relevant nok i forhold til den synop-sis, vi sad og arbejdede på. Så jeg forsøgte faktisk at anvende teorierne iforhold til de spørgsmål, jeg gerne ville svare på.” Katja tilføjede: ”Dettvinger én til at tænke over de teoretiske inputs, man har fået, hvorimodfeedback-øvelsen var rimelig nem. Det har de fleste prøvet før. Men at fånoget teori som man skal relatere til sit projekt – det har jeg i hvert faldmindre erfaring med, så det synes jeg, er en meget god øvelse.”

Skriveøvelser kan således hjælpe de studerende til at reflektere overde teoretiske inputs fra forelæsningen og omsætte dem analytisk – detteeksemplificerer, hvordan skrivning kan bruges til at udvikle viden (Krogh,


2010: 25). Ved at skulle formulere sig fagligt og være tvunget til i én elleranden grad at sætte den teoretiske terminologi i spil gennem den skriftligeformidling af stoffet bliver de studerendes læring internaliseret (Kringstadet al., 2014). Dog havde udbyttet af øvelsen utvivlsomt været større, hvisde studerende havde haft bedre tid til at omsætte de teoretiske indsigter tilfag- og kontekstrelevante tekster om deres individuelle problemstillinger.

Maria satte ord på, hvordan det at blive tvunget ud i skriftlighed ogsåindebærer en overskridelse: ”Nogle [studerende] skal jo bruge længere tidfor at komme i gang og over den der tærskel med at begynde at skrive no-get. Og hvor fint formuleret skal det nu være, når man skal sende det tilhinanden og alt det der? Selvom man måske ikke har så mange barrierer,så har man nogle barrierer i forhold til, hvis man skulle dele det mundtligt– så ville man være mere umiddelbar og ikke tænke så meget over ordene.”Det at skulle ’dele’ og skrive i den samme tekst gjorde de studerende merebevidste om teksten og dens ordlyd, og både Maria, Katja og Lars udtryktepå forskellige vis, hvordan øvelserne medførte en ny form for bevidstgørel-se af skriften. Lars forklarede det således: ”Jeg tænker også, at jeg måskeskrev lidt mere sådan akademisk, end jeg normalt ville gøre i noter til migselv. Jeg tror, jeg omformulerede nogle ting og tænkte, at nu må jeg hellereprøve at bruge nogle af de smarte ord, jeg lige har lært (. . . ) Hvis øvelsener også at få det teoretiske med ind, så er det jo smart. Så var det jo fintfor øvelsens formål at få os til at tænke over de [teoretiske] begreber ogat lære at skrive akademisk”. Selvom Lars altså oplevede, at han ikke barekunne formulere sig kreativt, frit og uden begrænsninger, så illustrerer ci-tatet, hvorledes skriftlige øvelser i undervisningen kan bygge bro mellemtænkeskrivning og den mere akademiske præstationsskrivning (Dysthe etal., 2005).

Skriveøvelserne viste sig også at indeholde et uforudset potentiale i for-hold til at etablere og styrke samarbejdet mellem de studerende. Ved kursetsbegyndelse havde de studerende hverken kendskab til hinanden eller erfa-ring med gruppearbejde. Katja forklarede, at hun betragtede samskrivningsom en øvelse i at komme til at kende hinanden som gruppe: ”For mig vardet egentlig en god og nem måde til at finde ud af, hvordan andre arbej-der skriftligt. Nu hvor vi skal til at lave en opgave sammen, så er det jorart at vide, hvordan andre skriver”. Maria havde på samme måde oplevetskriveøvelsen som relationsskabende. Adspurgt om hun synes, det gav ind-sigter at arbejde videre i en medstuderendes tekst, sagde hun: ”Ja, det synesjeg, det gjorde, absolut! (. . . ) Det er altid inspirerende at mærke, hvordanandre har oplevet den samme situation: hvad de bider mærke i; hvad de


udtrykker, som man måske selv har tænkt på. Sådan synes jeg altid, det er,når man læser [andres tekst] – altså man får nogle flere argumenter, manfår nogle flere indgangsvinkler til tingene, man får nogle flere faglige per-spektiver. Nu kommer vi jo fra forskellige professioner, og i den situation erdet helt sikkert givtigt, det er det”.

Muligheden for at producere fælles tekster var altså et værdifuldt red-skab til at etablere gode samarbejdsrelationer, til at frembringe forskelligefaglige perspektiver og til at få en fornemmelse af de øvrige gruppemed-lemmers kompetencer og måder at arbejde på (Kaae, 2004). Lars ærgredesig over, at hans gruppe grundet den knappe tid ikke nåede at cirkulere de-res tekster mellem alle i gruppen: ”Jeg kunne godt tænke mig, at vi var nåethele vejen rundt, så jeg kunne se min oprindelige tekst efter at to forskel-lige har kigget på den”. Med denne udtalelse tematiserer han potentialet iskriveøvelse nummer 3, nemlig peer-feedback.

Skriften som værktøj til fastholdelse og peer-feedback

Formålet med den tredje øvelse var at skabe rammer for, at de studerendekunne give og udveksle feedback til hinanden skriftligt. Lars så et umiddel-bart potentiale i feedback-øvelsens skriftlige komponent: ”Jeg tror aldrig,jeg ville have overvejet at give skriftlig feedback, hvis vi ikke var blevetbedt om det. (. . . ) I forhold til interviews tror jeg også, det er en god idé atskrive sine oplevelser ned lige bagefter, så man ikke glemmer det”. Katjaså også fordele i både at give og modtage skriftlig feedback, som dermedkunne fastholdes og frekventeres fortløbende i projektarbejdet.

Maria betragtede imidlertid mulighederne for fastholdelse fra et mereindividuelt perspektiv: ”Jeg synes, det var meget godt selv at sidde og sætteord på det, jeg lige havde observeret og været med til. (. . . ) Det handledemeget om at prøve at udtrykke, hvad man ser for sig. Altså man skal skrivedet ind i en opgave, og så tænker man; okay, de her refleksioner skal joned på skrift i opgaven også. Det skal jo gerne komme ud i ordform, såman også på en eller anden måde kan dokumentere den tankeproces, manhar i forhold til at være interviewer eller den, der bliver interviewet, ellerobservatør. Så den øvelse går også ud på at øve det lidt og til det, synesjeg, den fungerer godt”. Samlet set ser vi altså skrivningens potentiale tilat styrke de studerendes individuelle refleksioner, deres kompetencer til ogproces mod at levere et færdigt skriftligt produkt og til at levere og modtagepeer-feedback til fremadrettet brug.


Selve feedback-delen i skriveøvelsen lykkedes desværre med skiftendeheld, fordi ikke alle studerende fik udvekslet deres skriftlige refleksionermed hinanden. Særligt Maria oplevede denne del som mangelfuld, fordihun efter øvelsen ikke modtog nogen form for skriftlig feedback fra denanden gruppe. Maria fremhævede vigtigheden af, at underviseren sætterudvekslingen af feedback i system og opfordrede til, at der afsættes meretid til refleksion og opsamling på feedbacken.

Skrivning som læringsstrategi – potentialer og udfordringer

Ikke alle studerende har erfaring med at bruge skrivning som erkendelses-redskab (Dysthe et al., 2005: 133), og Maria understregede, at undervisereneller vejlederen må rammesætte skriveøvelser med tankeudvikling og læ-ring (og ikke i først omgang form og resultat) i fokus: ”[Det er vigtigt] atse hinandens bidrag – se hvad de andre har skrevet. Og det skulle der væreen helt tydelig rammesætning for inden, hvor man siger: Okay, det her erdelemateriale, men vi er i et learning-lab – vi øver os her, der er ikke nogetmed at tænke for meget igennem, hvordan man formulerer sig. Meningen er,at vi skal selv kunne skrive noget, men at vi skal være klar til at reflektereog dele det, vi har skrevet, med de andre i en opfølgende runde.”

Ligeledes er det vigtigt, at underviseren rammesætter feedback-procedurer og tydeliggør opgaven for de studerende, som skal give feedback. Herforsøgte Gjødsbøl at lægge vægt på, at feedbacken skulle være konstruktivog fremadrettet – dvs. formativ frem for summativ (Rienecker et al., 2015).Forløbet har imidlertid lært os, at man ikke kan forvente, at de studerendekommer i mål med udvekslingen af den skriftlige feedback, selvom de harfået instruktioner herfor. Det er også værd at overveje, i hvilken form fe-edbacken er mest brugbar. Eksempelvis kunne reply-funktioner i Absalonvære et værktøj til at sætte udveksling af feedback mere i system. Omvendtnævnte Maria, at der er fordele ved at tænkeskrive via håndskrevne noter istedet for computeren, hvilket alt andet lige ville komplicere udvekslingenaf peer-feedbacken. Den foretrukne strategi kan være forskellig fra persontil person, og det er en afvejning af fordele og ulemper ved forskellige stra-tegier i forhold til formålet med øvelsen. Eksempelvis kan det variere efter,om formålet er at udvikle ideer til et individuelt projekt eller at samarbejdeom et skriveprojekt.


Øget skriftlighed – på vej mod præstationsskrivning

I kursets sidste undervisningsgang fik de studerende mulighed for at fåfeedback fra underviseren med udgangspunkt i et skriftligt produkt. Herfik individuelle studerende eller grupper en halv time til at stille spørgsmålog få kommentarer til en fem siders synopsis, som de forinden havde sendttil Gjødsbøl. Denne mulighed var tiltænkt som motivation til de studerendefor at komme godt i gang med skriveprocessen. Generelt fremhævede samt-lige undervisere (tre i alt) fra kursets start vigtigheden af og potentialet i atskrive både metodiske, teoretiske og analytiske overvejelser ned gennemhele kurset og forskningsprocessen.

Allerede efter den anden skriveøvelse gav Katja udtryk for at have ta-get opfordringen om øget skriftlighed til sig: ”Jeg kan huske, at det blevsagt for en måned siden, da vi startede, at det var godt at skrive noget nedundervejs. Og jeg vil faktisk sige, at jeg de sidste par uger er begyndt atgøre det mere. F.eks. at skrive nogle undrende spørgsmål ned i teksten, sådet ikke bare er noter om, hvad afsnittende handler om men spørgsmål om,hvorvidt jeg har forstået teksten rigtigt eller noget, som jeg skal huske atspørge om”. Skrivning fremstår her som en bred læringsstrategi, der ogsåfungerer som en katalysator for læseudvikling, forståelse og aktiv deltagel-se i undervisningen (Bak et al., 2015).

Endelig har skriveøvelserne fremhævet det positive samspil mellemtænkeskrivning og præstationsskrivning. Lars oplevede, at hans gruppe al-lerede tidligt i kurset var godt på vej mod en eksamensopgave: ”Til denher opgave føler jeg også, at vi allerede har skrevet en masse undervejs.Vi har f.eks. allerede ting til en introduktion, fordi vi har nedskrevet no-get om, hvorfor vi interesserer os for emnet osv. Måske netop fordi vi harmange skriveøvelser, så starter man ikke helt fra bar bund”. Adspurgt omdette forhold betød, at eksamensopgaven føltes nemmere at gå til, uddy-bede han: ”Ja, det synes jeg nu. Men lige i starten var det ikke klart formig, hvad meningen var med de øvelser. Men dengang vidste jeg heller ik-ke, hvad opgaven gik ud på. Nu kan jeg bedre se meningen med øvelserneend helt i starten”. Lars’ erfaring viser, at løbende skriftlige øvelser ogsåfungerer som et værktøj til alignment mellem undervisning og eksamen –nemlig at sikre progression og sammenhæng mellem formål, indhold ogevaluering af et kursus (Biggs et al., 2007).


Tænkeskrivning og vejledning af større opgaver

Træning i at bruge skrivning som læringsstrategi fra starten af et projektkan være særligt vigtigt i store opgaver som f.eks. et speciale, hvor mangestuderende for første gang møder krav om at skulle overskue et stort tekst-materiale og omskrive deres tekster. Selvom mulighederne og udfordrin-gerne kan være anderledes i en vejledningssituation, sammenlignet med enundervisningssituation, kan der være lige så store fordele i at have mereopmærksomhed på tænkeskrivning i denne kontekst. Dysthe et al. (2005)fremhæver i denne sammenhæng, at et meta-perspektiv på skriveprocesserkan gøre studerende mere bevidste om fordele og ulemper ved forskelligestrategier.

I et interview fortalte Louise (tidligere specialestuderende ved Green),at hun havde haft gavn af tænkeskrivning til at håndtere skriveblokader.Hun fandt det periodevist svært at fokusere på at skrive, især hvis der varmeget materiale, der skulle ”koges ned”, og hun også skulle nå andre opga-ver. At skrive noter og spørgsmål ned hjalp Louise med at strukturere tan-kerne, så hun kunne ”parkere nogle ideer” og vende tilbage til dem senere(cf. Pedersen, 2004). Green foreslog også, at hun i perioder med koncen-trationsbesvær, i stedet for at udsætte skrivningen helt, skulle nøjes med atskrive en enkelt time uden fokus på slutresultatet. Til Louises overraskelseendte hun ofte med at blive grebet af at skrive, og mange af disse teksterkunne senere omskrives og indgå i specialet. Denne strategi har et formåltil fælles med vores første skriveøvelse, nemlig at få taget hul på udfor-dringen med at kombinere faglig viden og egne ideer gennem en kreativskriveproces.

For netop at fremme kreativiteten havde Louise god erfaring med at slåWord-stavekontrollen fra: ”Ellers bliver jeg forstyrret af at skulle skriveperfekt og stopper op for at slå ord op. Og når jeg så vender tilbage er ide-en måske væk”. Ligeledes fremhævede hun, at det er vigtigt, at vejlederenikke bare ”kommenterer på alt, hvad man kan komme i tanke om at kom-mentere på”. Trods gode intentioner kan retning af sproglige fejl forstyrreden kreative proces, hvor den studerende arbejder med at udvikle ideer ogen ”egen stemme”. Dette illustrerer fint komplementariteten af præstations-og tænkeskrivning.

Louises erfaringer giver også anledning til refleksion over fordele ogulemper ved at dele skriftligt materiale tidligere med andre (vejleder, un-derviser eller medstuderende). På den ene side understregede Louise vig-tigheden af at turde dele ufærdige tekster med vejlederen: ”Man har jo


vejledere af en grund. Det nytter jo ikke noget, at man skriver det hele fær-digt, og at man så har møde med vejlederen og finder ud af, at man er gåeti en helt forkert retning og så er nødt til at skrive det hele om”. Imidler-tid tydeliggjorde Louise, at der kan være gode grunde til modviljen modat dele skriftligt arbejde tidligt i processen: ”Jeg hader at sende noget, jegikke føler mig færdig med. Ikke så meget fordi jeg er bange for at få kritik.Mere fordi jeg skal have lov til selv at tænke tingene igennem, til at fyldenoget skriftligt i mine tanker. Nogle gange får man forslag fra andre hvorman tænker: Det ved jeg godt, jeg er bare ikke nået dertil endnu. Det kangodt være frustrerende, og jeg er ikke så modtagelig for feedback, hvis jegstadig er i gang med at udvikle en idé”. Citaterne tydeliggør vigtighedenaf at forholde sig til den enkelte studerende situation og ikke antage at al-le studerende oplever de samme udfordringer. At tage et meta-perspektivpå skriveprocesser i dialog med den studerende kan derfor ikke blot gøreden studerende mere bevidst om skrivestrategier, men også hjælpe vejlede-ren til at give den type feedback, der er behov for i forskellige stadier afskriveprocessen.

Konklusion

Undervisningsforløbet med skrivning som læringsstrategi har bekræftet osi, at der er et uudnyttet potentiale i at styrke skriftligheden i undervisningenpå universitet. At komme tidligt i gang med at skrive kan åbne blikket for,hvad der er de væsentlige spørgsmål i et projekt, gøre læsning af faglitte-ratur mere fokuseret og opgaven lettere at disponere. Derved kan skrivningsom læringsstrategi spare både tid og frustrationer og potentielt gøre destuderende mere motiverede i deres arbejde med en afsluttende opgave. Vier også blevet opmærksomme på, at tænkeskrivning kan være relationsska-bende og fungere som en værdifuld kommunikationsform, når studerendearbejder sammen i grupper om et skriftligt produkt. At blive god til tænke-skrivning handler således ikke blot om at kunne nedskrive ideer, men ogsåom at turde lade andre se og evt. skrive videre på en ufærdig tekst. Mangevil senere i studielivet og arbejdslivet opleve at skulle skrive rapporter ellerforskningsartikler sammen med andre, og skriveøvelser af denne type kantræne de studerende i den form for samarbejde.

Erfaringerne fra vores studerende viser, at prioriteringen af en ordentligtidsramme har afgørende betydning for læringsudbyttet af skriveøvelser-ne, og vælger man at bruge skriveøvelser, skal man som underviser sørge


for at afsætte tilstrækkeligt med tid. Desuden må man sikre tid til dialogmellem de studerende om de skriftlige produkter, da dette bidrager væsent-ligt til udbyttet af øvelserne. Ønsker man at benytte sig af kombinationenmellem skrivning og (peer-)feedback, er det særligt vigtigt at rammesætteformålet med øvelsen og instruere de studerende i, hvilken type feedbackder er brug for i den konkrete sammenhæng. Vi kan som undervisere ikkeforvente, at de studerende har erfaring med hverken tænkeskrivning ellerat give og modtage feedback, og vi må derfor sætte et eksempel og netopgive feedback med fokus på den kreative læringsproces og ikke det færdigeprodukt. Samtalerne med de studerende fremhæver desuden vigtigheden afmeta-kommunikation omkring skriftlighed, både i undervisning og i vej-ledningssituationer.

Skulle man som underviser etablere skrivning som en decideret læringsstrategi på et kursus, ville det kræve en opskalering af de her beskrevneøvelser. For at udnytte det fulde potentiale i at lade de studerende bruge ogkommunikere gennem skrivning, ville det være en fordel at lade skriveøvel-ser være en fast del af undervisningen. På den måde vil de studerende væn-ne sig til det nye krav om øget skriftlighed og forhåbentlig med tiden oplevedet som rutine at skulle ideudvikle og formulere sig skriftligt. Med tanke pådet tidsforbrug, der er nødvendigt for at opnå succesfulde skriveøvelser, måman naturligvis som underviser grundigt overveje, om prioriteringen af enøget og måske endda gennemgående skriftlighed i undervisningen overho-vedet er mulig inden for de givne tidsrammer, og om udbyttet står mål medtidsforbruget. Endelig kunne man styrke skrivning som læringsstrategi vedaktivt at inkludere og tage udgangspunkt i de studerendes skriftlige produk-ter i den efterfølgende undervisning. Som eksempel kunne de studerendesrefleksionstekster deles i klasserummet og dermed danne baggrund for enfælles diskussion på holdet. I den forbindelse vil det være afgørende, atman som underviser bruger de skriftlige produkter som afsæt til skabe dis-kussion, refleksion og at lære nyt og ikke som eksempler på hverken gode(’rigtige’) eller dårlige (’forkerte’) bidrag. Det er netop skrivningens poten-tiale som et formativt medium for tanken og ikke et summativt produkt, dermed fordel kan styrkes på universitetsuddannelserne.


Referencer

Bak, M. B., Brok, L. S., & Korsgaard, K. (2015). Skrivedidaktik: en vej tillæring. Klim.

Biggs, J., & Tang, C. (2007). Teaching for quality learning at universityMaidenhead. Berkshire, UK: McGraw-Hill Education.

Dysthe, O., & Breistein, S. (1999). Fagskriving og rettleiing ved universi-tet: del 2: intervjustudie ved Institutt for administrasjon og organisasjonsvi-tenskap ved Universitetet i Bergen Rapport/Program for læringsforskning.Program for læringsforskning, Universitetet i Bergen.

Dysthe, O., & Hoel, T. L. (2005). Skrive for at lære: faglig skrivning i devideregående uddannelser. Klim.

Entwistle, N. (2009). Teaching for understanding at university: Deep ap-proaches and distinctive ways of thinking. Palgrave Macmillan.

Kaae, A. (2004). Gruppeproces og – problemer. I: Olsen, B. & Pedersen, K.Problemorienteret projektarbejde – en værktøjsbog. Pp. 97-122. RoskildeUniversitets Forlag.

Kringstad, T. & Kvithyld, T. (2014). Fem prinsipper for god skriveopplæ-ring. Viden om læsning 15: 60-69.

Krogh, E. (2010). Videnskabsretorik og skrivedidaktik. Rapport om etforsknings-og udviklingsprojekt med deltagelse af Avedøre Gymnasium,Kongsholm Gymnasium & HF samt Syddansk Universitet. I: Gymnasiepæ-dagogik, (77). Tilgængelig online: http://www.gymnasieforskning.dk/wp-content/uploads/2013/09/Videnskabsretorik-og-skrivedidaktik-Rapport-om-


et-forsknings-og-udviklingsprojekt-med-deltagelse-af-Avedøre-Gymnasium-og-Hf-Kongsholm-Gymnasium-og-Hf-samt-Syddansk-Universitet.-.pdf

Kvale, S., & Brinkmann, S. (2008). Interview: Det kvalitative forsknings-interview som håndværk. Hans Reitzels Forlag.

Moore, C., & Teather, S. (2013). Engaging students in peer review: Feed-back as learning.

Pearce, J., Mulder, R., & Baik, C. (2009). Involving students in peer review:Case studies and practical strategies for university teaching. Centre for theStudy of Higher Education, University of Melbourne.

Pedersen, K. (2004). Skriveproces of skriveblokeringer. I: Olsen, B. & Pe-dersen, K. Problemorienteret projektarbejde – en værktøjsbog. Pp. 70-94.Roskilde Universitets Forlag.

Rieneker, L. & Bruun, J. (2015). Feedback. I: L. Rieneker, P.S. Jørgensen,J. Dolin og G.H. Ingerslev. Universitetspædagogik. København: Samfunds-litteratur. Pp. 259-279

Vygotsky, L. S. (1978). Mind in society: The development of higher mentalprocess. Cambridge, MA: Harvard University Press.

3

Enabling PhD students to write their ownmanuscripts

Duc Ninh Nguyen

Department of Veterinary and Animal SciencesUniversity of Copenhagen

Introduction

Many PhD students, even English native speakers, find academic writingdifficult (Thomson, 2006). The problem is not only due to insufficient writ-ing and language skills but also derived from other multiple academic skillsincluding reading and comprehension, literature review and data interpre-tation. There is a saying that to be a scientist is to be a writer, meaning thatone cannot be a good scientist if he/she is not able to formulate a good writ-ten story based on the obtained data. The writing process of PhD students isperformed in different ways, depending on their levels of the competencesmentioned above.

I am the daily supervisor and official co-supervisor of two PhD stu-dents which have completely different scientific and writing competences.One Danish student who has deep scientific understanding but limited la-boratory skills and one Chinese student who is capable of performing invivo and laboratory experiments but having limited scientific understand-ing, language and writing competences. Each student is expected to havetwo-three publishable scientific manuscripts before they can submit theirPhD thesis. While the former case can be dealt with by intensive labora-tory training, the latter case remains extremely challenging, especially inthe process of manuscript writing. At the time before this current projectstarted, the Chinese student lacked of most of the required competencesneeded for manuscript writing and this has remained challenging for bothhim and me as the supervisor to complete all required scientific manuscriptswithin the three years of his PhD period. His first manuscript was accepted

34 Duc Ninh Nguyen

for publication but it was done with my intensive involvement in the writ-ing and revision phases. Despite perfect laboratory work which was basedon previously prepared protocol, he had very limited understanding of re-levant literature and capacity to interpret his own data, probably derivedfrom inadequate English skills, insufficient reading, literature review andpeer discussion with other students. Now he is in the process of writing his2nd and 3rd papers followed by the PhD thesis.

Based on this background, the current pedagogical project focusedon the intervention in my supervision and feedbacks to change his writ-ing approaches, thereby improving his writing quality. After analyzing theprevious situations, I decided to change my methods of supervision andmade a written guideline for his remaining PhD period (1.5 years), whichmainly include some laboratory work and writing tasks of two remainingmanuscripts and the PhD thesis. The new supervision methods to be im-plemented were based on the idea “Research is writing” (Thomson, 2006),which was theoretical and has not been scientifically investigated in realsituation with sufficiently high number of participants. These include moreactivities for individual writing practice and co-writing, formative assess-ment, peer feedbacks and peer discussion. The aims are to maximize hislearning in both scientific understanding and writing, which would helphim to achieve the goal of finalizing manuscripts and thesis punctually inthe most possibly independent manner. The outcome of the interventionwill be assessed qualitatively by comparing the quality of his writing andthe independency of his writing management before and after intervention.

Problems and supervision format before intervention

For each sub-project (which will leads to one manuscript in the PhD the-sis), I designed the in vivo animal studies and in vitro studies and developedthe study protocol with this PhD student. Based on the templates of previ-ous studies in the group, he modified details according to the requirementsof his projects and performed the experiments. For the first study and firstmanuscript, I intentionally gave him time and freedom to collect, interpretdata and perform statistics with encouragements that he should find otherPhD students to discuss. After a few weeks, I discovered that he had ma-jor issues with writing and interpretation. The problem was that he wasnot proactive to promptly discuss with the supervisor about uncertain is-sues but instead he continued until being asked or until the problems were

3 Enabling PhD students to write their own manuscripts 35

identified by the supervisor. I felt he might have needed more helps withcloser supervisions. Therefore, I started to provide more inputs, includingdetailed plans of follow-up experiment, suggestion for analytical methodsand provision of relevant papers to support his reading. He performed wellwhat he was told to do, but nothing more than that. This was unusual, asmany PhD students often prefer to manage projects themselves. I tried toexplain to him about the needs of being more independent but everythingdid not progress unless he was provided with specific solutions for specificproblems. I suggested him to start writing the manuscript early as soon ashe collected sufficient amounts of data. For the first manuscript which wasaccepted for publication, it was under pressure to finish early so that thatthe paper can be used for a grant application. I asked him to write sessionby session (Introduction, Materials and Methods, Results, and Discussion),then revised his text and discussed directly the scientific problems and so-lutions. What happened was that he was not proactive and the quality ofhis texts was insufficient and I had to revise most of the text so that themanuscript can be published on time. In principle, he accepted all my re-visions and given tasks with limited critical thinking, reflection and opendiscussions.

Problem analysis

Academic writing is often misunderstood as a simplified procedure includ-ing thinking first followed by a gradual writing process to complete a draftand revision only by tidying and polishing (Thomson, 2006). This leadsto common advice to students with superficial features including gram-mar correction and reformulation of sentence structures. Instead, academicwriting has been gradually considered as a social practice with the pro-cess of writing practice during research time, not after the research beingfinalized. Therefore, academic writing includes literature review, thinking,linking theory and methods, interpreting data, formulating text, editing andrevising, and managing references as a whole (Wolcott, 2001).

With the limited skills in writing and data interpretation in the firstmanuscript, I realized that he had thought that a good PhD student shouldproduce as many data as possible and the writing part can be done laterwith minor problem. That was why he spent time to solve practical prob-lems rather than to read literature to accumulate knowledge in the field.This led to insufficient understanding of the study background and limited

36 Duc Ninh Nguyen

data interpretation. In addition, the inadequate English competences (ver-bal, reading and writing skills) have added another obstacle for him whenreading literature and discussing data with other peers. In summary, twokey problems were identified: language skills and inappropriate perspec-tives about a good PhD study. This was the background for me to initiatethe proper intervention.

Methods

I started the intervention in the middle of his PhD period (after one yearand six months). The intervention will in principle last until the PhD stu-dent submits his PhD thesis, but the evaluation of the intervention was con-ducted after 1.5 months due to the limited time for this final pedagogicalproject. First I wrote a guideline for writing practice (which was based onthe methods part) and sent to him few days prior to the official talk un-der supervision of my departmental supervisor. In the meeting, I explainedthe reason for this intervention (his strengths and weaknesses in the previ-ous manuscript writing phases), clarified details the writing guideline anddiscussed with him how we should work together during the interventionperiod. After 1.5 months, he was supposed to fill in the evaluation form(Appendix A) and have official talk for the assessment to conclude thisproject but he is expected to continue the writing practice under my super-vision until he submits his PhD thesis.

Specific points for the writing practice interventions included:1. Consider research as writing and perform daily writing practice: ThePhD student was encouraged to change his mind set about writing fromwriting up as the last part of a scientific study to embedding writing intodaily research activities. This was implemented by performing daily 30 minresearch writing session. The writing practice can be a summary of what thePhD student has worked on that day or the previous day including data in-terpretation, small report of new results, summary of new research idea orsummary of literature that is read during the day. All notes should thereforebe organized in a journal/folder. Notes can be bullet points and not nec-essarily completed sentences with perfect grammar. Some of the notes orparts of the daily writing products may become parts of the final manuscriptor PhD thesis. In this way, writing can be considered as thinking as well asa social practice rather than just being a skill: treating research as writingand writing as research (Thomson, 2006). By encouraging frequent writing


practice, the PhD student can train himself not only in writing skills butalso in sharpening his scientific thinking. As manuscript writing is a pro-cess of telling a good story based on the obtained data, frequent writingpractice about the related topics may also help students to gradually findthe optimal way to interpret data and organize data in a logical manner.2. Frequent discussion of writing products with supervisors and otherpeers: All ideas and writing products were suggested to be discussed withsupervisors and other peers. The student was encouraged to discuss thewriting products with supervisors 15 min every day or every two days toclarify unsolved problems or unclear knowledge and my tasks were to givehim formative assessments for the writing products and to suggest construc-tive feedbacks to further develop his ideas or writing quality. The studentwas also encouraged to share his idea or writing products with office mateand other students. For many persons, they cannot remember what theyread and write after a short time, and discussion will stimulate thinkingand memorizing important knowledge. Knowledge from literature can onlybecome one’s own knowledge if one can demonstrate, disseminate and dis-cuss it with others.3. Find a co-writer for each manuscript: For each of the remaining twomanuscripts, this PhD student was grouped with another PhD student work-ing in similar field (inflammation and infection in newborn pigs) so thatthey helped each other during the writing process. They acted as co-writer(not necessarily co-author) on each other’s manuscript, meaning that oneshould read and comment on the manuscripts of the other before I (as thesupervisor) edited. In this way, they may also discuss the unsolved ques-tions and future research questions to further develop their ideas.4. Discuss the outline of the manuscript before writing: The manuscriptwriting task was divided into small parts (Introduction, Methods, Results,Discussion) and the student and supervisor worked together to completeone part before moving to the next part. Division into small tasks shouldmake things easier to complete and immediately solve any obstacles. Allnotes produced by the daily writing practice should be considered to berevised to become parts of manuscripts if possible. In addition, in the initialphase of editing/revising manuscripts, I did not edit in details but only gavecomments in the commentary boxes and the student was supposed to workon those comments to improve the text quality. This should facilitate hislearning better, in contrast with giving him detailed edited text and he wouldhave accepted all suggested text without trying to understand his previousproblems to find a solution to solve.

38 Duc Ninh Nguyen

5. Implement formative assessment, constructive feedbacks and proactivediscussion: The student was encouraged to: a) be proactive in calling meet-ings, discussions with supervisors and other peers, and b) be critical withperspectives during discussion (do not just accept what being told but lookfor evidence and other alternatives). Constructive feedbacks and formativeassessments from supervisors and other peers were also implemented forall meetings (not only during the discussion of writing products in point 2).Notes, feedbacks and assessments from meetings should be recorded (inwritten form) for follow-up actions and the student would have opportuni-ties to improve the discussed materials.

Results and Discussion

After 1.5 months, the student filled in the evaluation form for the interven-tion (Appendix B) and we held an official talk to discuss the outcomes andwhether or not the student has benefited from the interventions. As seen inthe evaluation form filled by the PhD student (Appendix B), he was pos-itive with the intervention and he did feel himself that he has progressedwell with both writing competences and scientific communication compe-tences. According to the official evaluation, he felt it was much easier forhim to formulate English sentences in different manuscript parts as well asdiscuss scientific content of the manuscript, partly because he followed myrecommendation to allocate more time for literature searching and reading.He also mentioned that he prefers to have daily small conversations thanformal appointments and he also kept some important written notes afterthe meeting for records to follow up in later phase. Finally, he felt com-fortable and agreed to continue this way of working until the end of hisPhD.

From my own observations, during the first two-three weeks after thefirst talk about the intervention, the student was actively involved in vari-ous types of activities planned in the interventions. I felt he was extremelyactive and really put an effort to improve his academic understanding andwriting. In the remaining part of the intervention period (3 weeks), he wasless active than the first period, probably due to multiple busy tasks in thelab as well as his own manuscript writing. Some of the progresses were asfollow:

• We had daily conversations about writing and it seemed most of hisdaily writing practice was associated with his own manuscript, but not


so many other types of writing practice, e.g. notes from meeting, sep-arate notes following reading new literature. This may be due to theclassical tradition of scientific writing with only articles but not othertypes of writing that he used to. The period of intervention (1.5 months)may be too short to completely change his mindset of writing but weplanned to continue this way until he submits his PhD thesis and hope-fully he will show gradual improvements. Regarding the quality, I re-ceived several parts of a manuscript he has been writing and I feltthe text he produced showed slightly improved quality in manuscriptoutline and scientific understanding, but still with multiple errors ingrammar and sentence formulation. He understood critical parts of themanuscript and acquired sufficient knowledge related to the story ofthe manuscript, which was a great improvement relative to the firstmanuscript. In general, the daily writing practice intervention seemedto stimulate him to search and read literature related to his own workin a more active manner, thereby increasing his scientific understand-ing and writing quality. However, many other types of daily writingpractice was still lacking and he also showed limited understanding ofthe related projects that other group members have been working with.This part needs to be improved in the remaining time of his PhD.

• He frequently discussed with me and peers about new literature, ex-perimental setup and result interpretation. He did find two relevant co-writers for his two remaining manuscripts in his PhD. In my opinion,these two co-writers have been important colleagues to discuss the con-tent of his manuscript as they work in similar fields. The main issue ofthis part was that both co-writers are Chinese and Chinese membersaccount for 40% of the total group members. It was so natural for themto always speak Chinese when they discussed without the presence ofnon-Chinese speaking colleagues. In our group, we evaluated that thiswas not optimal as scientific discussion in English is an important partfor PhD students to think and write in English. However, it seemedto be difficult to change even though we have tried to discuss this ingroup meetings. On the other hand, I recognized after talking to his co-writers that they frequently discussed via informal forums, especiallyvia WeChat (chatting App in their mobile phones), and this seemed tohelp them acquire quickly the information they needed. The outcomeswould probably be more positive if these two co-writers are not Chi-nese so that he would have had more opportunities to have frequentscientific discussion in English.

40 Duc Ninh Nguyen

• He took initiatives promptly and properly to discuss with me aboutimportant matters related to his own work. The medium of discussionwas mainly in the form of informal conversation at either my or his of-fice rather than formal discussion with appointment and well-preparedagenda. I have accepted this type of communication due to the fact thathis personality fits well with informal but not formal conversations. Ievaluated that 1.5 months was rather short to change many things. Incontrast, I plan to push him to try various types of communication inthe remaining parts of his PhD as I believe it will be a good preparationfor his future jobs.

• In connection with the writing practice focusing solely on the manuscriptwriting, he only occasionally took detailed notes during meetings fol-lowed by written formative assessment. He explained that he remem-bered all follow-up points and he could solve problems quickly follow-ing meetings so that keeping notes was not necessary. Again, I did notpush further due to the project time constraints and it may take timeto implement this for a person who was not used to taking notes andwritten records of meeting and formative assessment.

Conclusion and perspectives

After 1.5 months, the intervention, to a certain extent, has helped this par-ticular PhD student engaged more in academic writing. In addition, re-consideration of academic writing as a social practice rather than “writingup” has reset his mindset about the definition of a good scientist from beingproductive with data and experiments to being more comprehensive in allrelevant competences with writing practice as a core component. More en-gagement in reading and writing were key to his development. The learningprocess was slower than expected due to two main reasons: 1) he mainlyfocused on his own manuscript rather than overall writing practice; 2) themain co-writers and colleagues he discussed with are Chinese, which lim-ited his capacity in scientific discussion in English. However, he has pro-gressed and he is expected to reach to a better level at the time of thesissubmission.

Following discussion with my departmental supervisor, a few importantperspectives from the project outcomes were discussed. For my own devel-opment as a supervisor, in this current project, I learned that students havetheir own developmental strategies and that supervisors need to have dif-


ferent and appropriate approaches to individual students so that both theirstrategies are respected and their weaknesses can be improved. Supervi-sors also need to find a balance in the way of supervision rather than onlyguide students to develop into the directions of supervisors’ preference. Thelearning processes of academic writing or other types of academic trainingmay be time consuming; therefore I found that patience, positive thinkingand constructive supervision are critical factors to obtain “win-win” part-nerships.

In principle, this type of intervention can also be applied to other PhDstudents or scientists. However, this is still a great challenge to all youngscientists, not only to non-native speakers, to balance these competencesdue to the time constraint (typically 3-4 years for PhD projects). On theother hand, the role of supervisors is key to guide their PhD students tobecome good scientists, but not only “data producers”, despite the fact thatscientific journals currently have tendency to demand more and more data.The main limitation of this project was to only include one PhD studentfor the intervention and the intervention started slightly late in his PhD(half way). It is therefore unknown how other students with different back-grounds, culture and language skills respond to the intervention. However,with those having difficulty with academic writing but good motivation forscientific research, this intervention is likely a “win-win” approach to bothstudents and supervisors.

References

Thomson, P. (2006). Helping doctoral students write: Pedagogies for doc-toral supervision. Taylor & Francis Limited.

Wolcott, H. F. (2001). Writing up qualitative research. Sage Publications.

42 Duc Ninh Nguyen

ADuc Ninh Nguyen Universitetspædagogikum 2018

8

Appendix 1

Evaluation form for academic writing intervention (1.5 months)

1. How often did you perform the everyday 30 min writing practice (everyday, twice, three times a week)?

2. What are the main writing activities in your writing practice (report new results, summarize new literature, write your own manuscript, summarize new ideas)?

3. How often did you discuss the content of the writing practice with your supervisor?

4. Which aspects did you learn the most from the frequent writing practice?

5. How often did you discuss with your co-writer? Did it help you to facilitate your writing process?

6. How often were you in charge of the meeting (agenda/progress) with your supervisors?

7. How did the formative assessment with your supervisor and co-writer during the writing process help you?

8. How did you take notes for records after each meeting with your supervisors?

9. Did you feel your academic writing competences have been improved in general?


BDuc Ninh Nguyen Universitetspædagogikum 2018

9

Appendix 2

Part II

Improving the lecture experience

4

Does a simple exercise of student-generated takehome message activate and thereby improve theattention level and learning outcome of thestudents during an academic lecture?

Marie Pedersen

Department of Public HealthUniversity of Copenhagen

Introduction

Compared to high school (In Danish: Gymnasieskolen) or other lower edu-cational schools, the university is characterized by many hours of self-studying and few hours of teaching during which the students and the teach-ers are together. Even though most of the learning takes place outside theuniversity, traditional lectures with the person lecturing doing most of talk-ing from some kind of stage, are still a major part of the educational processat most universities.

Lectures are superb to give an overview of a dense curriculum and areamong the most cost-efficient teaching method, operating with a teacher-student ratio of up to 1:450. Lectures can be inspiring, motivating andpromote high quality learning. Thus, students may look forward to somelectures, either because it’s a great topic that they’re very interested in, orbecause the person lecturing is really good and inspiring. Unfortunately,lectures can also be overwhelming (too much information and/or too fastcommunication) or boring. Traditional ways of lecturing are very difficultand challenged by the low level of student activation and feedback, twoessential elements of learning. Long monolog talk without any contact orinteractions with the students may easily result in a poor learning outcomedue to a high degree of passivity among the students and a documenteddrop in concentration after 20 minutes (Dahl et al., 2015). If the lecturing

48 Marie Pedersen

person is not engaged, well-prepared and/or not in tune with the studentsthe students may feel happy to skip the lecture, their attention shift fromthe teacher to other activities such as online communication on Facebookor Snap chat, some students may even leave the lecture hall if they are boredand as a result the learning outcome can be rather limited.

At most universities, lectures are provided in combination with smaller-sized classroom teaching is used (figure 4.1) to ensure student activationand student-centered teaching in order to optimize learning and to stimulatedeep learning approaches (Herrmann et al., 2014).

Lecture Overview Perspec,ve

Classroom teaching Student apply theory and method

Student feedback Student supervision

Student independent learning

Read

Exercises

Fig. 4.1. Relationship between lectures, classroom teaching and student preparation(Herrmann et al., 2014).

Many other factors contribute to the learning outcome of the students ofan academic lecture in addition to the relationship between lectures, class-room teaching and student preparation (Dahl et al., 2015). Some of thesefactors cannot be modified significantly during a single lecture by the lec-

4 Does a simple exercise... 49

turing person such as those related to the students, e.g. their motivation,time, capacity to understand the topic, etc.

The fact that lecturing in existing courses often is required by newly ap-pointed scientific staff at the University can be a challenge because in suchsituations, the ‘guest’ lecturing person has no or very little influence on thestructural factors related to the curriculum, the intended learning objectivesof the course, the schedule, the alignment of the course, the alignment ofthe education and the alignment of the between teachers as well as time andplace of the lecture.

Figure 4.2 summarizes some of the many modifiable factors that can in-fluence on the learning outcome of the students during an academic lectureand that are feasible to modify by the lecturing person in a guest lecturesettings.

Effec%ve Communica%on in Lecturing

Voice Body language

Illustra%ons

Words

Physical Space PowerPoint Slides

Structure Anima%ons Ques%ons

Size Ven%la%on Light

Shape

Arrangement

Temperature

Handouts

Breaks

Colors Topic

Equipment

Students Posture

Appearance

Body Contact Facial Expression

Head Movements Ways of talking Volume

Sounds

Hand, Eye and Preparedness

Exercises

Fig. 4.2. Simplified illustration of different factors that in addition to the spokenword and the context of the lecture may affect the effective communication in lec-turing and that may be modified by a co-lecturer in an existing course.

The context as well as a well-structured and engaged presentation isimportant for the learning outcome. A way for the lecturer to keep the at-

50 Marie Pedersen

tention of the students and hereby to improve their learning objectives is bychanging lectures towards more student activating situations.

There are different ways of activating the students during lecturing suchas:

• Introducing variation in the lecture• Directly engage the students

PowerPoint slides are more and more commonly being used in lecturesas they offer many great possibilities of quick presentation using beautifuland entertaining visual illustrations of complex matter. Sound and anima-tions can be used too in order to increase variation in the lecture. Anotherstrength of using PowerPoint slides instead of blackboards is that the per-son lecturing can face the students and have eye contact. Eye contact is veryimportant for connecting with the students and adaptation of the teaching.PowerPoint can more easily be read as compared to my handwriting. Fur-thermore, PowerPoint slides allow the students to prepare in advance and itcan save the teacher time as PowerPoint can be read easily so the teacherdoes not have to remember every detail and can re-use the slides. The maindisadvantages relate to the high tempo and overload of information.

It is not straight forward to prepare a good PowerPoint slide presenta-tion, too often the presentations are too dense and there is a risk of goingtoo fast as compared with traditional black board teaching. In my point ofview a good PowerPoint presentation is characterized by less is more, fewwords, use large sized text, and many visual stimulating illustrations. Lessor no text forces the students to take the notes and they may learn more.

Most students require handouts of the presentations before class. It isgood and bad. It may help the motivated student to study deeper, to prepareand take note, but there is a risk that the students pay less attention in classas they already feel the know the lecture and that they only pay attention tothe written words not the spoken words.

Variation can also be introduced by the lecturing person though thevoice, body language, by changing from a PowerPoint supported presen-tation to writing or drawing on the blackboard, playing animations or pod-casts, showing overheads or simply by moving around in the room.

In a lecture setting students can also be activated by posing questions,through quizzes, games and other exercises such as being ask to present ordo specific tasks.


Objectives

The aim of the present report is to evaluate whether the use of a simpleexercise of student-generated take home message as part of the traditionallecture to activate and hereby improve the attention level and learning out-come of an academic lecture. I am hypothesizing that it will help the stu-dents to pay attention and to recall the context of the lecture when they areactively reflecting about it at the very end of lecture.

Methods

My project is based on an exercise that I used during three lectures I hadat October the 10th, 11th and the 24th 2016 during which I was lecturingon the Environment, Reproduction, Birth Outcomes and Children’s healthas part of an existing, obligatory course for students at bachelor level inPublic Health, which is provided in the 5th semester, called EnvironmentalFactors, Occupation and Health.

Learningbydoing:Exerciseforyou:

• Writedown1-2thingsthatyoulearnduringthelecture

Fig. 4.3. The exercise applied.

52 Marie Pedersen

Thanksforyesterday– youdidagoodjobSummaryofyourreplies:

• Infertilityiscommonandaffect15-20%inDenmark

• Manyriskfactors,includingnon-environmentalandcocktaileffects

• Lightandnoise(i.e.non-chemical)exposuresmayalsocauseadversereproductiveeffects

• Malefishdevelopeggsafterexposuretoestrogensinwastewaterduetop-pills

• Differentcriticalwindowsofexposure

• Prenatalexposurecanaffectorganogenesisandcausedelayedreproductiveeffects

• Maternalexposuremattersfordaughter’sreproductivehealth,notonlythesons

• Reproductioninmanygenerationsmaybeaffectedbyexposuretoenvironmentalfactors

• Fathers-to-beshouldnotsmokepottominimizeriskofreducedspermquality

• Pregnancyisadynamicstageoflife,week6differfromweek30

• Maternalexposuretobothairpollutionandnoisecanincreasetheriskofpreeclampsia

• Childrenofmotherswithpreeclampsiahavehigherriskofdevelopingasthma

Fig. 4.4. Summary of the students’ take-home messages.

When the students entered the room, I greeted them and handed them anote paper and a pen. After introducing myself and the topic of the lectureI presented the students for the exercise. I asked them to prepare their owntake-home message (figure 4.3). I explained to them that it was important tobe able to summarize briefly what you learn and that they would gain fromdoing so on their own in the future. After the lecture, I collected their notesand in the start of the following lecture I summarized the student’s replies(figure 4.4) and we discussed if any issues were unclear.

Results and Reflections

The exercise was successfully completed. As expected the exercise waseasy to do. It required no preparation from me or the students. All the stu-dents wrote a line or two summarizing something they learned. There wassome overlap in their replies, but much less than I had foreseen. It was fun


for me to read what was important and new information for the students.I did not systematic evaluate what the students thought about the exerciseso I can’t formally assess the student’s perception of the exercise. I simplynoted how many notes I got, if the replies made sense and from discussionswith the students it is my impression that they liked the exercise. In my ownpoint of view, it worked well to quickly summarizes the students’ reply asa wrap-up the key-points of the earlier lecture in the start of the followinglecture.

Perspective of the exercise

My goal was simply to activate the students and make them reflect aboutwhat they learn being at the lecture. However, the if the goal had been toformally evaluate the learning outcome of students, one could had asked thestudents if they learn more doing the exercise or one could had given only arandom half of the students the exercise and compared their performance ina multiple-choice quiz that I made for the final lecture with the performanceof the non-exposed half. Finally, instead of using paper and pens, which Ido not recommend as the students kept the pens, one could simply ask thestudents to e-mail their replies.

The exercise can be viewed as a simple way to active the students whichmake the students reflect and remember (low level of cognition) and alsoevaluate (higher level of cognition) the context of the lecture as illustratedby the Bloom’s learning theory (figure 4.5).

54 Marie Pedersen

Fig. 4.5.

References

Dahl, B. & Troelsen, R. (2015). Lecturing (Chapter 4, p.179-199), in Uni-versity Teaching and Learning, editor Rienecker, L. & Jørgensen, P. S. &Dolin, J. & Ingerslev, G. H., Samfundslitteratur

Herrmann, K. J., & Bager-Elsborg, A. (2014). Effektiv holdundervisning:En håndbog for nye undervisere på universitetsniveau. Samfundslitteratur.

5

Student activation in theoretical physics lectures

Jan Olaf Haerter

Niels Bohr InstituteUniversity of Copenhagen

Summary. For theoretical courses, such as in physics or mathematics, it continuesto often be common, to present lecture material in a blackboard-type format. Thisstudy addresses the question, if students of such courses might perceive this teach-ing style outdated, as many other presentation styles are possible. To this end, itwas addressed, whether students of theoretical physics would prefer more studentactivation, that is, direct student participation in lectures. They were also asked todescribe, in how far they would benefit from pre-recorded lectures, available beforethe actual class date, or from other forms of presentation, such as slide show perfor-mances. The results obtained were quite clear: for the type of lecture in question,27 respondents mostly found that (i) blackboard presentations were the preferredteaching style; (ii) moderate student activation is useful in internalizing the teachingmaterial; and (iii) replacing lectures by additional exercises, yielding more studentactivity would not be advisable, even if lectures were pre-recorded and availablethrough the web.

Introduction

Evidence supports, that implementing changes in physics curricula, e.g.,in terms of more research-focused teaching, is cumbersome (Dancy et al.,2010). The challenge of such implementation was found to be rooted inmultiple factors, such as situational obstacles. A well-recognized teachingmodification, which the majority of instructors in physics are reported tobe aware of (Dancy et al., 2010), is peer instruction (Crouch et al., 2001).

56 Jan Olaf Haerter

Peer instruction aims to depart from classical teaching in physics courses,by allowed for more interaction between peers (student-student) as well asstudent-instructor. A key element of peer instruction is, that all students inthe classroom should become engaged in the questions posed, and try toarrive at a conclusion together with their peers. Peer instruction has beenshown to yield positive outcomes in student learning, when compared tomore traditional teaching styles, for a variety of classroom settings (Crouchet al., 2001). In times, when a vast array of media is available for teach-ing, such as web-based interaction (Dey et al., 2009), traditional teachingstyles in physics should be under scrutiny. Dey et al. (2009) mentions, thatthe use of multiple forms of media can allow for stimulation of various sen-sory input, e.g. words combined with pictures work better than using one ofthese alone. In their study, based on a specific experimental setting for un-dergraduate students, they found that recorded video presentations could beconsidered an alternative to “live” classroom presentations. They howeveralso mention, that “live” lecturers might come across as more convincing,and that students were found to spend more time looking at the instructorin live lectures. Conversely, Kiesler and Sproull (1997) found that humanswere more likely to interact with agents that had a human face, as opposedto a cartoon. Similarly, Lester et al. (1997) coined “the persona effect”,by which learners preferred interactions with a “live-like character”. Animportant point raised by Dey et al. (2009) is that students, who watchedonline presentations, in contrast to live lectures, were not motivated to takenotes during the online presentation. Also this point will be addressed inthe current study.

In this work, the key problem addressed is, whether, by which means,and to which extent, a lecture series in theoretical physics should be en-hanced by student participation and varied forms of presentation. It is quitecommon in traditional theoretical physics lectures, to carry out most lec-tures as blackboard presentations, where mathematical derivations are writ-ten out explicitly by the instructor, and students often simply copy thesenotes down. Student participation often is seen as optional or perhaps evenunnecessary, but can be encouraged by some instructors. It must be notedthat, at many universities, the lecture may be supplemented by student ex-ercise sessions, where the focus is on the students’ performance in solvingproblems.

The particular course “Complex Physics”, used here as an example, is atheoretical lecture series, geared towards MSc students in physics, but alsoallows students from other disciplines to participate. This year (Fall, 2018),

5 Student activation in theoretical physics lectures 57

the course was, for the first time, offered as a general degree course, that is,all students within the “general physics” study line were required to takethis course. In previous years, the course was run under the title “SpecialTopics in Complex Systems”, making it an elective.

With the new position in the course curriculum, Complex Physics re-ceived more students and posed a challenge of greater diversity in studentlevel to the course to the course instructors (Prof. Kim Sneppen and my-self). Adding student activation to the course lectures, mostly by interactivequizzes during the lectures, and more interaction during exercise sessions.

Context of the study - Course description of “ComplexPhysics”

Course style and role

This is a graduate course offered by the section Biocomplexity of the NielsBohr Institute, University of Copenhagen. In recent years, approximately20 to 30 students signed up for this course, and most of them participatedin, and successfully completed, the exam. This year, a far larger number of46 students have registered for the course.

To give some background, the Biocomplexity section is, roughly speak-ing, a disciplinary combination of statistical physics, biology and the geo-sciences. Most employees within Biocomplexity are theoretical physicistswith interests in interdisciplinary work, especially in evolution and geo-physics. The course “Complex Physics”, is now, for the first time sinceits existence as “Special topics in complex systems”, part of the standardcurriculum for Master’s level physics students at Copenhagen University.This explains also the larger number of students who have registered forthe course this year.

Course organization

The course consists of two lectures (90 minutes and 45 minutes) and corre-sponding exercise sessions (90 minutes each). There are two class meetingsper week, on Tuesdays and Thursdays. Currently, lectures are almost en-tirely “teacher-first” type lectures, carried out as black board derivations ofrelevant mathematical foundations for the different topics. Exercises, con-versely, are almost entirely “student-first” meetings, where students solve

58 Jan Olaf Haerter

homework assignments within small groups or on their own – as they pre-fer. The instructors are present to help discuss obstacles.

Range of topics covered

According to the course description, the topics covered are: Percolation,Networks, Phase transitions, critical phenomena, Monte Carlo simulations,interfaces, agent-based models, self organization, scale free phenomena,game theory, econophysics and models of social systems. Traditionally, thecourse also offered fluid dynamics and turbulence as a part of the standardcurriculum. This additional focus had already been abandoned in recentyears, partially because other courses at Niels Bohr Institute offered suchaspects, partially because the range of topics within this course was alreadyperceived as far too large (course website: https://kurser.ku.dk/course/nfyk18005u/2018-2019).

Intended Learning Outcomes

According to the course description, which is now fixed for Block 1, taughtin Autumn of 2018, the intended learning outcomes are, that, regardingskills, at the conclusion of the course students will be able to implementsimple quantitative models on a computer. The aim is to learn how torephrase a complex phenomenon into a mathematical equation or a com-puter algorithm.

In terms of knowledge, the student is expected to gain basic knowledgeon contemporary research in complex systems. This includes the ability touse fundamental concepts from statistical mechanics, non-linear dynamics,time series analysis, agent based models and self-organizing systems.

In terms of competences, students are expected to learn, how to de-scribe and analyze non-linear systems in terms equations and algorithms.They are furthermore expected to develop computer models of systems withmany interacting parts, including Monte-Carlo simulations, interfaces, net-works, and cellular automata. Implement agents based models to describeself-organized dynamics of structures, for example within network theoryand systems that behave similar across a wide range of scales.

Overarching objective

This course will provide the students with a competent background for fur-ther studies within complex system and biophysics, e.g. a M.Sc. project.


The course will provide the students with tools that have application in arange of fields within and beyond physics.

The exam is an oral exam of 30-minutes duration, typically taken sev-eral days after the final lecture, during which no further aids are allowed.The criteria for the exam assessment are mentioned to be the same as the“intended learning outcomes”.

Description of intervention, aims and analysis data

Description

My intervention consists of allowing students to get active within the lec-ture, not only during exercises, and, additionally, allowing them to steer,to some extent, which detailed path the lecture will take. This was done,by implementing several elements of peer instruction. Short (1-3 minutes)exercises within groups of 2-3 students (usually neighbors in the classroomsetting) were posed at irregular intervals during the lectures. Approximatelythree such exercises were presented to the students during a typical 90-minute lecture. During each exercise, when the time is elapsed, studentsare asked to propose solutions to the problems stated.

As a result of these exercises, I noticed, that in some cases the results ofstudents exercises can lead the lecture into a somewhat different direction,which I then accepted and followed. It turned out, in some cases, to bea very fruitful “detour”, as it allowed aspects to be covered that, mostlyseveral of, the students were already contemplating. The challenge on theinstructor side was, to finally gear the lecture back onto, or near, the pathoriginally envisioned. However, by allowing for detours, the class becamemuch more engaged and the lecture livelier.

Aims

The approach taken aims to put students in the center of the learning, asthey are required to absorb the lecture material immediately by workingwith it. By taking up student questions within the lecture and diverging(somewhat) from the path intended, given students the feeling that they areactually in charge of where things go, rather than me, as a lecturer, guidingthem through well-established material. Through the immediate feedback

60 Jan Olaf Haerter

obtained from listening to the students’ responses, the instructor may re-ceive a much more “real time” signal on when to develop a side topic, whento repeat in other words, or sometimes, that it is reasonable to increase thepace of instruction (since all have reached the correct answer). Peer instruc-tion, which could perhaps be seen as a form of inductive teaching, allowsstudents to re-live the process a researcher goes through when developingan unknown theory. Students need to take steps they have not seen anyonetake before them – hence, for them it is new and exciting. They may remem-ber better, why it is important, if they were able to construct it themselves“from scratch”. It will also be more satisfying to be able to develop a pieceof theory without the teacher first doing it for them.

Analysis

Two types of evaluation were employed.

A. Qualitative: observations by three peer supervisors as well as one de-partmental supervisor, in order to evaluate the intervention.

B. Quantitative: Students were requested to fill in a survey, provided tothem near the final lecture of the course. This survey was used to as-sess how students perceived the intervention. 27 out of 44 registeredstudents responded to this survey.

Results

In the following, the results peer supervision and survey data are presented.

A. Peer and departmental observations: Peer observation was conductedby two associate professor, one from computer science (A), the otherfrom forensic anthropology (B).Comments by A: getting more students to participate in questions, per-haps using more exercises, less questions (the particular lecture at-tended by A had a stronger focus on open class discussion, with fewerindividual exercises, this format should perhaps be improved).Comments by B: some more clarity with student activation could behelpful, some students were not taking part in the activations, otherswere working on their own (which may not be considered a problem,as some prefer to work on their own).Departmental observation by two colleagues.


B. Survey results: After completion of my lectures, I asked the students tocomplete an online survey, containing both general questions, regard-ing the students’ satisfaction with the lecture and exercise sessions, aswell as specific questions, regarding the intervention. Out of the 44students total, 27 responded to the questionnaire. Given that not all stu-dents were always present and not all students attempted the exam, thisis a reasonable turnout for the questionnaire, and represents the classaverage relatively well.

4

After completion of my lectures, I asked the students to complete an online survey, containing both general

questions, regarding the students’ satisfaction with the lecture and exercise sessions, as well as specific

questions, regarding the intervention. Out of the 44 students total, 27 responded to the questionnaire. Given

that not all students were always present and not all students attempted the exam, this is a reasonable

turnout for the questionnaire, and represents the class average relatively well.

(a)

(b)

(c)

General course assessment.

Fig. 1a shows, that the quality of teaching was considered high, and only one student found the quality

to be somewhat low. Almost all students were satisfied or very satisfied with the quality of teaching.

When asked about the academic level of the presentation (Fig. 1b), the response was similarly positive,

with most students finding the level high and nobody finding it low, i.e. trivial. Hence, the course difficulty

and presentation appears to be acceptable for the class “Complex Physics”. To assess, whether teaching

was overall too fast, students were asked to give their perception on pace (Fig. 1c). The response

indicates that the pace was perhaps somewhat high, but not overly so. It could be argued that, at least

at the graduate level, a somewhat too high level is preferable to an overall too low level of presentation,

as long as weaker students are not overwhelmed with the pace.

Figure 1, a, Overall teaching quality, b, academic level of the course, and c, pace at which material was developed during lectures.

Fig. 5.1. a, Overall teaching quality, b, academic level of the course, and c, pace atwhich material was developed during lectures.

62 Jan Olaf Haerter

General course assessment

Figure 5.1a shows, that the quality of teaching was considered high, andonly one student found the quality to be somewhat low. Almost all studentswere satisfied or very satisfied with the quality of teaching. When askedabout the academic level of the presentation (figure 5.1b), the response wassimilarly positive, with most students finding the level high and nobodyfinding it low, i.e. trivial. Hence, the course difficulty and presentation ap-pears to be acceptable for the class “Complex Physics”. To assess, whetherteaching was overall too fast, students were asked to give their perceptionon pace (figure 5.1c). The response indicates that the pace was perhapssomewhat high, but not overly so. It could be argued that, at least at thegraduate level, a somewhat too high level is preferable to an overall too lowlevel of presentation, as long as weaker students are not overwhelmed withthe pace.

5

(a)

(b)

(c)

As I exclusively used blackboard presentations during the lectures, and an interest in the present study

was to assess, whether modifications to such traditional blackboard style should be considered in this

type of theoretical physics lecture, students were encouraged to describe their take on the quality of the

blackboard presentation (Fig. 2a). The overall result is, that the presentation was generally considered

quite clear (most students gave the second best grade). Similarly, an aim was to assess, how satisfied

students were with the clarity of spoken explanations during the lectures, with satisfaction levels quite

high (Fig. 2b). Exercise sessions consisted of student group or individual work (based on preference),

with occasional intervention by either myself or the teaching assistant. Exercise sessions involved very

little “frontal” presentation, most work was done by students on paper or using their laptop computers.

Fig. 2c shows, that, in exercise sessions, the satisfaction with the advice given by the instructors was

reasonably high, with only few students less satisfied. Their partial lack of satisfaction during the

exercises may be related to the lack of staff (only two instructors for 44 students), hence, not all students

could receive help at the same level at all times, depending on their seating in the lecture hall, on their

Figure 2, Blackboard presentation, spoken explanations, and helpfulness ingiving advice during exercises.


5

(a)

(b)

(c)

As I exclusively used blackboard presentations during the lectures, and an interest in the present study

was to assess, whether modifications to such traditional blackboard style should be considered in this

type of theoretical physics lecture, students were encouraged to describe their take on the quality of the

blackboard presentation (Fig. 2a). The overall result is, that the presentation was generally considered

quite clear (most students gave the second best grade). Similarly, an aim was to assess, how satisfied

students were with the clarity of spoken explanations during the lectures, with satisfaction levels quite

high (Fig. 2b). Exercise sessions consisted of student group or individual work (based on preference),

with occasional intervention by either myself or the teaching assistant. Exercise sessions involved very

little “frontal” presentation, most work was done by students on paper or using their laptop computers.

Fig. 2c shows, that, in exercise sessions, the satisfaction with the advice given by the instructors was

reasonably high, with only few students less satisfied. Their partial lack of satisfaction during the

exercises may be related to the lack of staff (only two instructors for 44 students), hence, not all students

could receive help at the same level at all times, depending on their seating in the lecture hall, on their

Figure 2, Blackboard presentation, spoken explanations, and helpfulness ingiving advice during exercises. Fig. 5.2. Blackboard presentation, spoken explanations, and helpfulness in giving

advice during exercises.

As I exclusively used blackboard presentations during the lectures, andan interest in the present study was to assess, whether modifications to suchtraditional blackboard style should be considered in this type of theoreticalphysics lecture, students were encouraged to describe their take on the qual-ity of the blackboard presentation (figure 5.2a). The overall result is, thatthe presentation was generally considered quite clear (most students gavethe second best grade). Similarly, an aim was to assess, how satisfied stu-dents were with the clarity of spoken explanations during the lectures, withsatisfaction levels quite high (figure 5.2b). Exercise sessions consisted ofstudent group or individual work (based on preference), with occasionalintervention by either myself or the teaching assistant. Exercise sessionsinvolved very little “frontal” presentation, most work was done by studentson paper or using their laptop computers. Figure 5.2c shows, that, in exer-cise sessions, the satisfaction with the advice given by the instructors wasreasonably high, with only few students less satisfied. Their partial lack ofsatisfaction during the exercises may be related to the lack of staff (onlytwo instructors for 44 students), hence, not all students could receive helpat the same level at all times, depending on their seating in the lecture hall,on their ambition to get the instructors’ attention, and on the level of prepa-ration for the exercise sessions. Further improvements could involve, thatconcluding summary presentations of the assignments could be given at theend of each exercise session. This would however cut down on the activegroup work that students were able to carry out.

64 Jan Olaf Haerter

Assessment of intervention

The aim of this study was twofold: to assess the merit of increased studentactivation in theoretical physics lectures, which were entirely blackboardbased; to assess, whether the classical format of blackboard lectures shouldbe reconsidered in the setting of theoretical physics. Student activationswere implemented in terms of short quizzes or exercises, which were placedat irregular intervals during the lecture, with approximately three such exer-cises during a typical 90-minute lecture. The general objective of the inter-vention was, to invite students to work through short exercises alone or insmall groups of two to three students (usually neighbors in class), to reca-pitulate the course material in a brief way, to strengthen the understandingof the blackboard notes, and to provoke questions and feedback. This in-tervention gave me immediate feedback on students’ progress during thelecture, and occasionally allowed me to repeat crucial elements in a differ-ent way, so that more of the students were able to follow the subsequentmaterial.

6

ambition to get the instructors’ attention, and on the level of preparation for the exercise sessions.

Further improvements could involve, that concluding summary presentations of the assignments could

be given at the end of each exercise session. This would however cut down on the active group work

that students were able to carry out.

Assessment of intervention.

The aim of this study was twofold: to assess the merit of increased student activation in theoretical

physics lectures, which were entirely blackboard based; to assess, whether the classical format of

blackboard lectures should be reconsidered in the setting of theoretical physics. Student activations

were implemented in terms of short quizzes or exercises, which were placed at irregular intervals during

the lecture, with approximately three such exercises during a typical 90-minute lecture. The general

objective of the intervention was, to invite students to work through short exercises alone or in small

groups of two to three students (usually neighbors in class), to recapitulate the course material in a brief

way, to strengthen the understanding of the blackboard notes, and to provoke questions and feedback.

This intervention gave me immediate feedback on students’ progress during the lecture, and occasionally

allowed me to repeat crucial elements in a different way, so that more of the students were able to

follow the subsequent material.

To more quantitatively assess the merit of this intervention, students were asked, whether they found

student activation useful (Fig. 3). The majority of respondents replied, that these activations were useful,

however, this was not experienced in an overwhelming way (the average assessment only slightly tends

to the side “useful”, whereas approximately 20% of respondents tended towards “useless” and 30% had

a mixed impression.

To further differentiate the students’ perception, students were asked for specific reasons, why such

activations might be considered useful or not (Fig. 4). More than half of the respondents (16/27) found

Figure 3, Usefulness of student activation in lectures.

Figure 4, Advantages of student activation.

Fig. 5.3. Usefulness of student activation in lectures.

To more quantitatively assess the merit of this intervention, studentswere asked, whether they found student activation useful (figure 5.3). Themajority of respondents replied, that these activations were useful, however,this was not experienced in an overwhelming way (the average assessment


only slightly tends to the side “useful”, whereas approximately 20% of re-spondents tended towards “useless” and 30% had a mixed impression.

6

ambition to get the instructors’ attention, and on the level of preparation for the exercise sessions.

Further improvements could involve, that concluding summary presentations of the assignments could

be given at the end of each exercise session. This would however cut down on the active group work

that students were able to carry out.

Assessment of intervention.

The aim of this study was twofold: to assess the merit of increased student activation in theoretical

physics lectures, which were entirely blackboard based; to assess, whether the classical format of

blackboard lectures should be reconsidered in the setting of theoretical physics. Student activations

were implemented in terms of short quizzes or exercises, which were placed at irregular intervals during

the lecture, with approximately three such exercises during a typical 90-minute lecture. The general

objective of the intervention was, to invite students to work through short exercises alone or in small

groups of two to three students (usually neighbors in class), to recapitulate the course material in a brief

way, to strengthen the understanding of the blackboard notes, and to provoke questions and feedback.

This intervention gave me immediate feedback on students’ progress during the lecture, and occasionally

allowed me to repeat crucial elements in a different way, so that more of the students were able to

follow the subsequent material.

To more quantitatively assess the merit of this intervention, students were asked, whether they found

student activation useful (Fig. 3). The majority of respondents replied, that these activations were useful,

however, this was not experienced in an overwhelming way (the average assessment only slightly tends

to the side “useful”, whereas approximately 20% of respondents tended towards “useless” and 30% had

a mixed impression.

To further differentiate the students’ perception, students were asked for specific reasons, why such

activations might be considered useful or not (Fig. 4). More than half of the respondents (16/27) found

Figure 3, Usefulness of student activation in lectures.

Figure 4, Advantages of student activation.

Fig. 5.4. Advantages of student activation.

To further differentiate the students’ perception, students were askedfor specific reasons, why such activations might be considered useful ornot (figure 5.4). More than half of the respondents (16/27) found that activa-tions aided in re-thinking the material – which was the intention I had whengiving such assignment. Equally many found, that discussing the exerciseswith their neighbors was useful. When asked about the time allocated tothese exercises, most of those responding to this sub-question, found thetime appropriate, while some found it too short, and none found it too long.I would hence advise in allocating somewhat more time to these exercises,since the risk of boring students appears low (while that of exposing themto stress seems somewhat higher).

Notably, some (15%) of students in fact prefer to work in a more passiveway during lectures. Not all students find activations equally important.However, the general perception is, that some activation is beneficial.

66 Jan Olaf Haerter

7

that activations aided in re-thinking the material – which was the intention I had when giving such

assignment. Equally many found, that discussing the exercises with their neighbors was useful. When

asked about the time allocated to these exercises, most of those responding to this sub-question, found

the time appropriate, while some found it too short, and none found it too long. I would hence advise in

allocating somewhat more time to these exercises, since the risk of boring students appears low (while

that of exposing them to stress seems somewhat higher).

Notably, some (15%) of students in fact prefer to work in a more passive way during lectures. Not all

students find activations equally important. However, the general perception is, that some activation is

beneficial.

Since my lectures were entirely blackboard focused, I was wondering, whether students missed other

types of presentation. For the type of lecture given, a slide based presentation style could, in principle,

have been possible. Therefore, students were asked to contrast my presentation style to one involving

either: slides (e.g., power point), or pre-recorded lectures (e.g., made available online), to allow the

lecture time to be replaced by more exercises. Students were allowed to give multiple answers, as they

pleased (Fig. 5).

The results on these questions were quite clear: Almost three out of four students preferred the

blackboard presentation, stating that they prefer to take notes and like to copy material down from the

blackboard. In my interpretation, this is difficult, or not satisfying, in the case of power point

presentations, where the pace is generally much higher and material is therefore not copied down by

the audience. Furthermore, I believe that the act of copying from the blackboard onto paper gives a

stronger identification with the class material.

Regarding the option of pre-recorded lectures, available on the web, the results are clear. Only few

students found this to be a reasonable alternative to “live” lectures in the classroom.

Regarding student activation, the results also left little doubt: Most students were in favor of some

student involvement. Few, but not negligibly few (20%), of the students, however, preferred traditional

lectures without any student involvement.

Discussion and Conclusion

Modern technology allows many forms of communication within classrooms or lecture halls. Theoretical physics

lectures, which traditionally place strong emphasis on the development of derivations on the blackboard, make

little use of this array of opportunities. The present work has queried, whether (a) the use of a wider range of

Figure 5, Blackboard presentation vs other types of presentation media.

Fig. 5.5. Blackboard presentation vs other types of presentation media.

Since my lectures were entirely blackboard focused, I was wondering,whether students missed other types of presentation. For the type of lecturegiven, a slide based presentation style could, in principle, have been possi-ble. Therefore, students were asked to contrast my presentation style to oneinvolving either: slides (e.g., power point), or pre-recorded lectures (e.g.,made available online), to allow the lecture time to be replaced by moreexercises. Students were allowed to give multiple answers, as they pleased(figure 5.5).

The results on these questions were quite clear: Almost three out offour students preferred the blackboard presentation, stating that they pre-fer to take notes and like to copy material down from the blackboard. Inmy interpretation, this is difficult, or not satisfying, in the case of powerpoint presentations, where the pace is generally much higher and materialis therefore not copied down by the audience. Furthermore, I believe thatthe act of copying from the blackboard onto paper gives a stronger identifi-cation with the class material.

Regarding the option of pre-recorded lectures, available on the web, theresults are clear. Only few students found this to be a reasonable alternativeto “live” lectures in the classroom.


Regarding student activation, the results also left little doubt: Most stu-dents were in favor of some student involvement. Few, but not negligiblyfew (20%), of the students, however, preferred traditional lectures withoutany student involvement.


Modern technology allows many forms of communication within class-rooms or lecture halls. Theoretical physics lectures, which traditionallyplace strong emphasis on the development of derivations on the blackboard,make little use of this array of opportunities. The present work has queried,whether (a) the use of a wider range of presentation media would be use-ful to the students; and (b) whether stronger student activation would beadvantageous.

The outcome obtained regarding (a) is overwhelmingly clear: most stu-dents queried are content with a blackboard-type presentation style andstrongly oppose the notion of watching pre-recorded lectures as a video pre-sentation. They are similarly opposed to slide-show style presentations inreplacement for blackboard lectures. The main reason given for this choiceis, that students enjoy copying material down to their notebooks duringlecture – an activity which appears to be facilitated by the blackboard pre-sentation.

The outcome obtained regarding (b) is somewhat more varied: manystudents do advocate some form of student activation, which they find togive them opportunities of absorbing the material more directly. However,several students were content without activations, as they prefer to simplyfollow the lecture. Few students were in favor of even stronger student ac-tivation, than the one implemented in this course (approximately 3x2 min-utes during a 90-minute lecture). Overall, it should however be cautioned,that students may require a period of adjustment to a new teaching method,before the learning outcome improves (Sadler, 1998). Furthermore, the cur-rent study is limited in scope, as it only assesses the students’ perceptionof the different teaching formats, while the study does not assess the actualeffectiveness of the measures taken.

It should be emphasized here, that the majority of theoretical lecturesin physics are supplemented by exercise sessions, where students are con-fronted with explicit problems they are expected to solve alone or oftenwithin groups of peers, under the supervision of one or several instruc-

68 Jan Olaf Haerter

tors/teaching assistants. That said, the blackboard format of delivering lec-tures appears to be especially characteristic of quantitative, and stronglyderivation-based, lectures in theoretical physics or mathematics. Formats inother disciplines appear to vary widely, with slide show presentations con-stituting the more common presentation format. I can here only speculate,that ensuring a strong logical thread between different steps in a mathemat-ical derivation can best be achieved by the student writing down (copying)the derivation. This very act of activating “a writing process” may be themost basic form of identifying and activating a learning process in students.Writing was reported as a means of forming structured and coherent know-ledge (Rivard et al., 2000). This realization may also explain, why only asubset of students in fact is in favor of additional student activations in class,while some would even prefer fewer. Most students do enjoy the freedomto interrupt the instructor, to clarify questions.


References

Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experi-ence and results. American journal of physics, 69(9), 970-977.

Dancy, M., & Henderson, C. (2010). Pedagogical practices and instruc-tional change of physics faculty. American Journal of Physics, 78(10),1056-1063.

Dey, E. L., Burn, H. E., & Gerdes, D. (2009). Bringing the classroom tothe web: Effects of using new technologies to capture and deliver lectures.Research in Higher Education, 50(4), 377-393.

Henderson, C., & Dancy, M. H. (2008). Physics faculty and educational re-searchers: Divergent expectations as barriers to the diffusion of innovations.American Journal of Physics, 76(1), 79-91.

Kiesler, S., & Sproull, L. (1997, December). “Social” human-computer in-teraction. In Human values and the design of computer technology (pp.191-199). Center for the Study of Language and Information.

Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A.,& Bhogal, R. S. (1997, March). The persona effect: affective impact ofanimated pedagogical agents. In CHI (Vol. 97, pp. 359-366).

Sadler, P. M. (1998). Psychometric models of student conceptions in sci-ence: Reconciling qualitative studies and distractor-driven assessment in-struments. Journal of Research in Science Teaching: The Official Journalof the National Association for Research in Science Teaching, 35(3), 265-296.

Rivard, L. P., & Straw, S. B. (2000). The effect of talk and writing on learn-ing science: An exploratory study. Science education, 84(5), 566-593.

6

Split Information in PowerPoint AssistedScholastic Settings

David Jason Koskinen


Introduction

The use of multimedia presentations, and specifically PowerPoint slides incollege classrooms is prevalent in modern colleges and universities. Theclarity and consistency of visual content, ability to include complex anima-tions and graphs, and ease with sharing in-class material online are consid-erable strengths. As a presentation method, the use of electronic slides arenot without some cause for concern (Tufte, 2006; Liu et al., 2012), a partiallist being: over density of information, going too fast through the material,and a reliance on slides for all aspects of a presentation even when otherpedagogical tools (question and answer sessions, group discussion, etc.)would be more appropriate.

In addition to good practices for learning effectiveness with electronicslides in a classroom, there is also the issue of what information should bemade available online.1 For courses that do not use a textbook, or for coursetopics that are covered outside the scope of a textbook, there is a potentialdivergence between PowerPoint slides that work well in-class but then arenot adequately descriptive once students leave the classroom. Yet, slideswhich are heavy in content (diagrams, equations, and lengthy text) runsafoul of the ‘split-attention principle’ which occurs “when learners are re-quired to split their attention between and mentally integrate several sourcesof physically or temporally disparate information, where each source of in-

1 The issue of whether lecture slides posted online are beneficial to student perfor-mance (Worthington et al., 2015) is a separate discussion.

72 David Jason Koskinen

formation is essential for understanding the material" (Ayres et al., 2005). Asimple example shown in figure 6.1 is the teaching of trigonometry whereequations are separate from graphs and there is essential accompanyingtext.

(a) (b)

Figure 1: Example from [4] regarding the split-attention effect. In (a) the trigonometric equa-tions and information are spatially disjoint from the illustrated triangle, where in (b) the infor-mation is integrated.

Point slides are associated with a decrease in both a learners ability to retain knowledge as well

as transfer knowledge to other analogous scenarios. Coupled with students having to listen to

any lecture from teachers, it seems straightforward that the kind of information in slides that

work well in a classroom is different than what may be most useful when students are out of the

classroom.

In this project, I explore the appropriateness of including supplemental information in Pow-

erPoint slides that are posted online that are extended versions of the in-class lecture slides.

2 Project Background

This project for having different in-class and online versions of PowerPoint lecture notes was

motivated by the University of Copenhagen course Advanced Methods in Applied Statistics. It

is a graduate level science course which covers topics that are not comprehensively covered in

a single textbook, and relies on modern algorithms, statistical tools, and software development

that are the trademarks of cutting edge data analysis in the physical sciences. In a 3-4 hour class

session, there are normally 3 mini-lectures of 10-20 minutes followed by in-class exercises.

The primary focus of the course is to have practical examples and a working knowledge of

2

Fig. 6.1. Example from (Ayres et al., 2005) regarding the split-attention effect. In(a) the trigonometric equations and information are spatially disjoint from the illus-trated triangle, where in (b) the information is integrated.

The split-attention principle builds on research by R. E. Mayer and col-leagues highlighting that adding more information can overwhelm the cog-nitive load of the learner (Mayer et al., 2001), even in scenarios when theinformation is integrated, e.g. similar to figure 6.1b versus 6.1a. What is in-teresting in today’s multimedia classroom environment is that even relevantdetails in PowerPoint slides are associated with a decrease in both a learnersability to retain knowledge as well as transfer knowledge to other analogousscenarios. Coupled with students having to listen to any lecture from teach-ers, it seems straightforward that the kind of information in slides that workwell in a classroom is different than what may be most useful when studentsare out of the classroom.

6 Split Information in PowerPoint Assisted Scholastic Settings 73

In this project, I explore the appropriateness of including supplementalinformation in PowerPoint slides that are posted online that are extendedversions of the in-class lecture slides.

Project Background

This project for having different in-class and online versions of PowerPointlecture notes was motivated by the University of Copenhagen course Ad-vanced Methods in Applied Statistics. It is a graduate level science coursewhich covers topics that are not comprehensively covered in a single text-book, and relies on modern algorithms, statistical tools, and software devel-opment that are the trademarks of cutting edge data analysis in the physicalsciences. In a 3-4 hour class session, there are normally 3 mini-lectures of10-20 minutes followed by in-class exercises.

The primary focus of the course is to have practical examples and aworking knowledge of various statistical methods as well as experience indeveloping proper data analysis software in a modern programming lan-guage (C++/C11, Matlab, R, Python, etc.). By not having a textbook, thecourse lecture slides and a few scholarly articles are the resource material.The concepts can often times seem understood and obvious by studentswhile in the classroom where a teaching assistant and teacher are available,but due to the imperfection of human memory the same concepts can be-come ‘fuzzy’ while out of class. This is where explanatory lengthy text andexplanations can be of significant utility, which brings about the divergencebetween what’s good for in-class slides (less text and more discussion) andout of class (more text).

While I have reduced the amount of text in the PowerPoint slides overthe past 3 years of teaching the course, I have received feedback from stu-dents that they would enjoy having more material available for their ownpersonal review. Hence, some of the previous PowerPoint slides had toomuch explanatory text to be well-suited for the in-class lecture portion, buttoo little when students had questions and were reviewing for the exam.

Proposal

To better align the in-class and out-of-class learning styles and resources forstudents, it may be appropriate to have two different versions of the same


lecture slides. In this fashion, there would be a version that is optimized forpresenting in front of an audience and another as review material. From thestudents’ perspective this approach is intended to avoid overwhelming theirworking memory (Baddeley, 2010) while they listen to the teacher discussthe material and understand the visual information presented (either in theform of graphs, equations, animations, or scientific sketches).

Execution and Challenges

On a limited scale I implemented having different slides from what is shownin class to what is posted online. This was for the Advanced Methods inApplied Statistics course, which has approximately 30-35 students split be-tween studying for their M.Sc. or Ph.D. degrees. While the course is listedas an elective for physics degrees, approximately 30% of the students werenon-physicists, and of that group only a few were not following a physicalscience specialization (e.g. economics). As such, the response from stu-dents at either the undergraduate level or in other subject areas may differfrom what I experienced.

For the slides that were different between the in-class and online ver-sions, the online versions themselves had two different additions:

• Transition slides that covered what was discussed verbally in classabout the statistics topic, such as Boosted Decision Trees and how thenumber of nodes, number of variables, and depth of the tree can in-fluence the outcome. These were sometimes listed as ’comments’ onthe titles of the slides, and were reminders of best-use practices, seeAppendix A for an example.

• There was also the addition of material that was not covered in-depth inthe lecture portion of the course, but constituted more review materialfor any interested students. For example, the mathematics for weightinga data sample in Adaptive Boosting.

Because each lecture centers on understanding and solving around 3 in-class exercises, each example exercise is specifically worded and was leftthe same between the in-class and online version. For 3 of the lectures inthe Advanced Methods class, as well as two guest lectures in the AppliedStatistics:From Data to Results, I used more streamlined lecture slides forthe in-class portion versus what was posted online.


Since only 30-50 minutes of each 3-4 hour lecture session is actuallylecture, there is a substantial amount of time to interact with the students.Initially some of the students were concerned that the version of the lectureslides they saw online and were following during lecture were not the sameas what was being shown on the projector. This was mostly fixed by one oftwo interventions: including the slides in the actual lecture but only flashingthrough them quickly while explicitly telling the class "This is what we justgot done discussing and is only included as a reminder" or by saying that theonline lecture slides have additional text, and that all the core informationwas present in both versions.

Besides the oversight regarding stating that online version has moreexplicit text- and readingbased slides, the students did not appear to con-sciously notice that there was a difference between those lectures which hadthe extra material in the slides, and the other lectures which did not includeany extra transition slides. Where there was an impact was regarding classparticipation and questions asked during the lecture portion of the class.For lectures where the slides were the same online and in-class, there werefewer questions and comments during the lecture portion with the wholeclass; an average of <1 per 10-20 minute mini-lecture. When there weredifferent versions, students appeared to follow the in-class lecture more in-tently and there were correspondingly 1-2 questions or comments for eachmini-lecture. In both scenarios, the amount of questions and discussion dur-ing the exercise portion each of class was the same.

One challenge in having two different versions of the slides is gaugingwhat impact it has for the students. While the number of questions askedduring the lecture portion of a course can be a proxy for student participa-tion, it might also be that having less material produces less understandingand the increase in a questions are not a reflection of student participation,but of a worse learning outcome. In the future it would be interesting tocompare exam scores from problems associated with lectures that had in-class and online optimized versions, versus lectures where there is only oneset of slides.

Besides gauging the impact of split versions of slides, it should be notedthat it takes a non-negligible amount of additional time to produce addi-tional material for the same lecture. Students can get a sense that whateverappears as explicit text during in slides carries more importance than whatis discussed during the lecture. This is especially true when there is notextbook to serve as the absolute authority. Students which understood theconcepts during the lecture and exercise portion of a class, were sometimes


confused by any divergence between what is contained in the additionalmaterial and what was covered in class.


Having an in-class optimized set of slides was helpful in keeping the stu-dents moving at a similar pace through the lecture content. When there isless text and less material, whatever is being shown or discussed becomesthe obvious focal point. Students do not have to decide between listening tothe instructor, understanding the equation or graphs, while simultaneouslyreading lengthy passages of explanatory text. Even when text is shown in abullet-format and only after a discussion has occurred, many students canfeel rushed. Having the additional material in an online version also helpsthose students who would not focus on the text during the lecture portion,but benefit from clarification and additional information when it comes todoing the written homework and exam.

While there is an additional burden on a teacher to create two (slightlyor significantly) different versions of any PowerPoint slides, there appearsto be a benefit to the students and their in-class concentration. Being explicitabout the differences, explaining why there are two versions, and beingconscious about the extra material that is included in any online versionsare important aspects to consider when taking this approach.

References

Tufte, E. R. (2006). The Cognitive Style of PowerPoint: Pitching Out Cor-rupts.

Liu, J., Yuemin, D., & Xingeng, D. (2012). Role of Blackboard Platformin Undergraduate Education A case study on physiology learning in nurse


major. International Journal of Education and Management Engineering,2(5), 31.

Worthington, D. L., & Levasseur, D. G. (2015). To provide or not to providecourse PowerPoint slides? The impact of instructor-provided slides uponstudent attendance and performance. Computers & Education, 85, 14-22.

Ayres, P., & Sweller, J. (2005). The split-attention principle in multimedialearning. The Cambridge handbook of multimedia learning, 2, 135-146.

Mayer, R. E., Heiser, J., & Lonn, S. (2001). Cognitive constraints on multi-media learning: When presenting more material results in less understand-ing. Journal of educational psychology, 93(1), 187.

Baddeley, A. (2010). Working memory. Current biology, 20(4), R136-R140.


A Transition slide for material that was dicussed in-classbut was included explicitly in the online version of thelecture slides

D. Jason Koskinen - Advanced Methods in Applied Statistics - 2018

• It is common to throw an absurd number of variables into a BDT and have it signify the variables of importance. The more variables used in any supervised learning algorithm, the more difficult it is to debug when something goes wrong, e.g. user error.

• The number of nodes, variables, events, and depth of each tree can influence the classification outcome. Because BDTs are generally fast to train, play around with the settings/options to see the effects.

• Ensure that the variables used in training match the distribution shapes in data. Poor variable agreement will bias the BDT, and if the BDT uses many variables it can be hard to notice that a problem exists.

BDT Comments

!19

Figure 2: Transition slide for material that was dicussed in-class but was included explicitly inthe online version of the lecture slides.

A Appendix

7

7

Kan simple ændringer i underviserensundervisning og fremtoning ændre destuderendes engagement?

David Møbjerg Kristensen

Neurologisk Klinik, HovedpinecentretRigshospitalet

1

Kan simple ændringer i underviserens undervisning og fremtoning ændre de studerendes engagement?

Universitetspædagogikum jan. 2018 – jan. 2019

Afsluttende projekt David Møbjerg Kristensen

Forskningsleder, Ph.d. Neurologisk Klinik Hovedpinecentret

Rigshospitalet Dec 2018

Vejleder: Kirstine Dahl Proceskonsulent, Certificeret coach, Cand.mag.pæd. Det Sundhedsvidenskabelig Fakultet Københavns Universitet

Indledning

Undervisere har til alle tider oplevet forskellige niveauer af motivation ogdeltagelse hos de studerende (Bizzell & Bruce, 2000). Den centrale prob-lemstilling for indeværende projekt er dog, at mine kollegaer og jeg påtværs af vores forskellige faggrene oplever en tiltagende mangel på en-gagement hos KU-studerende. Mit fokus i indeværende forløb har væretudfordringer i forbindelse med holdbaseret undervisning med molekylære

80 David Møbjerg Kristensen

biomedicinere på andet år.1 På trods af at de studerende generelt er dygtige,ressourcestærke og studiet er kendetegnet ved et af de højeste adgangskvo-tienter på KU, da har vi som undervisere haft udfordringer med: (i) man-glende forberedelse, (ii) manglende lyst til at deltage på trods af forbere-delse og (iii) manglende respekt for underviseren og andre studerende.Både min pædagogiske mentor og andre undervisere har oplevet dennetendens, som giver et dårligere læringsmiljø for de studerende selv ogderes medstuderende og desuden sænker motivationen/moralen hos under-viserne, hvilken kan resulter en ond spiral.

Som undervisere har vi stor indflydelse på de studerendes motivation ogderigennem deres engagement og læring. I for-projektet undersøgte jeg der-for med mine kollegaer blandt studerende ved veterinærmedicin, molekylærbiomedicin og folkesundhedsvidenskab, hvordan fremtoning, mere speci-fikt relationskompetencen2, hos underviseren har indflydelse på engage-ment. Det var her tydeligt, at de undervisere, som er villige til at indgå i entæt og humoristisk kontakt fremmede indlæring. Desuden var det tydeligt,at de undervisere, der forlangte noget af deres studerende og samtidigt sattesig i respekt, blev betegnet, som de mest engagerende.

Dette mindre projekt antydede således, at der er en ensartet forståelseaf den gode motiverende underviser, som medfører god indlæring. Det dogher også centralt, at motivering for læring er i vid udstrækning også påvirketaf studerendes forventninger til egen mestring af det pågældende fag elleropgave, som er knyttet til undervisning (Skaalvik & Skaalvik, 2015). Deter derfor vigtigt for os som undervisere, at undervisningen tilrettelæggessåledes, at de studerende ikke alene udfordres, men også støttes i deres ud-vikling – at de ikke føler at kravene overstiger deres mestring af opgaven(Vygotsky, 2004). I indeværende opgave vil prøve at gå et spadestik dy-bere end denne tematik i forbindelse med de ovennævnte problemstillinger.Vores erfaring er nemlig, at ”kulturen” indenfor den individuelle årgang ogdet i særdeleshed indenfor det specifikke hold har en central betydning.

”Elefanten i rummet”3 i disse år, når vi beskæftiger os med de studeren-des indlæring på KU, er det faktum, at KU-underviserne generelt er blevetmere og mere presset ikke mindst på forberedelsestiden (Lauridsen, 2005).

1 https://studier.ku.dk/bachelor/molekylaer-biomedicin/2 Relationskompetence er her defineret som evnen til at skabe relationer til stud-

erende, der fremmer læringen.3 ‘Elefanten i rummet’ er en engelsk talemåde for et indlysende problem, som

ingen har lyst til at tage sig af, endsige tale om.

7 Kan simple ændringer... 81

Derfor er store ændringer i undervisningen mange gange ikke mulig for ossom undervisere pga. det enorme tidspres.

Målet for indeværende forløb har således været at vurdere, hvorvidt”små” ændringer i den pædagogiske metode (undervisningsformen) og re-lationskompetencen kan skabe mere engagerede studerende med en respek-tfuld tilgang til undervisningen og hinanden.

Problemformulering

Kan vi med simple ændringer i (i) undervisningen (didaktikken) og (ii)ved fremtoning (relationskompetence) ændre de studerendes motivation ogopfattelse af os og derved øge deres engagement.

Resume af udvalgt litteratur

Til at belyse problemformuleringen har jeg valgt at bruge Pierre Bourdieusfeltbegreb (Bourdieu, 1994), Karl E. Weicks enactment begreb (Weick,1979) samt relationskompetence baseret på Aristoteles’ treenighed logos,patos og etos ( Aristotle, 1959; Bizzell & Bruce, 2000). Samlet set giverdenne litteratur en forståelse for de dynamikker, som kan værende til grundfor de studerendes manglende engagement på et hold og derigennem mu-lighed for at håndtere den bedre ved små ændringer. Herunder giver jeg enkort redegørelse for essensen i den valgte litteratur.

I Bourdieus forståelse af sociologi er det helt centralt, at samfundetikke kan betragtes som en enhed, men som bestående af en række mindresociale rum (Bourdieu, 1994). Bourdieu kalder et sådan socialt mikrokos-mos et felt. Vores holdbaseret undervisning med de molekylær biomedicin-studerende er således et felt. Disse sociale rum består af et netværk afsociale relationer mellem aktører, der er fastlagt i forhold til aktørernesfordeling af den magt og kapital (ressourcer), der er specifikt anerkendti det pågældende felt (Bourdieu, 1994). Felterne kan ses som autonomearenaer, hvor der hersker specifikke love, værdier og interesser. Såledesbevæger underviseren, der går fra laboratoriets arena til undervisningensarena sig mellem forskellige felter med hver deres love, værdier og in-teresser (Bourdieu, 1994). I denne analogi fra min egen verden, da er derforskellige typer egenskaber og ressourcer, der er anerkendt i et molekylær-biologisk laboratorie på Rigshospitalet sammenlignet med et undervis-


ningslokale med anden årsstuderende. I laboratoriet feltet er det forskn-ingsgennembrud og diverse bevillinger, der giver prestige, mens økonomisksucces negligeres. I undervisningslokalet er uddannelse ved ledende f.eks.amerikanske eller engelske universiteter (Harvard, Cambridge etc.), ud-seende samt økonomisk succes, der giver prestige. Feltet indeholder deru-dover doxa, der er regler for, hvad der er rigtigt eller forkert. Det er hercentralt for Bourdieus arbejde, at der til stadighed hersker en kamp ommagten til at definere hvilke regler og værdier, der gør sig gældende i feltet.Dem som kan definere regler og værdier vha. flere ressourcer og kapitaler gerne dem, som trækker sig fra magtkampene sejrrigt. Ser man på helesamfundet, da bruger Bourdieu sine kapitalbegreber og kampen om definer-ingen af doxa til at forklare, hvordan ulige adgang til materielle, sociale ogkulturelle ressourcer skaber og reproducerer bestemte magt- og uligheds-former i de sociale felter fra toppen ned til det ydmyge undervisningslokale(Bourdieu, 1994).

Bruger vi feltbegrebet i forståelse af de problematikker, som vi bliverpræsenteret for i undervisningslokalet, da er det tydeligt, at det kan ses somen bølgende magtkamp mellem aktørerne på holdet imellem og mellemdele af holdet (eller hele holdet) og underviseren. Denne kamp er mereeller mindre bevidst, men underoverfladen ligger der andre strukturer påindividniveau, som jeg personligt ikke mener, at Bourdieu helt inkorpor-erer i sin model. Der foregår en helt masse subtilt og underbevidst ageren,som Weicks enactment begreb behandler (Weick, 1979; Murphy, 2015).Ved enanctment skaber vi en personlig mening i verden bl.a. ved at handleud i den (enacte) ud fra vores forestillinger, og opleve hvad de handlingerafføder (Weick, 1979; Murphy, 2015). Hvad enacter studerende under un-dervisningen? Hvis de til stadighed kun taler og interagerer med specifikkemedstuderende fra f.eks. deres læsegruppe eller fra samme kulturelle bag-grund, da enacter de måske, at de andre studerende ikke er værd at samar-bejde sammen med, og så vil det også være den virkelighed, de selv og deekskluderede medstuderende oplever. Således bliver en specifik adfærd tilden verden, som de omgiver sig med. Hvis alle derimod begyndte at løse ar-bejdsopgaver sammen på et hold, da vil de enacte, at der er en fælles enhedpå tværs diverse grupperinger (Weick, 1979; Murphy, 2015).

Når vi bevæger os ind i undervisningslokalet, da kan vi således forestilleos med baggrund i Bourdieus og Weicks arbejde, at der løber en række bev-idste og underbevidst strømninger. Vi kan som undervisere tage udgangspunkti Aristoteles appelformerne logos, patos og etos for at sætte os i fokus i dettevirvar. Her er logosappellen til fornuften hos de studerende, der er intellek-


tuel funderet og er især knyttet til argumentationen. Patosappellen har fokusmod de studerendes følelser, hvor underviseren vækker en følelsesmæs-sig reaktion hos den studerende. Patos kan også gøre tungere stof letteremodtageligt. Etosapellen har fokus på underviserens troværdighed, hvordet naturligvis centralt, at de studerende har tillid til os som afsendere afinformation (Aristotle, 1959; Bizzell & Bruce, 2000).

Interventioner og dokumentation

På baggrund af det beskrevne for-projekt, diskussioner med min pæda-gogiske mentor og pædagogiske vejleder samt med udgangspunkt i denovenfor beskrevne litteratur, designede vi 2 interventioner for holdbaseretundervisning under kurset Signaltransduktion.

Første intervention havde fokus på et af de centrale kompetencemålfor kurset: At kunne læse den videnskabelige originallitteratur og videreformidler viden herfra. For at opnå denne kompetence, da er en stor delaf undervisningen baseret på, at de studerende sammen med underviserenlæser originallitteratur og derefter diskutere forståelsen af denne. Forvent-ningen var:

(i) At afhjælpe den manglende forberedelse ved, at jeg lavede et intro-foredrag, der fungerede som en appetitvækker samt gav information,så de studerende var forberedt i læsning af diverse artikler. Jeg brugther bevidst appelformerne.

(ii) At bruge af ”theory of didactic situations” (TDS) modellen (Chris-tiansen & Olsen, 2006) i introforedraget til, at de studerende blev mereinvolveret. De fik en mulighed for at smage på diverse signal stoffer, såde fik en personlig oplevelse med emnet sammen med deres medstud-erende på tværs af diverse interne grupperinger.

(iii) At afhjælpe effekten af de sociale relationer (anactment, feltets magtkam-pene) ved, at undervisningen blev brudt op i mindre grupper af mig somunderviser. Herved kunne de også diskutere diverse spørgsmål inden,at der skulle svares.

Et centralt fokus for kurset er en bevidst kobling mellem basale forståelseigennem klassisk lærebogslæsning/forelæsninger og originallitteraturlæs-ning med tung empiri. Sat det på spidsen, da vil kurset have, at den induk-tive og deduktive læring kører hånd i hånd (Rienecker & Jørgensen, 2015).


Anden intervention, som vi gennemførte, var betydelig mere vidt-gående i ændringerne af den pædagogiske metode i forsøget på at skabeengagement og nedbryde magtstrukturerne intern på holdene. Målet var atbruge de studerende som en ressource og lade dem tale frem for os forsåledes at lette byrden for underviserne. For at således at ”tvinge” de stud-erende til engagement, havde vi til slut i kursusforløbet et induktivt for-løb baseret på udleveret originale forskningsdata (empiri), som de efterføl-gende skulle sætte i kontekst vha. originallitteratur. De studerende præsen-terede således deres hypoteser og forslag til yderligere forskning for derespeers i grupper af 2-4 studerende med et efterfølgende formativt feedbackforløb. Forventningen var:

(i) At ved skabelsen af en indre dialog i den enkelte studerende blev insti-tutionaliseringsprocessen betragteligt fremmet igennem den induktivebearbejde af undervisningsmaterialet (Christiansen & Olsen, 2006).

(ii) At fjernelsen af os som undervisere som fokuspunktet i undervisnin-gen vil medføre, at de studerende åbnede sig mere og en dialog mellemde medstuderende vil blive skabt. Herved vil feltet blive fuldstændigtbrudt op, hvorved et nyt rum (felt) kunne skabes, hvor engagement blevsat i højsædet pga. ekstra kapital efter en god præsentation og dialog.Der var således en præmie for bedste præsentation/dialog og min pæd-agogiske mentor stille kaffe, te og kage til rådighed for at underbyggeden gode stemning.

(iii) At det at alle studerende skulle give konkret formative peer feedback tilmedstuderende, og efterfølgende fra os som undervisere, vil styrke ikkealene institutionaliserings-processerne (Rienecker & Jørgensen, 2015),men også nedbryde enactment mellem gruppering på holdene.

Inden dette forløb med præsentation og feedback, da holdt jeg et kortintroduktionsforedrag om, hvorfor vi havde planlagt dette forløb, og hvor-dan feedback fungerer bedst. Der blev lagt vægt på, at generel kritik ogligeledes generel ros ikke er så konstruktivt, mens specifik kritik og ros erbedst for det formative udviklingsforløb (Rienecker & Jørgensen, 2015).

Dokumentationen af de to interventioner var rent kvalitativt funderet påobservationer foretaget af den pædagogiske mentor, pædagogiske vejlederog mig selv. Vi sammenlignede med andre lignende timer indenfor sammekursusforløb med de samme studerende, der forløb parallelt med interven-tionen. Derudover havde den pædagogiske mentor og jeg har undervist idet pågældende fag igennem en årrække, hvorfor vi også kunne sammen-ligne med forløbene fra andre år. Udover denne dokumentation baseret på


kollegaobservation, da evaluerede jeg også ved et kort interview blandt destuderende efterfølgende.


Interventionen forløb med få justeringer undervejs, og de studerende vargenerelt imødekommende. Det var bevidst, at jeg som underviser tilret-telagde undervisningen således, at de studerende ikke blev udfordret ek-stra i udhold til den ”normale” undervisning under interventionen, og at deikke følte at kravene oversteg deres mestring af opgaven (Vygotsky, 2004).En fordel var her, at jeg har haft indgående kendskab til denne type stud-erende og således kunne læne mig op ad tidligere undervisnings- og ve-jledningserfaringer. Derudover kunne jeg indlednings vha. af feltbegrebeterkende diverse magtkampe/magtstrukturer på holdende. Det var her ty-deligt, at en gruppe meget dygtige kvindelige studerende dominerende denfællesgruppe på tværs af holdene. Det var således gennemsigtigt, at mangeaf de andre studerende følte sig ”underlegne/domineret” og derfor gerne varstille, når denne gruppe engagerede sig. Jeg var ikke i tvivl om, at enactmentbegrebet, som beskrevet ovenfor, her var udelt tilstede.

Første intervention fungerede godt, og der var ros fra både vejledereog studerende. Rent didaktisk anvendte jeg for første gang bevidst re-lationskompetence (appelformerne) og feltbegrebet i tilrettelæggelse afmin fremtoning og undervisning. Jeg præsenterede mig selv som forskn-ingsleder ved Rigshospitalet og brugte herved etosapellen for at fremhævemin troværdighed og styrke min kapital i forhold til de studerende. Jeg ar-gumenterede stærkt indledningsvis, at engagement er et krav og satte reglerom for god adfærd vha. logosappellen til fornuften – det var i deres egen in-teresse! Under mit introforedrag viste jeg billeder af forskellige smertesce-narier (undervisning omhandlede molekylærbiologien bag smerte), som destuderende kunne genkende. Her blev således patosappellen sammen medTDS modellen brugt, hvilket tydeligt vækkede en følelsesmæssig reaktionhos de studerende og et dybere engagement. I forbindelse hermed, blevholdene brudt om i mindre grupper på tværs af allerede etablerede grupperinternt, og den ovennævnt gruppering af ”stærke” kvindelige studerendeblev ikke formet. I disse mindre grupper smagte de studerende på diverse”smerte fremkaldende” stoffer, hvilket skabte en meget familiær stemn-ing under undervisningen og tydeligvis bryd diverse faste strukturer endnumere ned. Centralt var det, at der ikke var en rigtig eller forkert oplevelse


af den personlige smerte. I denne forbindelse, da enactede jeg bevidst, atjeg ikke for fordelte de ”stærke” allerede engagerede studerende, men ap-pellerede til alle på holdet og opsøgte bevidst de mere stille for at give demmulighed for at i tale sætte deres oplevelser med de smerte fremkaldendestoffer.

Den pædagogiske vejleder nævnte, at de studerende nok aldrig glemtedenne undervisning, hvis ikke andet så i hvert fald pga. aktiveringen afderes egen smerte. Vi har enige om, at vi til slut i undervisningen havdeen årgang af studerende, hvor alle var betydeligt mere engagerede, og dervar generelt tilslutning. De studerende i tale satte og efterfølgende en storinteresse. Centralt i dette forløbs succes var: (i) etableringen af min kapitalindledningsvis igennem appelformerne, (ii) opbrydningen af de etableredegrupperinger (feltet) på holdet, og (iii) den TDS modellen kombineredemed den personlige oplevelse.

Anden intervention fungerede ikke så godt som den første. Det var etklart mere kompliceret forløb for de studerende, der forlangte meget afdem. Det var således klart med det samme, at det at bruge de studerendesom en ressource, til at forstå deres eget materiale igennem et induktivtforløb, også kræver meget fra undervisererne. Indledningsvis lavede jegderudover en central fejl: De studerende fik lov til selv at forme deres egnegrupper på tværs af holdene! Således blev den stærke gruppering af kvin-delige studerende formet, hvorved det var tydeligt, at vores tanke om atskabe en friere dialog igennem et nyt felt og nedbryde ”enactment” pro-cesserne fik et indledende skud fra boven. Desuden var det tydeligt, at destuderende skulle have været blevet hjulpet mere inden, at de selv skullekomme med kritik af deres peers.

Rent undervisningsmæssigt, da blev præsentationerne og den forma-tive feedback givet i 2 særskilte undervisningsgange, hvor den stærke grup-pering dominerede helt de andres præsentationer og dialogen under denene af undervisningsgange. I anden præsentationsrunde var der mere di-alog og stemningen var meget mere fri. På trods af disse problemer, davar det tydeligt, at de studerende havde investeret meget tid i deres arbe-jde og tydeligvis havde nydt den induktive tilgang til undervisningen. Deitalesatte en glæde over at arbejde med rigtig forskningsempiri og at skulleopbygge teorier på baggrund af deres egne erfaringer og ideer. Den pæda-gogiske mentor, pædagogiske vejleder og jeg var efterfølgende også over-beviste om, at institutionaliserings-processerne igennem dette forløb varblevet klart fremmet, og de studerende havde styrket deres kompetencer i


forståelsen af original litteratur. Derudover fik de også styrket deres kom-petencer i præsentation af komplekse koncepter og selvstændigt arbejde.

Jeg blev efterfølgende konfronteret med, hvordan dette induktive for-løb relaterede sig til eksamen. Her kunne min pædagogiske mentor under-strege, at dele af eksamen vil være på en lignende måde, hvor de stud-erende skulle udlede konklusioner igennem dataanalyse med baggrund ideres faglige forståelse. Dette aspekt skulle have været underbygget undermin introduktion til forløbet.

Konklusion

Målet for indeværende forløb var at vurdere, hvorvidt ”små” ændringer iden pædagogiske metode og fremtoningen kunne skabe mere engageredestuderende med en respektfuld tilgang til undervisningen og hinanden.

Den første intervention fungerede bedst, hvis man tager udgangspunkti alene i dette formål. Vha. Bourdieus feltbegreb (Bourdieu, 1994) og We-icks enactment begreb (Weick, 1979) havde vi en analyse platform, der gavos mulighed for at ændre nogle af interne strukturer blandt de studerende,der skabte manglende engagement. Derudover gav appelformer (Aristotle,1959) et redskabssæt til styrke relationskompetencen hos mig som under-viser. Svaret til opgavens centrale spørgsmål er således ”ja” - små æn-dringer, som f.eks. bevidst at skille ”dominerende” studerende ad og brugaf appelformer medførte et tydeligt bedre engagement.

Anden intervention var ikke helt så vellykket, da tanken om at brugede studerende som en ressource var mere krævende end forventet. Det varsåledes ikke en ”lille” ændring i den pædagogiske metode, hvilket krævedemere af både de studerende og underviseren. Derved fejlede denne inter-vention formålet for opgaven. Hvis man dog er kynisk som underviser ogberegner, hvor meget forberedelse jeg havde i forhold til antal konfronta-tionstimer under forløbet, da er regnskabet straks bedre. Samtidigt vil jegsom underviser være bedre forberedt næste år og f.eks. bestemme, hvemder etablerer grupper med hinanden for således at hjælpe med nedbrydningaf dominante grupperinger. Selvom at det var udenfor formålet med inter-ventionen, da var der derudover ingen tvivl om, at denne type undervisning,baseret på induktiv læring i kombination dialog og formativt feedback frapeers, glædende de studerende meget og forbereder dem bedre til verdenuden for KU end klassisk gennemgang af kursusmateriale.


Perspektivering

En væsentlig erfaring fra de to interventioner er, at en erkendelse af rela-tionerne intern blandt de studerende og mellem de studerende og undervis-eren har stor anvendelighed. Der er ingen tvivl om, at f.eks. TDS modellen,forventninger til egen mestring etc. er centrale virkemidler og koncepter igod undervisning. Men vil vi som undervisere til bunds i mangel på en-gagement, da skal vi også have en forståelse for, hvorfor studerende ikkeengagerer sig. Når denne erkendelse er nået, da kan vi efterfølgende nem-mere isolerer problemer og derved ændre på det mikrokosmos, som under-visningslokalet er, og strategisk opbryde negativt indvirkende grupperingereller appellerer til de studerendes igennem patos, logos og etos.

På den praktiske side er forløbet med de to interventioner blevet diskuteretmed både min pædagogiske mentor og pædagogiske vejleder. Specielt an-den intervention havde rum forbedringer, men også samtidigt et stort poten-tiale, som kan blive forløst ved småændringer. Vi skal således fremadrettethave styrket forløbet ved: (i) En bedre introduktion til formativt feedback;(ii) efter en given præsentation give nogle minutter, hvor de studerendesnakker intern om den givne præsentation inden en diskussion i plenum;(iii) og evt. give et guidet forløb, hvor nogle kendte klassiske problemer, derer gennemgående for arbejdsområdet, bliver defineret i forvejen og hervedfungerer som konkrete eksempler.


Referencer

Aristotle Rhetorica. (1959). Edited by W. D. Ross. OCT. Oxford: OxfordUP.

Bizzell, P., & Herzberg, B. (Eds.). (2000). The rhetorical tradition: Read-ings from classical times to the present. Boston: Bedford Books of St. Mar-tin’s Press.

Bourdieu, P. (1994). Raisons pratiques. Sur la théorie de l’action. På danskAf praktiske grunde: omkring teorien om menneskelig handlen. Hans Re-itzel, 5. oplag 2009. København.

Christiansen, F. V., & Olsen, L. (2006). Analyse og design af didaktiskesituationer – et farmaceutisk eksempel. MONA (3) p, 7-23.

Lauridsen, L. K. (2005). Midt i en katastrofefilm. https://universitetsavisen.ku.dk/-dokument9/nyhedsarkiv/2005/2005-06/050611n/

Murphy, T. (2015). Sensemaking: Introduktion til Karl Weick.

Rienecker, L. & Jørgensen, P. S. (2015). Universitetspædagogiske prak-sisser. Samfundlitteratur 1. udg.

Skaalvik, E. M. & Skaalvik, S. (2015). Motivation for læring. Teori og prak-sis. Dafolo Forlag, 1. udg., 1. oplag.

Vygotsky, L. S. (2004). Pædagogisk Psykologi. I Lindqvist G. (red.). Vy-gotsky, om læring som udviklingsvilkår. Forlaget Klim, Århus.

Weick, K. E. (2015). Karl E. Weick (1979), the social psychology of orga-nizing. Second Edition. McGraw-Hill Language.

Part III

Flipped Classroom and blended learning

8

Effect of using online active learning platformson students’ engagement and the teachingoutcomes

Abbas Jafari

Department of Cellular and Molecular MedicineUniversity of Copenhagen

Summary

Increasing evidence has shown that engagement of the students in the pro-cess of teaching and their interaction with the teacher and the teaching ma-terial; collectively known as active learning, improves student learning andperformance. Oral communication and discussion of the teaching materialis a crucial element that ensures interaction of the disseminated materialwith students’ pre-established knowledge. However, majority of the stu-dents remain silent during the class and do not participate in the discussionsor answer teachers’ questions. This study investigates whether use of onlineactive learning platforms can improve student participation in the discus-sions or answering of teachers’ questions, as well as other outcomes such asstudent learning and course/teaching satisfaction. Online platforms (Men-timeter and/or Kahoot) were used during 8 classroom teaching (SAU: stu-dent activating teaching) sessions (medical cell and tissue biology course,bachelor level) and one lecture (Molecular pathology course, master level).The results show that use of online active learning platforms increased par-ticipation of the students in oral communication and discussions by 160%in the SAU sessions. In addition, students’ opinion about the impact of us-ing online active learning platforms was determined using a questionnairein which majority of the students indicated that online platforms helped toincrease their learning in the class and improved students’ satisfaction ofthe teaching and their interest in attending the class. This study indicatesthat use of online active learning platforms can increase student engage-

94 Abbas Jafari

ment in oral communications and class discussions and provides prelimi-nary evidence for beneficial effects of using these online platforms on stu-dent learning and satisfaction of the courses investigated in this study.

Background

Active learning is a student-centered teaching approach in which studentsare actively involved in the teaching process through engagement in activi-ties that require working with the new material and relating them to theirpre-established knowledge (Bonwell et al., 1991; Michael, 2006). Therehas been diverse opinions about whether or not active learning has any ben-eficial effects on student performance and learning outcomes (Andrews etal. 2011; Haak et al., 2011). However, the largest analysis done to dateon the effects on active learning had meta-analyzed 225 studies compar-ing learning from traditional lecturing to active learning and provides com-pelling evidence that active learning improves course and performance stu-dent learning in science, engineering and mathematics education programs(Freeman et al., 2014). Participation in the discussions in the class has akey role in attaining knowledge through active learning (Bonwell et al.,1991; Leitão, 2000; Michael, 2006; Osborne, 2010; Larrain et al., 2014).However, often majority of the students are silent during the class and itis only a small group of students ( 19% of the students) who contributeto oral communications in the class by participating in the discussions oranswering teachers’ questions (non-published, pre-project by Jafari, Villa,and Thomsen, UP2018, University of Copenhagen). Such low level of stu-dent participation in the discussions may not only compromise the benefi-cial effects of active learning on student performance and learning, but alsowould create a concern for the teacher about whether the subject matter isat all understandable for the passive students (who comprise the majority ofthe students in the class)? Such concern and uncertainty can have adverseeffects on the teachers’ performance.

In a previous study on class room teaching, we indicated that major-ity of the passive students (52%) do not contribute to the discussions oranswer teachers’ questions due to uncertainty about the validity of theiranswer/opinion, highlighting fear of failure as a major hurdle to students’contribution to the discussions in the class (non-published, pre-project byJafari, Villa, and Thomsen, UP2018, University of Copenhagen). Onlineactive learning platforms offer the opportunity for anonymous contribution

8 Effect of using online active learning platforms . . . 95

to the discussions during the teaching and therefore have the potential toaddress the challenge of fear of failure. Therefore, in this study, 2 onlineactive learning platforms were used during different teaching sessions andtheir impact on the students’ participation in the discussions, as well asstudent learning and satisfaction was determined.

Methods

The study was performed during two different types of teaching: class roomteaching (SAU: student activating teaching) and lecture. Class room teach-ing sessions were related to the course medical cell and tissue biology or-ganized by the Department of Cellular and Molecular Medicine, Universityof Copenhagen. Two classes of medical students (2nd semester) were in-volved in the study, and each class had 4 individual 2-hours SAU teachingsessions (i.e. total of 8 individual SAU teaching sessions). The number ofregistered students in the classes was 25 and 21; however the number ofstudents participating in the classes was variable throughout different ses-sions. The lecture that was included in the study was related to the courseMolecular Pathology organized by BRIC (Biotech Research & InnovationCentre), University of Copenhagen. 35 master (MSc) students from Mo-lecular Biomedicine and Human Biology programs were present at this 1hour lecture.

To assess the impact of online active learning platforms, Mentimeterand/or Kahoot were employed during the teaching sessions. Kahoot plat-form was used only in the SAU sessions and at the beginning of the classand used for asking questions related to the subject matters covered in theprevious sessions (i.e. using of Kahoot started from session 2). Mentime-ter was implemented in both teaching types (i.e. SAU & lecture) and wasused during the class for asking questions related to the new knowledge andmaterial discussed in that teaching session.

Based on the experience of teaching the same SAU sessions in the pre-vious semesters (since 2016) without using online platforms, the percent-age of students who participated in the discussions has been variable, but inmajority of the cases was found to be around 20%. Interestingly, our pre-project study performed on the students enrolled in the programming courseat the Department of Computer Sciences, Faculty of Science, University ofCopenhagen, indicated an almost similar level (19%) of student participa-tion in class room discussions in the absence of using online active learn-

96 Abbas Jafari

ing platforms (non-published, pre-project by Jafari, Villa, and Thomsen,UP2018, University of Copenhagen). To assess the impact of using the on-line platforms on the student participation in the discussions, the followingnumbers were recorded during the class: total number of students present inthe class, number of students who used the online platforms to answer thequestions or contributed to the discussions, and number of students whovolunteered to answer questions or contributed to the discussions. Usingthese numbers, the percentage of the students who participated in the dis-cussions or answered the questions online or orally was determined in eachclass at each teaching session. Finally, the average percentage of studentonline and oral participation throughout all sessions was calculated usingthe recorded data from the 4 different SAU teaching sessions of both classes(i.e. average of 8 SAU teaching sessions).

To determine students’ opinion about the impacts of using online activelearning platforms on different outcomes, a questionnaire was used at theend of the last SAU teaching session and after the lecture and students wereasked to rank the different statements on a scale 1-5, 1 meaning totallydisagree and 5 meaning totally agree. The following statements were usedin the questionnaire:

1. Using active learning online platforms made the class more attractiveand increased my interest to attend the class.

2. Using active learning online platforms made the learning experiencemore satisfactory.

3. Using active learning online platforms inspired me to participate in thediscussions and to volunteer to answer the teacher’s questions.

4. Using active learning online platforms helped to increase learning inthe class.

For the questionnaire used after the lecture, the statement number 1(please see above) was replaced with the following statement: Using ac-tive learning online platforms helped to increase duration that I remainedengaged & focused on the lecture.

Results

We have previously shown that in the absence of using online active learn-ing platforms, around 20% of the students present in the class participatein the discussions or answer teachers’ questions during class room (SAU)


teaching. Interestingly, using online active learning platforms significantlyincreased student participation in oral discussions, as the average oral parti-cipation in the discussions or answering the questions during SAU sessionswas increased to 51% (Figure 8.1A). On the other hand, average online(anonymous) participation of the students in the discussions or answer-ing questions during SAU teaching sessions was found to be 80% (Figure8.1A). The detailed results indicating the percentages of online and oralcontributions of the students in every SAU teaching session of individualclasses is indicated in Figures 8.1B-C.

4

For the questionnaire used after the lecture, the statement number 1 (please see above) was replaced with

the following statement: Using active learning online platforms helped to increase duration that I remained

engaged & focused on the lecture.

Results

We have previously shown that in the absence of using online active learning platforms, around 20% of the

students present in the class participate in the discussions or answer teachers’ questions during class room

(SAU) teaching. Interestingly, using online active learning platforms significantly increased student

participation in oral discussions, as the average oral participation in the discussions or answering the

questions during SAU sessions was increased to 51% (Figure 1A). On the other hand, average online

(anonymous) participation of the students in the discussions or answering questions during SAU teaching

sessions was found to be 80% (Figure 1A). The detailed results indicating the percentages of online and

oral contributions of the students in every SAU teaching session of individual classes is indicated in

Figures 1B-C.

Figure 1: Impact of using online active learning platforms on students’ participation in discussions/answering

questions. (A) Percentage of online and oral participation of students in discussion/answering questions, before

using online active learning platforms and after using these platforms in 4 SAU teaching sessions of 2 independent

classes of second semester medicine students. Error bars indicate standard deviation. (B, C) Detailed results

indicating the percentage online and oral contributions of the students in each of the 2 classes at each SAU teaching

session.

To determine students’ opinion about the impact of using online active learning platforms on their learning

and their experience of the teaching session, a questionnaire was used. Analysis of the questionnaire results

indicated that 86% of the students agreed/totally agreed that using active learning online platforms made

Fig. 8.1. Impact of using online active learning platforms on students’ parti-cipation in discussions/answering questions. (A) Percentage of online and oralparticipation of students in discussion/answering questions, before using online ac-tive learning platforms and after using these platforms in 4 SAU teaching sessionsof 2 independent classes of second semester medicine students. Error bars indicatestandard deviation. (B, C) Detailed results indicating the percentage online and oralcontributions of the students in each of the 2 classes at each SAU teaching session.

To determine students’ opinion about the impact of using online ac-tive learning platforms on their learning and their experience of the teach-ing session, a questionnaire was used. Analysis of the questionnaire re-sults indicated that 86% of the students agreed/totally agreed that using ac-tive learning online platforms made the class more attractive and increasedtheir interest to attend the class (Figure 8.2A). In addition, 86% of the stu-dents agree/totally agree that using active learning online platforms madethe learning experience more satisfactory (Figure 8.2B). Furthermore, 62%

98 Abbas Jafari

of the students agreed/totally agreed that using active learning online plat-forms inspired them to participate in the discussions and to volunteer to an-swer the teacher’s questions (Figure 8.2C). Finally, and most importantly,81% of the students agreed/totally agreed that using active learning onlineplatforms helped them to increase learning in the class (Figure 8.2D).

5

the class more attractive and increased their interest to attend the class (Figure 2A). In addition, 86% of the

students agree/totally agree that using active learning online platforms made the learning experience more

satisfactory (Figure 2B). Furthermore, 62% of the students agreed/totally agreed that using active learning

online platforms inspired them to participate in the discussions and to volunteer to answer the teacher’s

questions (Figure 2C). Finally, and most importantly, 81% of the students agreed/totally agreed that using

active learning online platforms helped them to increase learning in the class (Figure 2D).

Figure 2: Results of questionnaire about the impacts of using online active learning platforms in SAU teaching sessions on student performance and satisfaction. Students’ opinion about impact of using online active learning platforms on (A) attraction of the class and student interest to attend the class, (B) satisfaction of the teaching experience, (C) inducing interest in participating in oral discussions or answering teachers’ questions, (D) learning in the class. Analysis of the questionnaire after the lecture indicated that 97% of the students agreed/totally agreed that

using active learning online platform helped to increase duration that they remained engaged & focused on

the lecture (Figure 3A), and 100% of the students agreed/totally agreed that using active learning online

platforms made the learning experience more satisfactory (Figure 3B). In addition, 77% of the students

agreed/totally agreed that using active learning online platforms inspired them to participate in the

discussions and to volunteer to answer the teacher’s questions (Figure 3C) and finally, 94% of the students

Fig. 8.2. Results of questionnaire about the impacts of using online active learn-ing platforms in SAU teaching sessions on student performance and satisfac-tion. Students’ opinion about impact of using online active learning platforms on(A) attraction of the class and student interest to attend the class, (B) satisfaction ofthe teaching experience, (C) inducing interest in participating in oral discussions oranswering teachers’ questions, (D) learning in the class.

Analysis of the questionnaire after the lecture indicated that 97% of thestudents agreed/totally agreed that using active learning online platformhelped to increase duration that they remained engaged & focused on thelecture (Figure 8.3A), and 100% of the students agreed/totally agreed that


using active learning online platforms made the learning experience moresatisfactory (Figure 8.3B). In addition, 77% of the students agreed/totallyagreed that using active learning online platforms inspired them to partic-ipate in the discussions and to volunteer to answer the teacher’s questions(Figure 8.3C) and finally, 94% of the students agreed/totally agreed that us-ing active learning online platforms helped them to increase learning in theclass (Figure 8.3D).

6

agreed/totally agreed that using active learning online platforms helped them to increase learning in the

class (Figure 3D).

Figure 3: Results of questionnaire about the impacts of using online active learning platforms in lecture on student performance and satisfaction. Students’ opinion about impact of using online active learning platforms on (A) Duration of focus on the engagement and focus on the teaching, (B) satisfaction of the teaching experience, (C) inducing interest in participating in oral discussions or answering teachers’ questions, (D) learning in the class.

Discussion

This study was aimed to investigate whether using online active learning platforms can increase

contribution of the students to whole-class oral communication during the teaching session by participating

in the discussions or answering teachers’ questions. To this end, two online platforms (Mentimeter and

Kahoot) were used during student activating teaching (SAU) sessions and it turned out that this approach

had a significant increasing effect on the student participation in oral communications during the class, as

number of the volunteers for oral communication in SAU sessions was increased by ~ 168% compared to

similar SAU sessions in which online platforms were not used.

In addition, the employed questionnaire in both SAU and lecture teaching determined the opinion of

students about the impacts of using online active learning platforms and indicated that majority of the

Fig. 8.3. Results of questionnaire about the impacts of using online active learn-ing platforms in lecture on student performance and satisfaction. Students’ op-inion about impact of using online active learning platforms on (A) Duration offocus on the engagement and focus on the teaching, (B) satisfaction of the teachingexperience, (C) inducing interest in participating in oral discussions or answeringteachers’ questions, (D) learning in the class.

100 Abbas Jafari

Discussion

This study was aimed to investigate whether using online active learningplatforms can increase contribution of the students to whole-class oral com-munication during the teaching session by participating in the discussionsor answering teachers’ questions. To this end, two online platforms (Men-timeter and Kahoot) were used during student activating teaching (SAU)sessions and it turned out that this approach had a significant increasingeffect on the student participation in oral communications during the class,as number of the volunteers for oral communication in SAU sessions wasincreased by 168% compared to similar SAU sessions in which online plat-forms were not used.

In addition, the employed questionnaire in both SAU and lecture teach-ing determined the opinion of students about the impacts of using onlineactive learning platforms and indicated that majority of the students be-lieved that using these platforms had helped to increase their learning inthe class and made the teaching session more attractive and satisfactory.However, one of the interesting observations in this study was that in bothSAU and lecture teaching sessions, there were several students who did notagree that using online active learning platforms inspired them to volunteerto participate in oral discussions or answer teachers’ questions (Figure 8.2C& 8.3C). This observation is well in line with the findings of our previousstudy (non-published, pre-project by Jafari, Villa, and Thomsen, UP2018,University of Copenhagen) that indicated some students are not willing toparticipate in oral discussions under any circumstances possibly due to dif-ferent personality backgrounds (e.g. being shy), while these students areindeed active during the class and follow the teaching session.

This study provides preliminary evidence that suggest positive impactsof using online active learning platforms on students’ oral communica-tion, learning and course satisfaction during lecture and SAU teaching ses-sions related to the courses; medical cell and tissue biology, and molecularpathology. However, it has to be taken into account that this study had beenperformed in small groups of students and future studies are required tovalidate these results using not only larger numbers of classes/students, butalso other approaches such as objective testing of the students in terms oftheir learning outcomes and performance in relation to the use of online ac-tive learning platforms. Such studies can include direct comparison of thefinal examination scores in randomly assigned classes that had been taughtby the same teacher, with and without use of online active learning plat-


forms. A limitation of such approach would be possible differences in theperformance of the students in these classes, regardless of the use of onlineplatforms. In addition, it has to be taken into account that the results ob-tained in this study are preliminary and only related to the 2 courses thatwere investigated and whether the observations of this study would be rele-vant to other courses in the area of medical biology or other study programis not known and requires further investigation.

Looking at the project retrospectively, it might have been better if thequestionnaire was designed to have 3 scale answers (e.g. agree, do notagree, do not know/not sure). The rationale for this statement is that theemployed 5 scale answer scheme did not provide extra information in re-lation to the main question of the questionnaire, which aimed to determinewhether students agree or disagree about the different statements and in-deed, the level of the students agreement/disagreement (i.e. agree/disagreevs. totally agree/disagree) have not necessarily provided additional infor-mation that could be used to help answering the question.

In summary, the results obtained in this study suggest that use of onlineactive learning platforms may be a promising strategy to enhance activeparticipation of the students in the class dissuasions and oral communica-tions and may exert beneficial effects on student learning and performance.

References

Andrews, T. M., Leonard, M. J., Colgrove, C. A., & Kalinowski, S. T.(2011). Active learning not associated with student learning in a randomsample of college biology courses. CBE—Life Sciences Education, 10(4),394-405.

Bonwell, C. C., & Eison, J. A. (1991). Active Learning: Creating Excite-ment in the Classroom. 1991 ASHE-ERIC Higher Education Reports. ERICClearinghouse on Higher Education, The George Washington University,One Dupont Circle, Suite 630, Washington, DC 20036-1183.

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N.,Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student

102 Abbas Jafari

performance in science, engineering, and mathematics. Proceedings of theNational Academy of Sciences, 111(23), 8410-8415.

Haak, D. C., HilleRisLambers, J., Pitre, E., & Freeman, S. (2011). In-creased structure and active learning reduce the achievement gap in intro-ductory biology. Science, 332(6034), 1213-1216.

Larrain, A., Howe, C., & Cerda, J. (2014). Argumentation in whole-classteaching and science learning. Psykhe, 23(2).

Leitão, S. (2000). The potential of argument in knowledge building. Humandevelopment, 43(6), 332-360.

Michael, J. (2006). Where’s the evidence that active learning works?. Ad-vances in physiology education, 30(4), 159-167.

Osborne, J. (2010). Arguing to learn in science: The role of collaborative,critical discourse. Science, 328(5977), 463-466.

9

Students’ perceived learning outcome andattitudes towards non-traditional lecturing

Pieter Jan Kerstens

Department of Food and Resource EconomicsUniversity of Copenhagen

Over the years consensus seems to have grown in the pedagogy literaturethat traditional lecturing is ineffective, because it generally does not pro-mote active learning. Through active learning students actively work withthe course content and it helps transferring the content in their long-termmemory. Thus, teachers should embrace alternatives such as flipped class-room and other peer learning methods.

In a flipped classroom, the usual “content delivery” is moved primarilyoutside the classroom so that time in the classroom can be devoted to a vari-ety of other activities that promote active learning. However, this necessar-ily requires more preparation from students than for traditional lectures anda successful implementation of a flipped classroom hinges on the willing-ness of students to cooperate by preparing in advance for the lectures. Sincestudents are so accustomed to traditional lectures they might be hesitant (oreven opposed) to a flipped classroom, because they might not perceive it asmore beneficial to their own learning than a traditional lecture format. Fur-thermore, students might think that teachers are shifting the “burden” ontostudents such that they are “doing our (teaching) job”. This motivates aninteresting research question: How do students perceive their own learningin flipped classroom/peer learning vs traditional lectures? What are theirattitudes towards traditional lectures, flipped classroom and peer learning?

A constructive alignment analysis as part of the Universitetspæda-gogikum revealed some misalignment between intended learning outcomes(ILOs), learning activities and assessment of the “Economic efficiency andbenchmarking” (NIFK16001U) course. In order to improve the constructivealignment of this course and to investigate the above research questions, I

104 Pieter Jan Kerstens

conducted an intervention in this master course where I am a guest lecturerfor 4 lectures in total (4 x 3h) during the academic year 2018-2019. Thus,(figuratively) “killing two birds with one stone”.

Context of the study

The course “Economic efficiency and benchmarking” (NIFK16001U) is anelective master-level course in English of 7.5 ECTS open to all students, butmainly taken by students in the MSc in Agricultural Economics and MSc inEnvironmental and Natural Resource Economics (and some students fromCopenhagen Business School). The ILOs are in Figure 9.1 (or online atfollowing link: https://kurser.ku.dk/course/nifk16001u).

The teaching format consists of traditional lectures with in class exer-cises. It is a relatively small student group: although the course capacity isset at 50, around 20 students signed up for the course in 2017-2018. Thisacademic year (2018-2019) 15 students signed up and 13 completed theexam.

A project report conducted in small groups of students (usually 2 stu-dents, sometimes 3 students) is the basis for the individual oral assessment.Students are completely free to choose the topic for their project and theresearch question. We expect them to do their project using the tools theylearn in the course. During the oral exam, we critically discuss their projectand ask questions about material from the course (which they applied ordid not apply in their project).

9 Students’ perceived learning outcome and attitudes... 105

A project report conducted in small groups of students (usually 2 students, sometimes 3

students) is the basis for the individual oral assessment. Students are completely free to choose the topic for

their project and the research question. We expect them to do their project using the tools they learn in the

course. During the oral exam, we critically discuss their project and ask questions about material from the

course (which they applied or did not apply in their project).

Figure 0.1: Intended learning outcomes of the “Economic efficiency and benchmarking” (NIFK16001U) course

Related literature

A poll conducted by Herreid and Schiller (2013) among STEM teachers identified some

pitfalls of the flipped classroom: one reported pitfall is that “students new to the method may be initially

resistant to because it requires that they do work at home rather than be first exposed to the subject matter in

school.”

Roach (2014) reports on a case study of an implementation of a partially flipped classroom in

a microeconomics course: only 1 lecture out of 3 per week was flipped. Interestingly for our case, the case

study’s focus is to gauge the students’ reaction and perceptions towards a flipped classroom. The study finds

that students respond positively to the partially flipped classroom.

Similarly, Butt (2014) surveyed the attitudes of students towards lectures in general and

towards the flipped classroom in a final year actuarial course. Students response was generally positive with

a 25% minority that viewed flipped classroom as not beneficial to their learning.

Bishop and Verleger (2013) survey recent literature on flipped classroom and report on 11

studies conducted at the undergraduate and high school level that focus on student perceptions of the flipped

Fig. 9.1. Intended learning outcomes of the “Economic efficiency and benchmark-ing” (NIFK16001U) course.

Related literature

A poll conducted by Herreid and Schiller (2013) among STEM teachersidentified some pitfalls of the flipped classroom: one reported pitfall is that“students new to the method may be initially resistant to because it requiresthat they do work at home rather than be first exposed to the subject matterin school.”

Roach (2014) reports on a case study of an implementation of a par-tially flipped classroom in a microeconomics course: only 1 lecture outof 3 per week was flipped. Interestingly for our case, the case study’s fo-cus is to gauge the students’ reaction and perceptions towards a flippedclassroom. The study finds that students respond positively to the partiallyflipped classroom.

Similarly, Butt (2014) surveyed the attitudes of students towards lec-tures in general and towards the flipped classroom in a final year actuarialcourse. Students response was generally positive with a 25% minority thatviewed flipped classroom as not beneficial to their learning.


Bishop and Verleger (2013) survey recent literature on flipped class-room and report on 11 studies conducted at the undergraduate and highschool level that focus on student perceptions of the flipped classroom. Theresults are generally consistent: general student perception of flipped class-room is positive with a significant minority opposed to it.

Set-up

Motivation

As part of the Universitetspædagogikum course I analysed the construc-tive alignment of the course “Economic efficiency and benchmarking”(NIFK16001U). This revealed that the learning activities do not entirelysupport the intended learning outcomes: the traditional lectures (with smallexercises and examples on paper) mainly support the “Knowledge” ILOsand to a lesser extent the “Competences” and “Skills” ILOs. This misalign-ment is a problem for students, because we assess the students by an in-dividual oral presentation of a written group report and subsequent oralexamination. This report is the result of a small research project where thestudents choose their own case for which they do a benchmarking analysisusing the tools they learned in the course. For their project however, theyneed the “Skills” and “Competences” described in the ILOs. Here, flippedclassroom for some relevant lectures might help in resolving this misalign-ment as well by stimulating active learning.

Intervention description

After brainstorming with the course coordinator, we decided to (i) split onelecture into two and; (ii) flip this new lecture (“DEA pitfalls and proto-cols”) and introduce a new flipped lecture (“Critical evaluation of empiricalbenchmarking applications”). Thus, there is an even split between lecturestaught in a traditional lecture format and a flipped classroom format. Thisalso has the advantage that there is no variation in the quality of the teacherwhen we compare the results of the survey between traditional lectures andflipped classroom (assuming my qualities as a teacher are independent ofthe teaching method of course!).

We settled on this particular choice of two flipped classroom lectures,because we thought them well suited to develop the students “Skills” and“Competences” as described in the ILOs:


1. “DEA pitfalls and protocols”-lecture: this lecture deals with frequentlyencountered modelling mistakes and draws attention to some importantmodelling choices that the students will also encounter during theirproject.

2. “Critical evaluation of empirical benchmarking applications”-lecture:we introduced this new lecture this academic year 2018-2019 wherestudents read three anonymized student reports from a similar courseat Copenhagen Business School a couple of years ago. The goal is thatstudents learn to evaluate critically the appropriateness of the chosenmodels; the conclusions drawn from the results and identify possiblelimitations and alternative approaches.

In case of the “DEA pitfalls and protocols”-lecture, I asked the studentsto carefully read a paper, available on Absalon, at home. Students could alsotake an online multiple-choice questionnaire on Absalon before class to testtheir understanding. This questionnaire consists of simple questions, whichthey can answer when they have read the paper. For the “Critical evaluationof empirical benchmarking applications”-lecture1, I simply asked them toread the reports and prepare both positive and critical comments. To aidthem, I provided a “rubrik”: this contains simple questions about a partof the analysis to ask themselves while reading the reports. Two examplesof questions the rubrik contained: “What is the research question of thepaper?” and “What are the assumptions of the model? How plausible arethey in the current setting?”.

I also gave students the option to submit a project proposal for peer-review through Peergrade.io on a voluntary basis.2 The motivation wasthat this gives the students some early, formative feedback on their project.This gives the students the chance to correct and change their project asneeded before doing the actual analysis. Another advantage is that studentscould more easily learn from each other and that I offer a different wayfor students that are less comfortable with speaking-up in class to inter-act with their peers. The “Critical evaluation of empirical benchmarkingapplications”-lecture thus served as a training to prepare them for the peer-review exercise.

1 Unfortunately, this lecture was scheduled the Thursday before the start of theChristmas break and only 4 students participated in this lecture as a result.

2 9 students submitted a project proposal and 5 subsequently gave feedback.


Data collection

Before the start of my own teaching I made a pre-intervention survey avail-able to students on Absalon. The survey asked students about their (i) previ-ous experiences with, (ii) attitudes towards and (iii) perceived effectivenessof traditional lecturing, flipped classroom and peer learning.3 The full sur-vey is in Appendix A. The survey had a response rate of 61.5% (8/13).

I distributed a similar post-intervention survey to each student right af-ter the exam for them to fill in while they waited for our deliberation on theirgrade.4 I chose this approach for two reasons: (i) I expected that studentswould only see the full benefits of the intervention after they had handed intheir project and had finished their exam; (ii) I did not expect many studentsto finish the survey after the exam period. The post-intervention survey hada response rate of 76.9% (10/13). In both cases I collected the students’email so I can match pre-intervention and post-intervention responses.

Results

Given the small sample size, these results are descriptive at best and nowider conclusions can be drawn from it.

Pre-intervention survey

The first question asked them how frequently they have experienced thesedifferent teaching methods in the past. Figure 9.2 provides a summary ofthe results. Clearly, traditional lecturing is still very popular as all studentsselected “most of the time”. In contrast, only 3 students experienced flippedclassroom more than once before and 2 students had one earlier experiencewith it. 5 students experienced peer learning more than once before and1 only once before. Overall, I conclude that the students are very used totraditional lecturing and much less used to (i.e., never or only once) flippedclassroom (5 students) or peer learning (3 students).

3 I provided a definition of these different teaching methods at the start of thesurvey.

4 Naturally, they were also allowed to hand in the survey later.


Figure 2: Frequency of different teaching methods in the past.

Next, I asked about their attitudes towards these teaching methods. All students had a positive

attitude towards traditional lecturing: 6 “like it” and 1 even “love it” (Figure 3). Attitudes towards the flipped

classroom was less positive with only 1 student “like it” and 2 students indicating “it’s fine”. As many

students (i.e., 3) were neutral towards flipped classroom. Results are even less positive for peer learning

where only 1 student had positive attitude towards it and 4 were neutral. Finally, 1 or 2 students gave

inconsistent responses when comparing with the previous question: 3 students indicated they never

experienced flipped classroom in Figure 2 vs 2 students in Figure 3 (and 2 students vs 1 student for peer

learning). This could indicate that they have a prejudice against both teaching methods.

Following-up on this question, the survey asked them to clarify why they “feel this way about

[teaching method]”. A large majority of students gave responses for traditional lecturing that can be

summarized as “I like it because I’m used to it” or “This is the best method for me to learn”. One student

answered:

“Depending on the barrier to asking questions, I think traditional lecturing (with projects/exploitative

elements on the side) is the most efficient, when looking at time spent vs material learned. A professor

became a professor for a reason, so in my opinion you should learn from them, rather than from other

students.”

Another student wrote:

“I like this way of teaching because it works well for me. I think that when trying something else it often

fails, and becomes a waste of time.”

Thus, the students in this sample prefer traditional lecturing, because (i) they think they learn best from

(passive) listening to a lecture; (ii) Other teaching methods they experienced before failed and felt like a

waste of time; and (iii) a university teacher is seen as an authority figure that has all the knowledge to be

learnt.

For flipped classroom, one student admitted that it “could actually boost more our learning experience”, but

expressed fear that it would not work that well if the course material is difficult. Another student wrote: “If

there is enough time to prepare before class, it is probably the best method! Nevertheless, it is not always

possible to read everything is needed for the class, then it become somehow counterproductive.”

Fig. 9.2. Frequency of different teaching methods in the past.

Next, I asked about their attitudes towards these teaching methods. Allstudents had a positive attitude towards traditional lecturing: 6 “like it” and1 even “love it” (Figure 9.3). Attitudes towards the flipped classroom wasless positive with only 1 student “like it” and 2 students indicating “it’sfine”. As many students (i.e., 3) were neutral towards flipped classroom.Results are even less positive for peer learning where only 1 student hadpositive attitude towards it and 4 were neutral. Finally, 1 or 2 students gaveinconsistent responses when comparing with the previous question: 3 stu-dents indicated they never experienced flipped classroom in Figure 9.2 vs 2students in Figure 9.3 (and 2 students vs 1 student for peer learning). Thiscould indicate that they have a prejudice against both teaching methods.

Following-up on this question, the survey asked them to clarify whythey “feel this way about [teaching method]”. A large majority of studentsgave responses for traditional lecturing that can be summarized as “I like itbecause I’m used to it” or “This is the best method for me to learn”. Onestudent answered:

“Depending on the barrier to asking questions, I think traditional lec-turing (with projects/exploitative elements on the side) is the most efficient,when looking at time spent vs material learned. A professor became a pro-fessor for a reason, so in my opinion you should learn from them, ratherthan from other students.”Another student wrote:


“I like this way of teaching because it works well for me. I think thatwhen trying something else it often fails, and becomes a waste of time.”

Thus, the students in this sample prefer traditional lecturing, because(i) they think they learn best from (passive) listening to a lecture; (ii) Otherteaching methods they experienced before failed and felt like a waste oftime; and (iii) a university teacher is seen as an authority figure that has allthe knowledge to be learnt.

For flipped classroom, one student admitted that it “could actually boostmore our learning experience”, but expressed fear that it would not workthat well if the course material is difficult. Another student wrote: “If thereis enough time to prepare before class, it is probably the best method! Nev-ertheless, it is not always possible to read everything is needed for the class,then it become somehow counterproductive.”

For peer learning, some students wrote that it is very dependent on stu-dent cooperation in the class and therefore can have mixed results. Onestudent expressed feeling insecure that he/she “won’t get it right” whenpresenting. Another acknowledged that presenting to peers contributes to-wards their own understanding of a topic, but found it hard to do becausehe/she lacks “teaching methodology”.

For peer learning, some students wrote that it is very dependent on student cooperation in the class and

therefore can have mixed results. One student expressed feeling insecure that he/she “won’t get it right”

when presenting. Another acknowledged that presenting to peers contributes towards their own

understanding of a topic, but found it hard to do because he/she lacks “teaching methodology”.

Figure 3: Attitude towards different teaching methods in the past.

The final question asked students about the perceived effectiveness towards achieving the

ILOs of these teaching methods (Figure 4). All students perceived traditional lecturing as (somewhat)

effective in helping achieve the ILOs: 6 students indicated that it is “effective” or “very effective” and 2

stated it is “somewhat effective”.

Opinions are more mixed for the flipped classroom, but still quite clear: while 3 students think it is

“effective” and 1 thinks it is “somewhat effective”, 3 students indicated “neutral” and 1 “not really

effective”. Looking into the individual answers in more detail it is interesting to see that students that

actually experienced flipped classroom “occasionally” or “only once” before think it is neutrally effective (1

student), somewhat effective (1 student) or effective (3 students). The student judging flipped classroom as

“not really effective” also “never” experienced it before. Thus, flipped classroom might suffer from some

negative prejudices against it by students.

For peer learning the results seem to indicate that students have had mixed experiences in the past. Of the 5

students perceiving it as “not really effective”, 3 students experienced it “once” or “occasionally” before

while holding a “neutral” or negative (“not at all”) attitude towards it. The 2 remaining students that perceive

it as “not really effective” have never experienced it before with 1 indicating a negative attitude towards it.

The 2 students perceiving it as “somewhat effective”/“effective” have experienced it “occasionally” and have

mildly positive attitude towards it (“neutral” or “it’s fine”).

Fig. 9.3. Attitude towards different teaching methods in the past.

The final question asked students about the perceived effectiveness to-wards achieving the ILOs of these teaching methods (Figure 9.4). All stu-dents perceived traditional lecturing as (somewhat) effective in helping


achieve the ILOs: 6 students indicated that it is “effective” or “very ef-fective” and 2 stated it is “somewhat effective”.

Opinions are more mixed for the flipped classroom, but still quite clear:while 3 students think it is “effective” and 1 thinks it is “somewhat effec-tive”, 3 students indicated “neutral” and 1 “not really effective”. Lookinginto the individual answers in more detail it is interesting to see that stu-dents that actually experienced flipped classroom “occasionally” or “onlyonce” before think it is neutrally effective (1 student), somewhat effective(1 student) or effective (3 students). The student judging flipped classroomas “not really effective” also “never” experienced it before. Thus, flippedclassroom might suffer from some negative prejudices against it by stu-dents.

For peer learning the results seem to indicate that students have hadmixed experiences in the past. Of the 5 students perceiving it as “not re-ally effective”, 3 students experienced it “once” or “occasionally” beforewhile holding a “neutral” or negative (“not at all”) attitude towards it. The2 remaining students that perceive it as “not really effective” have neverexperienced it before with 1 indicating a negative attitude towards it. The 2students perceiving it as “somewhat effective”/“effective” have experiencedit “occasionally” and have mildly positive attitude towards it (“neutral” or“it’s fine”).

Figure 4: Perceived effectiveness of different teaching methods in the past.

The conclusions I draw from these are:

Traditional lecturing is used most of the time, students have positive attitude towards it and perceive it as

a (rather) effective teaching method.

Those students that experienced flipped classroom before perceive it as rather effective and none have a

negative attitude towards it.

Students have had mixed experiences from peer learning with a rather negative attitude towards it.

These less clear-cut answers for peer learning are not really a surprise: peer learning is very dependent on the

students’ participation and a teacher generally has less control over what they get out of it. Flipped classroom

and traditional lecturing are much more teacher controlled.

Post-intervention survey

The basic questions of the post-intervention survey are the same as the pre-intervention

survey. I first present the results of the entire post-intervention survey (10 students) before limiting to the

students that filled in both the pre- and post-intervention survey (6 students).

Figure 5 and 6 present the results for all respondents. Students are generally positive towards

traditional lecturing and see it as quite effective in achieving the ILOs of the course. A majority of students

(7/10) has a positive attitude towards flipped classroom and only 1 student does not like it at all. Further, an

even larger majority of 8/10 students perceives it as (somewhat) effective in achieving the ILOs of this

course. The 2 remaining students perceive it as “not really effective”. Finally, attitudes towards peer learning

are more mixed: 4 students are (somewhat) positive towards it, 4 are neutral and 2 do not like it at all. In

terms of perceived effectiveness, 6 students see it as (somewhat) effective, 1 is neutral and 3 perceive it as

not really effective.

Finally, the post-intervention survey also asked students “Did [teaching method] help you prepare for the

final project?”. For traditional lecturing and flipped classroom students overwhelmingly agreed with this

statement. Only 1 student answered “maybe” for both teaching methods. Opinions are more mixed for peer

learning: 3 students answered “Yes”, 1 “Maybe” and 5 students answered “No”.

Fig. 9.4. Perceived effectiveness of different teaching methods in the past.


The conclusions I draw from these are:

• Traditional lecturing is used most of the time, students have positive at-titude towards it and perceive it as a (rather) effective teaching method.

• Those students that experienced flipped classroom before perceive it asrather effective and none have a negative attitude towards it.

• Students have had mixed experiences from peer learning with a rathernegative attitude towards it.

These less clear-cut answers for peer learning are not really a surprise:peer learning is very dependent on the students’ participation and a teachergenerally has less control over what they get out of it. Flipped classroomand traditional lecturing are much more teacher controlled.

Post-intervention survey

The basic questions of the post-intervention survey are the same as the pre-intervention survey. I first present the results of the entire post-interventionsurvey (10 students) before limiting to the students that filled in both thepre- and post-intervention survey (6 students).

Figure 9.5 and 9.6 present the results for all respondents. Students aregenerally positive towards traditional lecturing and see it as quite effectivein achieving the ILOs of the course. A majority of students (7/10) has apositive attitude towards flipped classroom and only 1 student does not likeit at all. Further, an even larger majority of 8/10 students perceives it as(somewhat) effective in achieving the ILOs of this course. The 2 remainingstudents perceive it as “not really effective”. Finally, attitudes towards peerlearning are more mixed: 4 students are (somewhat) positive towards it, 4are neutral and 2 do not like it at all. In terms of perceived effectiveness, 6students see it as (somewhat) effective, 1 is neutral and 3 perceive it as notreally effective.

Finally, the post-intervention survey also asked students “Did [teachingmethod] help you prepare for the final project?”. For traditional lecturingand flipped classroom students overwhelmingly agreed with this statement.Only 1 student answered “maybe” for both teaching methods. Opinions aremore mixed for peer learning: 3 students answered “Yes”, 1 “Maybe” and5 students answered “No”.


Figure 5: Attitude towards different teaching methods after the course.

Figure 6: Perceived effectiveness of different teaching methods after the course.

Impact of intervention

In order to get an idea of the impact of the intervention, one must compare the results of the

survey for the same students. In both surveys, I asked for the students’ email so that it allows me to match

results of both surveys to the same students. In doing this I can directly compare the results of 6 students

(6/13 ~= 46%).

Figure 7 and 8 summarize the results for these 6 students. The top figure shows the results of

the pre-intervention survey and the bottom figure shows results of the post-intervention survey. For

traditional lecturing, there has not really been much change in attitude and perceived effectiveness: attitude

Fig. 9.5. Attitude towards different teaching methods after the course.

Figure 5: Attitude towards different teaching methods after the course.

Figure 6: Perceived effectiveness of different teaching methods after the course.


In order to get an idea of the impact of the intervention, one must compare the results of the

survey for the same students. In both surveys, I asked for the students’ email so that it allows me to match

results of both surveys to the same students. In doing this I can directly compare the results of 6 students

(6/13 ~= 46%).

Figure 7 and 8 summarize the results for these 6 students. The top figure shows the results of

the pre-intervention survey and the bottom figure shows results of the post-intervention survey. For

traditional lecturing, there has not really been much change in attitude and perceived effectiveness: attitude

Fig. 9.6. Perceived effectiveness of different teaching methods after the course.



In order to get an idea of the impact of the intervention, one must comparethe results of the survey for the same students. In both surveys, I asked forthe students’ email so that it allows me to match results of both surveysto the same students. In doing this I can directly compare the results of 6students (6/13 = 46%).

Figure 9.7 and 9.8 summarize the results for these 6 students. The topfigure shows the results of the pre-intervention survey and the bottom figureshows results of the post-intervention survey. For traditional lecturing, therehas not really been much change in attitude and perceived effectiveness:attitude remains positive and there seems to be some more consensus thatit is “effective” in helping them achieve the ILOs of the course.

The results for flipped classroom are a little bit more interesting: al-though there is not really a shift in attitudes towards flipped classroom, thestudents have a more positive view on its perceived effectiveness (i.e., allstudent responses in Figure 9.8 are “effective” or “somewhat effective”).The 2 students that never experienced it before now answered “it’s fine”.Furthermore, 2 students changed opinion: 1 from “neutral” to “it’s fine”and 1 vice versa.

The biggest shift occurred for peer learning: 1 student now likes it andonly 1 student does “not like it at all” (vs. 2 in the pre-intervention sur-vey). Only 2 students’ attitude remained unchanged: 1 answered “not likeit at all” and 1 answered “neutral” in both surveys. Thus, the interventiongenerally seems to have had a positive impact on the students’ attitude.This shift is even more pronounced for the perceived effectiveness (Figure9.8): whereas in the pre-intervention survey 4 students indicated “not re-ally effective”, only 1 student did in the post-intervention. In fact, this isthe only student whose perceived effectiveness changed negatively for peerlearning (i.e., from “neutral” to “not really effective”). Moreover, in thepost-intervention survey 3 students perceive it as “effective” (vs. 0 studentsin the pre-intervention survey) and 1 student as “somewhat effective” (vs.1 student before).

Thus, I conclude that the most dramatic impact of the intervention hasbeen on (i) the attitude towards and perceived effectiveness of peer learningand (ii) the perceived effectiveness of flipped classroom.


Figure 7: Impact of intervention on attitudes towards different teaching methods.

Fig. 9.7. Impact of intervention on attitudes towards different teaching methods.


Figure 8: Impact of intervention on perceived effectiveness of different teaching methods.

Conclusions This project sought to investigate the attitudes of students towards different teaching methods

(i.e., traditional lecturing, flipped classroom and peer learning) and their perceived effectiveness in helping

them learn (here specified as achieving the ILOs). At the same time I tried to improve the constructive

alignment of the course “Economic efficiency and benchmarking” by introducing 2 flipped classroom

lectures and a voluntary peer-review exercise. The impact of this intervention was recorded through a pre-

intervention and a post-intervention survey. The results show a positive impact on (i) the attitude towards

and perceived effectiveness of peer learning and (ii) the perceived effectiveness of flipped classroom. There

has been no real impact for traditional lecturing.

Fig. 9.8. Impact of intervention on perceived effectiveness of different teachingmethods.

Conclusions

This project sought to investigate the attitudes of students towards differ-ent teaching methods (i.e., traditional lecturing, flipped classroom and peer


learning) and their perceived effectiveness in helping them learn (here spec-ified as achieving the ILOs). At the same time I tried to improve the con-structive alignment of the course “Economic efficiency and benchmark-ing” by introducing 2 flipped classroom lectures and a voluntary peer-review exercise. The impact of this intervention was recorded through apre-intervention and a post-intervention survey. The results show a positiveimpact on (i) the attitude towards and perceived effectiveness of peer learn-ing and (ii) the perceived effectiveness of flipped classroom. There has beenno real impact for traditional lecturing.

Traditional lecturing is most often used, students generally like it andperceive it as an effective tool for their learning. Students are less used toflipped classroom and peer learning. While flipped classroom is generallyperceived as quite effective and students have no negative attitude towardsit, results are more mixed for peer learning. This is probably linked to thefact that what students get out of it is very dependent on the cooperation oftheir peers. Thus, this easily results in mixed experiences.

References

Bishop, J. L., & Verleger, M. A. (2013, June). The flipped classroom: Asurvey of the research. In ASEE national conference proceedings, Atlanta,GA (Vol. 30, No. 9, pp. 1-18).

Butt, A. (2014). Student views on the use of a flipped classroom approach:Evidence from Australia. Business Education & Accreditation, 6(1), 33.

Herreid, C. F., & Schiller, N. A. (2013). Case studies and the flipped class-room. Journal of College Science Teaching, 42(5), 62-66.

Roach, T. (2014). Student perceptions toward flipped learning: New me-thods to increase interaction and active learning in economics. Internationalreview of economics education, 17, 74-84.

Figure 9: Pre-intervention survey questions.


A Pre-intervention survey questions

Figure 9: Pre-intervention survey questions.

10

Transforming an online course to blended andstudent centred learning

Xi Jiao

Department of Food and Resource EconomicsUniversity of Copenhagen

Introduction

Blended learning provides great opportunities to integrate face-to-faceclassroom synchronous verbal and online asynchronous written commu-nication to achieve deep learning through construct meaning and confirmunderstanding through discourse. Garrison and Vaughan (2008) suggestsusing Community of Inquiry (CoI) framework to guide blended learningdesigns, which highlights the importance of interactive (social) and reflec-tive (cognitive) dimensions of higher educational experience. We probablyshould not cover too much content by reading or lectures as too much con-tent may become barriers for deep learning; and most importantly, studentsneed time to process content and reflect upon. It is also found that face-to-face learning experience can enhance connectedness and satisfaction (Con-rad, 2005).

The course “Tropical Forests, People and Policies” is a 7.5 ETCs creditmaster course. It provides an introduction to essential contemporary issuesin forestry in developing countries. It takes a people oriented approach toissues that all natural resources managers in developing countries should befamiliar with. Emphasis is on the relationships between people and forestuse and conservation in developing countries. The aim is to give studentsa thorough understanding of the current and potential role of forests andtrees in improving rural livelihoods. It is stressed that forestry is an integralpart of society and should not be considered in isolation. The course hadbeen a purely online-based course for the past 8 years. Main componentsof the online course include: i) quizzes after reading mandatory material;

122 Xi Jiao

ii) e-tivities for group discussion on specific key topics in the online fo-rum; and iii) tutor marked essay assignments with written feedback andcomments. This year (2018), we plan to transform it into a blended course,which contains both classroom and e-learning components.

Transforming a purely online course to a blended learning that com-bines both face-to-face classroom sessions and online learning activities,the key question would be “how to employ different learning environmentand communication medium to enhance student engaged learning experi-ence”. There are two specific problems to be addressed:

• According to the course evaluation, students feel overloaded with read-ing materials and online activities. How to avoid content overloadingwithout compromising intended learning outcomes?

• Asynchronous written communication in online group discussion e-tivities were quite effective in promoting critical thinking through col-laborative work. However, students may still feel lack of social pres-ence. How to take advantage of face-to-face classroom sessions to pro-mote a stronger sense of community, and interactive and collaborativelearning?

Method

To ensure the effectiveness of the transformed blended learning course inpromote students’ learning (ILOs) and satisfaction, the proposed composi-tion of the blended learning course include:

• Online:1) Self-learning: fewer core readings followed by multiple-choice

questions to test students’ understanding of terminology and con-tents.

2) E-tivities: facilitated group discussions of a key topic in an onlineforum.

3) Teacher marked assignment: individual essay assignment sub-mitted online, detail feedback provided by teacher and uploadedfor students’ access.

• In class:1) Interactive lectures: lectures with interactive student activities

and discussions.

10 Transforming an online course... 123

2) Group exercise: a practical problem oriented group exercise, fol-lowed by group presentation and plenary discussions.

3) Wrap-up session: open and facilitated discussions regarding keypoints learnt in the module, Q&A, and module assessment againstthe weekly ILOs.

Besides assessing the effectiveness of the composition and various ele-ments of blended learning activities, a specific intervention on reading ad-dressing the first problem mentioned above - guided reading questions,was designed, implemented and evaluated. Students were provided a listof key questions for each reading material that aims to guide and engagestudents in active and reflective reading, followed by collaborative groupexercises and interactive discussions in class.

Results and Discussions

A participatory online-based student evaluation for various learning activi-ties with a scale of 0-10 were conducted at end of the last in-class session ofthe course (see results in the figure below), followed by some open discus-sions on specific interventions, particularly on the guided reading questions.

questions for each reading material that aims to guide and engage students in active and reflective

reading, followed by collaborative group exercises and interactive discussions in class.

Results and Discussions

A participatory online-based student evaluation for various learning activities with a scale of 0-10

were conducted at end of the last in-class session of the course (see results in the figure below),

followed by some open discussions on specific interventions, particularly on the guided reading

questions.

The intervention of guided reading questions was rated high, and 5 out of 13 students gave it 10

scores. Students found the guided reading questions particular useful for connecting the key

learning points of a module as well as between the literatures, and one student also mentioned that it

guided the reading process and actually saved time for reading. Effective reading before lecture

ensures active student participation and achieving intended learning outcomes (McGinn and

Schiefelbein, 2015; Hwang and Hsu, 2011; Moravec et al., 2010). However, not all of the students

read all the required material pre-lecture that has been reflected by previous studies (Hiener et al.,

2014; Hobson, 2004). The consequences of this was reflected in the group exercises and discussions

in the class, some students need to read first in class before they engage in any discussions or

Fig. 10.1.

124 Xi Jiao

The intervention of guided reading questions was rated high, and 5out of 13 students gave it 10 scores. Students found the guided readingquestions particular useful for connecting the key learning points of a mod-ule as well as between the literatures, and one student also mentioned thatit guided the reading process and actually saved time for reading. Effec-tive reading before lecture ensures active student participation and achiev-ing intended learning outcomes (McGinn et al., 2015; Hwang et al., 2011;Moravec et al., 2010). However, not all of the students read all the requiredmaterial pre-lecture that has been reflected by previous studies (Hiener etal., 2014; Hobson, 2004). The consequences of this was reflected in thegroup exercises and discussions in the class, some students need to readfirst in class before they engage in any discussions or exercises that eventu-ally prolong the whole process of the group work. Another learning activityreflecting lack of pre-class reading is quiz competition, where quite a fewstudents end up with guessing the answers of the key terminology and theycould not participate in the further discussion afterwards. This lead to inef-fectiveness of classroom learning and demotivation of students. One way ofensuring effective pre-lecture reading is to have pre-lecture activities (e.g.pre-lecture quiz, and online discussion) as implemented by my colleagueSolomon, where he find some positive results. Another approach could behaving informed follow-up learning activities regarding the guided learningquestions or reading material in the classroom sessions.

Classroom discussions was another activity rated high in the evalua-tion, as well as in class exercises. Students were also provided multipleopportunities to engage in discussions regarding the course content andmaterial, and they actively asked questions and acquire teacher’s inputs onparticular content throughout the classroom sessions. And students foundit is quite effective with an interactive wrap-up session with key questions,where students were first asked to discuss and reflect on those questionson their own, followed by inputs and summaries from the teacher. In gen-eral, students were actively participated in the group exercises in class andfollow-up discussions; the most challenging part is that it is often time con-suming for group work, especially if some of the group members did notread required material in advance. Even though it is often get delayed withgroup work or students always complained or demanded for more time,they often found the process of in class group exercises useful and fulfill-ing in achieving ILOs, I believe part of it is also attributed to the socialdimension of the learning environment. One of the particular experimentalactivities was Debate on a key topic in a module, it demonstrated that it can

10 Transforming an online course... 125

effectively activate and engage students in critical thinking and reflectionof key learnings.

Surprisingly, students prefer classroom exercises and discussions ratherthan online based discussion (e-tivities), while asynchronous written com-munication in online group discussion e-tivities was found quite effective inpromoting critical thinking through collaborative work. However, teachermarked assignments (TMA) was ranked the highest, indicating that stu-dents value individual written discourse a necessary and effective approachfor achieving learning outcomes. In contrast, group project (particularly onthe writing project) was rated the lowest (6.1 out of 10), and it’s largely dueto the challenging nature of the subject itself as well as group dynamics.However, different groups managed and performed the assignment quitedifferently and the quality of the final products vary much. This impliesmore facilitation and simplification of the group project is needed.

Conclusion and reflection

The transformation of online course to blended learning is proven to be use-ful and effective. According to the final course evaluation, 26 (of 27, 1 sick)students attended the exam with an average grade of 8.5 (last year: 5.9),ranging from 2-12; failure was 0% (last year 7%). And the course has beenevaluated as Category A course. However, there are still many challengesand room for improvement especially regarding workload and promotingstudent-centred learning. Possible action points for improvement include:

• Develop reading guide for each module including short summary indi-cating the linkages of the readings and guided reading questions (thatis also in a way to reduce the workload for students by facilitating thereading process).

• Explore further with learning activities that excite and activate studentcritical thinking and reflection, such as debate.

• Conduct writing workshop and peer-feedback process for the first writ-ing assignment (TMA1), plus a possible in class discussion and feed-back session on TMA1 after grading.

• Balance the class and online learning activities, considering the overallworkload and learning outcomes, as well as diversity and variation oflearning activities.

126 Xi Jiao

• Utilize after class time more effectively, design some of the group exer-cises across the two class sessions to allow students to get better prepa-ration. (In addition, it is possible to designate Tuesday afternoon 16.00-17.00 for group work, since afternoon class often ends by 16.00.)

• Encourage and facilitate more of the shift from teacher led to student-centred wrap-up discussions progressively, and hopefully students wouldfeel more comfortable to take the leading role and ownership to honetheir learning outcomes.

References

Conrad, D. (2005). Building and maintaining community in cohort-basedonline learning. Journal of Distance Education, 20(1), 1-20.

Garrison, D. R., & Vaughan, N. D. (2008). Blended learning in higher ed-ucation: Framework, principles, and guidelines. John Wiley & Sons.

Heiner, C. E., Banet, A. I., & Wieman, C. (2014). Preparing students forclass: How to get 80% of students reading the textbook before class. Amer-ican Journal of Physics, 82(10), 989-996.

Hobson, E. (2004). Getting students to read: Fourteen tips. Idea Paper No.40, KS: Kansas State University. Center for Faculty Evaluation and Devel-opment.

Hwang, W. Y., & Hsu, G. L. (2011). The Effects of Pre-Reading and Shar-ing Mechanisms on Learning with the Use of Annotations. Turkish OnlineJournal of Educational Technology-TOJET, 10(2), 234-249.

McGinn, N. F., & Schiefelbein, E. (2015). Getting students to read beforeclass: Innovation in a university in Chile. Prospects, 45(4), 447-464.

Moravec, M., Williams, A., Aguilar-Roca, N., & O’Dowd, D. K. (2010).Learn before lecture: a strategy that improves learning outcomes in a largeintroductory biology class. CBE—Life Sciences Education, 9(4), 473-481.

11

Enhancing student research projects with newtechnologies

Darach Watson


Summary. A critical component in supervising thesis students is helping themmaintain focus on the final outcome and continue working consistently for the fullproject duration. However, it can be difficult for a supervisor to gather concrete dataon the progress of a student throughout the project period. Poor outcomes can occurin the middle part of the project as initial excitement flags, complexity grows andthe supervisor is typically busy with other tasks. I report here on a test of a coupledset-up of an integrated rich text, code, and data application, a web-based versioncontrol system, and cloud-based team collaboration tool, designed to improve con-sistency of overview of the progress of thesis students. The tools may also increasethe rapidity and quality of communication and help maintain consistent supervisionand focus for the student over the full project period. I used Jupyter Notebooks,Github, and Slack respectively as a supervision tool with a Master’s student over afull year. Quantitative data from these tools are used to analyse how well the moni-toring set up worked. The technical set up works well. There is an initial investmentcost for the student to learn the various tools and from the supervisor in ensuring thestudent gets used to using them consistently and correctly. However, the overall ben-efits appear to be significant in terms of the supervisor’s overview of the student’sprogress, leading to potentially greater consistency of supervision. Secondary ef-fects were also positive, with significant development of transferable skills such assoftware coding ability and team-working. Modifications were made to improve theuse of the set up for the following year and the initial data on a student in the nextyear are provided.

128 Darach Watson

Introduction

Master’s students at research-intensive institutes, or going on to research-related careers will often do a year-long Master’s thesis based on originalresearch. Long theses are particularly prone to loss of focus and motivation(Litalien et al., 2015), as well as feature- and content-creep. Without reso-lution, the thesis can be severely delayed, or, ultimately, never completed(Bair et al., 2005), both of which have serious negative consequences forboth the student and the hosting department (Wendler et al., 2010).

The same problems can occur in any thesis work, and a significant issuefor any supervisor is keeping track of the progress of a student, in particulara student pursuing research work, where there is no well-defined curricu-lum. One solution to this problem is to curtail the duration and ambition ofthe thesis project. However, for the ambitious student or supervisor, suchsolutions are either sub-optimal or simply not acceptable, or limits the typeof projects that can be undertaken.

The other solution is to increase the amount or quality of feedback andengagement (Price et al., 2011) either through group work or directly fromthe supervisor. Research undertaken in a group or as a team can help mit-igate the problems of oversight and motivation and focus loss (Hertel etal., 2000), but such group working may not be available in every situationor type of research. On the other hand, in an ambitious research milieu,significantly increasing the amount of supervision time in a simple way isnot typically viable, may not be necessary or desirable where the studentis doing well and beginning to behave independently, and is certainly not ascalable solution. It may, on the other hand, be possible to use a technolog-ical solution to keep tabs on the progress of a student and flag a drop-offin activity, as well as increase the speed, quality, and quantity of communi-cation with a student, without substantially increasing the workload on thesupervisor.

In this paper, I examine the integrated use of tools for recorded, inter-active data analysis and coding, web-based sharing and versioning control,and collaboration work, to keep track of the progress of thesis projects, andcommunicate effectively and quickly with research students in the physicalsciences.

11 Enhancing student research projects with new technologies 129

Method

I supervised two Master’s students in consecutive academic years usingthe following communication methodology. Meetings were arranged twiceevery week, where this was possible, lasting 30 to 60 minutes. The studentswere requested to do coding and data analysis using Python primarily withthe Numpy, Scipy, Matplotlib and Astropy libraries in Jupyter Notebooksand to keep notes of their work and reading in the same format. They shouldthen use the Git software and the Github website to maintain versions oftheir notes, code, and analysis that are shared with their supervisor. Finally,their Github repository is integrated with the Slack team communicationsoftware, which is used as the primary electronic communication tool. Anyqueries, concerns, or brief explanations can be requested through Slack, andnotifications of updates to the student’s work are made automatically to theSlack channel associated with the project. The software above is availablefor free. Advanced features are available from Github and Slack with a paidaccount.

Jupyter Notebooks1

Jupyter Notebooks is a format for writing code and doing data analysisprimarily in the languages Julia, R, and Python. My students were usingPython. Jupyter Notebooks works as an interactive coding and data analy-sis environment running in a web browser that executes blocks of code inindividual cells, allowing an interactive environment coupled with a recordof the code, so that it can be altered and tested on the fly. It also has thecapability to produce and include plots (typically using a library like Mat-plotlib), and to include notes written in Markdown or LATEX, with images.Jupyter Notebooks therefore enables the recording of a really comprehen-sive and comprehensible record of data analysis, code testing, and researchnotes. Part of a Jupyter Notebooks is shown in figure 11.1.

Github2

Git is a version control system, recording in a repository each version, thechanges from the previous version, and a note of what the changes were.

1 See https://jupyter.org for more details.2 See https://github.com

130 Darach Watson

Github is a website that acts as a remote storage for Git repositories. Theserepositories can be shared and changes made cooperatively in a team asrequired.

Slack3

The team collaboration software Slack is widely used and has many inte-grations with other software systems. Here a channel was set up for theproject and all communications related to the project were made on thischannel, providing a record of conversations, advice, and feedback, as wellas permitting rapid interchange on small questions. The Github plugin forSlack was used, which sends a notification and a link to the relevant chan-nel automatically when a new version was uploaded to the repository. Partof a Slack session is shown in figure 11.2.

Results

The students with whom this method was trialled, B, and D, were instructedin the approach at the first meeting before they selected their supervisor.Both were happy with the approach. Neither had significant coding experi-ence, with very little Python expertise and no experience with the softwaretools used above.

3 See https://slack.com


Figure 1: Section of a Jupyter notebook on Github from student B.

4

Fig. 11.1. Section of a Jupyter notebook on Github from student B.

132 Darach Watson

Figure 2: Slack session with student B.

5

Fig. 11.2. Slack session with student B.


Table 11.1. Students’ Github commit days, commits with useful comments, andSlack messages.

Table 1: Students’ Github commit days, commits with useful comments, and Slack messages.

Student B Student D

Project No. Github (commented No. Slack No. Github (commented No. SlackMonth commit dates commits) messages commit dates commits) messages

1 0 (0) 7 0 (0) 12 0 (0) 8 6 (6) 33 9 (0) 22 3 (1) 04 7 (0) 4 – – –5 2 (0) 3 – – –6 1 (0) 1 – – –7 2 (0) 14 – – –8 1 (0) 3 – – –9 2 (0) 12 – – –10 1 (0) 11 – – –11 1 (0) 5 – – –12 7 (0) 25 – – –13 1 (0) 47 – – –

3.1 Observations

Activity can be monitored using the number of Github commits (a commitsaves a snapshot of the current document(s) or code to the repository),avoiding some of the problems of subjective impressions. As seen from thedata in table 1, B got properly started with Github in two months afterstarting the thesis project. Commits were made frequently in the earlymonths, with fewer in the middle period, and then many more in the lastmonth. The numbers of Slack messages from B followed a similar pattern.

This strongly suggests that after a strong start there was a lull in activityor recording of activity and communication after the first semester, with asimilar number of communications and 80% more Github commits in thelast four months of 2017 compared to the first seven months of 2018. Thiswas followed by a strong surge in activity in both Github commits and Slack

messages in the last six weeks of the project.My own observations agree with the quantitative data, that student B’s

progress slowed considerably after the first four months and only reallypicked up again at the end of the year. The reason for this appeared tobe that B became lost in the fine detail of analysing the data. While at thetime each incremental step seemed necessary and plausible, looking at thedata, it seems that there is a case to be made for a more vigorous supervisionintervention.

The answer lag time is a key parameter in the responsiveness of a su-

6

Observations

Activity can be monitored using the number of Github commits (a com-mit saves a snapshot of the current document(s) or code to the repository),avoiding some of the problems of subjective impressions. As seen from thedata in table 1, B got properly started with Github in two months after star-ting the thesis project. Commits were made frequently in the early months,with fewer in the middle period, and then many more in the last month. Thenumbers of Slack messages from B followed a similar pattern.

This strongly suggests that after a strong start there was a lull in activityor recording of activity and communication after the first semester, with asimilar number of communications and 80% more Github commits in thelast four months of 2017 compared to the first seven months of 2018. Thiswas followed by a strong surge in activity in both Github commits and Slackmessages in the last six weeks of the project.

My own observations agree with the quantitative data, that student B’sprogress slowed considerably after the first four months and only reallypicked up again at the end of the year. The reason for this appeared to bethat B became lost in the fine detail of analysing the data. While at the timeeach incremental step seemed necessary and plausible, looking at the data,it seems that there is a case to be made for a more vigorous supervisionintervention.

134 Darach Watson

103

102

101

100

101

102

Time to answer (hours)

0

2

4

6

8

10

12

14

Num

ber o

f rep

lies

Fig. 11.3. Time to rst reply to a student query on Slack.

The answer lag time is a key parameter in the responsiveness of a su-pervisor. We can measure that parameter on Slack by looking at the timebetween requests by the student and answers from the supervisor. Thesedata are shown for student B in figure 11.3. The data are plotted on a loga-rithmic scale, showing a fairly even response per decade. More than 90% ofresponses are within 24 hours. However, the median response time is only5.8 minutes.

Other outcomes of using the system were that students B and D becamemoderately proficient with Python programming and version control tools.

Discussion

The commit and Slack chat frequency data show clearly a drop-off in activ-ity in the middle of the project period as anticipated from previous studentsand consistent with direct observation. This indicates that using this com-bined technique can be successful in monitoring the progress of studentsdoing work without a well-defined curriculum.


These observations show that using this technological approach withoutstrong control of the process did not entirely correct the problem of a loss offocus and motivation in the middle of the project for student B. Monitoringthe number of Github commits and the Slack communication statistics overtimescales of weeks or months could allow the supervisor to intervene tocorrect the motivation and focus of the student.

The lag time analysis indicates that using a communications platformlike Slack does offer very fast responses to student queries. This may be dueto the direct notifications to mobile phone that Slack offers via the mobileapp, the relatively informal method of communicating, encouraging quick,short questions, with a chat-like interface, as well as the easy inclusion ofplots and data and integration with Github.

This approach was trialled on students working with the analysis ofphysical science data and theoretical models. Jupyter Notebooks is particu-larly well suited to this type of work, and Github is designed especially forcomputer code versioning. However, there is no obvious reason why thisapproach would not work well in areas that do not require data analysistools. Jupyter Notebooks is certainly very capable of fully marked up notesincluding figures. And Github can maintain versions of files even in binaryformats, though it works best with plain text formats. However, the methodoutlined here: work-recording software integrated with a shared versioningsystem and a team communications software, is not exclusive to the soft-ware used here and could be used with other tools. There seems no obviousbarrier to using this technology in areas requiring lab work, interviews,fieldwork, or literature review. However, certainly other tools exist someof which allow for document preparation and version control together. Oneof the easiest and most robust is Google Docs4, which automatically han-dles versioning cleanly and elegantly and might be suitable in less technicalsituations.

It was expected that using the approach outlined in this paper would alsoimprove familiarity with coding in Python, with structuring code well, withversioning, and with the basic mechanics and sharing mindset required forreally effective collaboration. These aims were achieved with the exceptionof well-structured code.

4 See https://docs.google.com

136 Darach Watson

Adaptations

While this methodology makes insisting on well-structured code and keep-ing clear, comprehensive notes straightforward, this was not achieved verysuccessfully with student B. These issues led to a few problems during theresearch project that this method is designed to solve. My current conclu-sion is that tighter control of and better feedback to the early versions of thestudent’s commits would solve this problem. This conclusion will be testedin the future.

The following adaptations have so far been made to the programme.First, the use of Github for version control was insisted on early. This tooksome additional supervision effort, however, the first commit was a monthearlier than the previous year. Second, note-taking using Jupyter Notebookswas also insisted on, resulting in early use of Jupyter Notebooks and exten-sive, detailed notes being posted, which was not the case the previous yearand had been at least partly responsible for the rushed end period of thatproject. Fewer commits have so far been made and fewer lines of code sub-mitted, however, with student D. I believe this is largely due to less manualversioning done by the student, a positive development in coding practice.

Conclusions

In this paper I have laid out a technical framework to monitor the progressof students undertaking research projects, to maintain student focus over thecourse of the project, and to increase the speed and quality of communica-tion. The framework was successful in keeping track of how the studentswork developed and increased the rapidity of communication. So far, themethod has not been completely successful in maintaining focus over thefull project period or in improving the quality of code structuring. Witha greater focus on the feedback of early commits, coupled with monthlymonitoring the commit and communication statistics, these goals may alsobe achieved.

Acknowledgements

I would like to thank Anja C. Andersen and Camilla Østerberg Rump foruseful feedback on this paper.


References

Bair, C. R., & Haworth, J. G. (2005). Doctoral student attrition and per-sistence: A meta-synthesis of research. In Higher education: Handbook oftheory and research (pp. 481-534). Springer, Dordrecht.

Hertel, G., Kerr, N. L., & Messé, L. A. (2000). Motivation gains in perfor-mance groups: Paradigmatic and theoretical developments on the Köhlereffect. Journal of personality and social psychology, 79(4), 580.

Litalien, D., Guay, F., & Morin, A. J. (2015). Motivation for PhD studies:Scale development and validation. Learning and Individual Differences, 41,1-13.

Price, M., Handley, K., & Millar, J. (2011). Feedback: Focusing attentionon engagement. Studies in higher education, 36(8), 879-896.

Wendler, C., Bridgeman, B., Cline, F., Millett, C., Rock, J., Bell, N., &McAllister, P. (2010). The path forward: The future of graduate educationin the United States. Educational Testing Service.

12

Can the flipped classroom enhance studentlearning?

Melanie Ganz-Benjaminsen

Department of Computer ScienceUniversity of Copenhagen

Introduction

The flipped classroom (FC) is an interesting teaching tool and its usersclaim they can engage their students more, increase the students’ learningopportunities and get instant feedback on their own teaching at the sametime. An in depth treatment on the rise of FC as well as the theoreticalbackground is given in Bishop et al., 2013. The authors also highlight thekey concept behind FC as in being pre-recorded lectures or reading materialthat can be assigned to students as homework, leaving class time open forinteractive learning activities—activities that cannot be automated or com-puterized. Specifically, Bishop et al., 2013, define FC as interactive grouplearning activities inside the classroom, and direct computer-based indivi-dual instruction outside the classroom. While Bishop et al., 2013, in theiranalysis strictly exclude designs that do not employ videos as an outsideof the classroom activity, in general assigning reading outside of class iscommonly employed in FC and will be employed in our study as well.

A front figure for FC at the University of Copenhagen is ProfessorJan H. Jensen from the department of chemistry (Jensen, 2014). Profes-sor Jensen’s approach (Jensen, 2015) lets the students acquire knowledgebefore they come to class by watching videos or reading a book and checksthe knowledge they have acquired by performing a quiz at home. The lec-ture then serves as a question and answer session where he discusses theissues/points that most of the students missed. He picks up a common ques-tion/misconception and starts clarifying the issue but first lets the studentstry to understand the issue by discussing it with their neighbors (peer in-

140 Melanie Ganz-Benjaminsen

struction). He then asks questions via a student-response system (SRS). TheSRS can be used to ask the students a question, to receive their answers inreal time by smartphones, tablet or web browsers and see their responseslive in class. After students have discussed for a short while, he lets themvote on the issue to check if the knowledge was acquired correctly. If morethan, 75% of the students get the answer correct, he moves on to the nextissue. If less than 75% but more than 40% give the correct answer, he dis-cusses further and lets them vote again. If less than 40% of the students getthe answer right, he then gives an additional short lecture on the topic to en-sure the correct understanding of the issue. According to Professor Jensen’sexperience (Jensen, 2014) this type of teaching format and the use of SRScan counteract the one-way communication of traditional lectures and pro-vide a break every now and then, which is necessary to maintain studentconcentration. Furthermore, SRS can stimulate class discussions and peer-to-peer instructions as the students become active participants in their ownlearning. A nice feature is that the teaching format also works well for largeclasses, but of course the Q&A session needs to be better prepared if theclass is larger.

In this work, FC is applied in the course Medical Image Analysis. Afew lectures are changed from “knowledge-dissemination” lectures to FClectures that support a more active learning approach. The goal of this isto check if student learning is enhanced after the use of FC. This is evalu-ated by two quantitative quizzes before and after the lectures as well as aqualitative interview afterwards.

Background

The course used for this experiment is Medical Image Analysis (MIA,2017) at the Department of Computer Science of the University of Copen-hagen. The course is a 7,5 ECTS course running in block 1 and one ofthe restricted elective courses in the Master of Science (MSc) program inComputer Science. Furthermore, the course is also an elective course in theMSc in Physics program for the study area Medical Physics. The teachingformat mentioned in the course description consists of lectures, exercises,and assignments. In practice, this results in two lectures (Monday and Fri-day morning) of about 2 hours each accompanied by one afternoon session(Tuesday) with a teacher present, where the students can get help with theassignments. The expectation that is communicated at the start of the course

12 Can the flipped classroom enhance student learning? 141

is that the students should read the relevant parts of the course book beforethe lecture, and hence the “lectures” should be quite interactive. Unfortu-nately, even though the intention of the course is to be very interactive,the mere request to have read the relevant chapters in the book before thelecture, does not lead the students to do this. This in turn often leads toa one-way lecture that might sometimes be interspersed with study groupwork, but where the students mostly only sit and listen. While the course ingeneral receives good evaluations, it seems the students lack the ability totransfer the learned knowledge to a new context such as project work afterthe course. This is a problem for the master students attending the course,since they should be able to transfer what they have learned in the course toresearch or Master projects that they conduct in the department of computerscience.

In the interest of congruency and to avoid confusing the students, FCis tested in two lectures. The first is a short (20 minute) expert lecture ona freely chosen topic by the lecturer. The second lecture is a full lecture (2hours) and covers the topic of basic medical statistics. This lecture is largelyindependent of the other content covered in the course and therefore offersthe possibility to test FC in an isolated fashion.

Fig. 12.1. An overview of the experimental setup. Lecture 1 and 2 were given oneweek apart. In between, the students had to fill out a quiz on the Absalon platformthat checked their knowledge about basic statistics. After the second lecture, theywere asked to take the same quiz again. Finally, a student was interviewed regardingher experience of the flipped classroom approach.


Methods

The experimental setup to test if FC improves student learning was plannedand discussed together with Camilla Østerberg beforehand. The study setupconsisted of two lectures interspersed with quantitative quizzes and com-pleted by a qualitative interview. An overview of the experimental setup isshown in Figure 12.1.

The first lecture introduced the students to the lecturer and to a simpli-fied version of FC. The topic of the lecture was an aggression project car-ried out at the Neurobiology Research Unit at Rigshospitalet (www.nru.dk).The students received a scientific article (da Cunha-Bang et al., 2017) de-scribing the project as well as a link to a documentary about the project byDanmarks Radio [Vold på hjernen] beforehand. Small quizzes and feedbacksession were used throughout the lecture in order to introduce the studentsto a flipped classroom type of teaching approach as well as to acquaintthe students with the student-response system (SRS) used in the lectures,Shakespeak (https://www.shakespeak.com/). The first lecture was also usedto get demographic information about the students.

After the first lecture, the students were assigned a quiz via Absalon thatchecked their knowledge about basic statistical tests. The quiz consisted often multiple-choice questions. A full list of all questions and answers can befound in the supplementary material. There was no time limit on the quiz,the correct answers were not accessible to the students afterwards and theyhad only one attempt at performing the quiz. The questions were shownone at a time and shuffling of the questions not possible. The students hadto hand-in the quiz five days after the first lecture and two days beforethe second lecture. Additionally, the students were given literature (VanEmden, 2012) to refresh their knowledge if they came upon areas of thequiz where they felt a need to do so.

The second lecture, given one week after the first, introduced the stu-dents to basic statistical tests used in medical imaging. The lecture con-sisted of a very short introduction of theory and then a series of exampleproblems where the students had to vote which statistical test would beappropriate for the example problem. After each small quiz, the exampleproblems were discussed and solved together in class. The second half ofthe lecture consisted of interactive group work in groups of four to fivestudents where the students designed flowcharts guiding them to the appro-priate statistical test given some example data. Finally, the students applied


their flowcharts to a final problem that had been introduced at the very be-ginning of the lecture.

After the second lecture, the students had to complete another quiz,which was identical with the pre-lecture quiz and again checked their know-ledge.

Statistical significance of improvement between quiz 1 and 2 was testedwith a non-parametric one-sided (left-tailed) Mann Whitney U-test due toa lack of normality of the scores.

Results

Basic demographic information about the students was collected in the firstlecture. Based on the student population from previous years, the studentswere asked to identify their main field of study between computer science,math, physics or other fields. In the current class the students main fieldof studies were computer science (24%) and physics (4%) as well as otherfields (72%). The students from the other field were mainly students inbioinformatics. The students were also asked to identify their seniority andthe majority of students was in the second year of their Master of Scienceeducation (83.3%) whereas a few first year (8.3%) and bachelor studentscomprised the rest of the class (8.3%).

Next, the quantitative results based on the first and second ten-questionquiz on Absalon will be described. Thirty-three students filled out the pre-lecture quiz on Absalon and all of them completed it before the flippedclassroom lecture. The average student performance was a score of 7.76 ±0.29 out of 10. Thirty-four students completed the post-lecture quiz on Ab-salon. Three students completed it right after the lecture, while the majorityof class submitted the second quiz within the following days. The averagestudent performance on the second quiz was 8.36 ± 0.27 out of 10.

The quiz can further be analyzed in two fashions. Either looking at theindividual students or looking at the individual questions in the pre- andpost-lecture versions of the quiz. Thirty students filled out both the pre-and post-lecture quizzes. When comparing the individual students’ scorespre- and post-lecture, they show a significant improvement after the lecture(p<0.05). Means and standard deviations of the two quizzes as well as theoutcome of the statistical test are shown in Table 12.1. When examining theten different items in the quiz, the students improved their performance onaverage by 11% for seven questions, while they declined in performance


on average by 5% for three questions. The largest improvement was onquestion 7 where correct answers increased by 30%. As example, the threequestions with the largest improvements are shown in Table 12.2.

Table 12.1. An overview of the student performance in terms of the average stu-dent score and its standard deviation in the pre- and post-lecture quiz on Absalon.Statistical significance of improvement between quiz 1 and 2 was assessed with anon-parametric one-sided (left-tailed) Mann Whitney U-test and yielded a signifi-cant improvement.

4

of studies were computer science (24%) and physics (4%) as well as other fields (72%). The students

from the other field were mainly students in bioinformatics. The students were also asked to identify

their seniority and the majority of students was in the second year of their Master of Science education

(83.3%) whereas a few first year (8.3%) and bachelor students comprised the rest of the class (8.3%).

Next, the quantitative results based on the first and second ten-question quiz on Absalon will be

described. Thirty-three students filled out the pre-lecture quiz on Absalon and all of them completed

it before the flipped classroom lecture. The average student performance was a score of 7.76 ± 0.29

out of 10. Thirty-four students completed the post-lecture quiz on Absalon. Three students completed

it right after the lecture, while the majority of class submitted the second quiz within the following

days. The average student performance on the second quiz was 8.36 ± 0.27 out of 10.

The quiz can further be analyzed in two fashions. Either looking at the individual students or looking

at the individual questions in the pre- and post-lecture versions of the quiz. Thirty students filled out

both the pre- and post-lecture quizzes. When comparing the individual students’ scores pre- and post-

lecture, they show a significant improvement after the lecture (p<0.05). Means and standard

deviations of the two quizzes as well as the outcome of the statistical test are shown in Table 1. When

examining the ten different items in the quiz, the students improved their performance on average by

11% for seven questions, while they declined in performance on average by 5% for three questions.

The largest improvement was on question 7 where correct answers increased by 30%. As example,

the three questions with the largest improvements are shown in Table 2.

Table 1. An overview of the student performance in terms of the average student score and its

standard deviation in the pre- and post-lecture quiz on Absalon. Statistical significance of

improvement between quiz 1 and 2 was assessed with a non-parametric one-sided (left-tailed) Mann

Whitney U-test and yielded a significant improvement.

Quiz 1 Quiz 2 p-value of one-sided

Mann Whitney U-test

Student

Performance (mean

± std)

7.77 ± 0.31 8.57 ± 0.27 0.033

Table 2. An overview of the three (out of ten) items where the correct student answers improved the

most between the pre- and post-lecture quiz.

Quiz 1

–

Correct

answers

Quiz 2 –

Correct

answers

Improvement

in correct

answers

”What is an example of a statistical test I could use if I have

non-normal distributed data?”

67 % 97 % 30 %

”You are a high school teacher interested in how your

students compare to others in college readiness. The

55 % 68 % 13 %

Table 12.2. An overview of the three (out of ten) items where the correct studentanswers improved the most between the pre- and post-lecture quiz.

5

students grades are normally distributed around an average

that we call mu. You compare their SAT scores to the

national average of 500. What of the following statements

is true?”

”My friend, Bob, believes that his supermarket's prices are

lower than mine. We construct a list of identical items and

purchase them at our respective stores every week for two

months. We realize that our items' prices are normally

distributed. Then Bob wants to know if his hypothesis is

supported. What should he use?”

63 % 79 % 16 %

While the quantitative (the two quizzes) as well as the qualitative (the interview) results indicate that

student learning improved, this needs to be interpreted cautiously. The most severe limitation of

drawing strong conclusions is in the single arm design of our study. A randomized design as e.g.

presented in [Wozny 2018] where half the students experience a traditional lecture, while the other

half are exposed to FC would clearly have been preferable. This was unfortunately not possible due

to the small number of students in this course (approx. 30). In general, the small sample size is also a

limitation for the validity of our statistical finding. While we try to mitigate the small sample size by

using non-parametric tests, a larger sample would allow stronger conclusions.

Furthermore, it is not clear if the positive learning effect and students positive perception stems

directly from FC or simply from an active learning approach. In a recent review of the literature across

higher education [O'Flaherty 2015] noted that much of the emerging evidence of improved academic

performance might be indirectly related to FC. For example, [Jensen 2015] compared a flipped and

non-flipped approach and found no significant difference between either approaches when both

groups employed active learning techniques in class. Again, our small study cannot address this issue.

Indirectly, there are though some clear benefits of FC that became apparent in our study. First, since

the class size was small and the number of students known, the lecturer could ensure that all students

participate in the lecture quizzes by simply waiting for the same number of votes to come in as there

were students present. This ”forces” the students to participate. Even though this doesn’t ensure that

they participate actively, it makes it more likely that they do. Second, the group work in the second

half of the statistics lecture engaged the students in a different fashion. They not only had to answer

Quiz 1

–

Correct

answers

Quiz 2 –

Correct

answers

Improvement

in correct

answers

”What is an example of a statistical test I could use if I have

non-normal distributed data?”

67 % 97 % 30 %

”You are a high school teacher interested in how your

students compare to others in college readiness. The

55 % 68 % 13 %

Finally, we will describe the results based on a qualitative interviewwith a student. Four students were approached for an interview, but onlyone student could make the time for it. During the interview the student was


asked if she preferred the more active type of lesson exhibited in the statis-tics lecture that involves group work, if she had a better learning experienceby being forced to use the mathematical concepts in examples/quizzes rightaway and finally if she had experienced this type of active learning or FCbefore. The student answered that she definitely preferred the more activetype of lesson. She also liked to use the concepts right away to ensure sheunderstood them and could apply the newly acquired knowledge. The stu-dent has had classes using FC before and mentioned that she felt bettersuited to this type of learning than passive lectures.

Discussion

While the quantitative (the two quizzes) as well as the qualitative (the in-terview) results indicate that student learning improved, this needs to beinterpreted cautiously. The most severe limitation of drawing strong con-clusions is in the single arm design of our study. A randomized design ase.g. presented in Wozny, 2018, where half the students experience a tradi-tional lecture, while the other half are exposed to FC would clearly havebeen preferable. This was unfortunately not possible due to the small num-ber of students in this course (approx. 30). In general, the small sample sizeis also a limitation for the validity of our statistical finding. While we tryto mitigate the small sample size by using non-parametric tests, a largersample would allow stronger conclusions.

Furthermore, it is not clear if the positive learning effect and studentspositive perception stems directly from FC or simply from an active learn-ing approach. In a recent review of the literature across higher educationO’Flaherty et al., 2015, noted that much of the emerging evidence of im-proved academic performance might be indirectly related to FC. For ex-ample, Jensen et al., 2015, compared a flipped and non-flipped approachand found no significant difference between either approaches when bothgroups employed active learning techniques in class. Again, our small studycannot address this issue.

Indirectly, there are though some clear benefits of FC that became ap-parent in our study. First, since the class size was small and the number ofstudents known, the lecturer could ensure that all students participate in thelecture quizzes by simply waiting for the same number of votes to comein as there were students present. This ”forces” the students to participate.Even though this doesn’t ensure that they participate actively, it makes it


more likely that they do. Second, the group work in the second half of thestatistics lecture engaged the students in a different fashion. They not onlyhad to answer pre-defined questions, but also had to design a flow chartthemselves. This allowed them to be creative and think about the topic athand in an alternative fashion which should also support knowledge reten-tion. Finally, after both lectures five students approached me and asked forproject or thesis topic work. By now it is clear that two students will begina master project under my supervision in spring 2019. Two master studentprojects from a mere contact of two lectures seems also to show that thelectures were interesting. Whether the students interest in project work wasinspired by the topic of the lectures or the lecture style is unknown.

Conclusion

The experience with testing a flipped classroom approach was thoroughlypositive and is encouraging me to re-design other parts of the course tosupport either FC or at the least an active learning approach.

For future work, other lecturers at computer science will be presentedwith our results to stimulate them to try out FC in their courses, since manyof the courses at the department could be re-designed to follow an activelearning approach. Overall, I gained a lot of experience with modern teach-ing methods during the course on teaching in higher education and thisexperiment and am hoping to apply that knowledge in my future teachingendeavors.

Acknowledgements

The author is grateful to Camilla Østerberg, Pieter Jan Kerstens and MartinNørgaard for helpful hints and valuable suggestions.

References

Bishop, J. L., & Verleger, M. A. (2013, June). The flipped classroom: Asurvey of the research. In ASEE national conference proceedings, Atlanta,GA (Vol. 30, No. 9, pp. 1-18).

da Cunha-Bang, S., Hjordt, L. V., Perfalk, E., Beliveau, V., Bock, C., Lehel,S., ... & Knudsen, G. M. (2017). Serotonin 1B receptor binding is associated


with trait anger and level of psychopathy in violent offenders. Biologicalpsychiatry, 82(4), 267-274.

Jensen, J.H. (2014). I lecture no more. Keynote speech at “Make a differ-ence - Teach and learn with technology 2014”, Copenhagen 12 November2014.

Jensen, J. H. (2015). My flipped classroom: what I did and how I did it (No.e1546). PeerJ PrePrints.

Jensen, J. L., Kummer, T. A., & Godoy, P. D. D. M. (2015). Improve-ments from a flipped classroom may simply be the fruits of active learning.CBE—Life Sciences Education, 14(1), ar5.

O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms inhigher education: A scoping review. The internet and higher education,25, 85-95.

MIA (2017). Medical Image Analysis, NDAK10005U, http://kurser.ku.dk/course/ndak10005u.

Van Emden, H. F. (2012). Statistics for terrified biologists. John Wiley &Sons.

Vold på hjernen (2017), https://www.dr.dk/tv/se/kriminel-hjerne/vold-pa-hjernen/vold-pa-hjernen

Wozny, N., Balser, C., & Ives, D. (2018). Evaluating the flipped classroom:A randomized controlled trial. The Journal of Economic Education, 49(2),115-129.

Supplementary material to “Can the flipped classroom enhance student learning?”

Melanie Ganz-Benjaminsen

Department of Computer Science, University of Copenhagen

Quiz questions

1) Statistical hypothesis tests can be used to…

distinguish real effects from random variation True make sure my paper gets published False prove my hypothesis is true False

2) Statistical significance, or the probability of finding statistical significance is also known as

p-value True standard deviation False degrees of freedom False

3) What is the name of a common statistical test?

t-test True q-test False r-test False s-test False

4) A t-test is a significance test that assesses

The means of two independent groups True The medians of two independent groups False The standard deviation of three independent variables

False

The modes of two independent variables False

5) What should be true of your data before using a t-test?

normally distributed True skewed False two-tailed False uniformly distributed False


A

7) What is an example of a statistical test I could use if I have non-normally distributed data?

Mann-Whitney test True t-test False Delta test False

8) You are a high school teacher interested in how your students compare to others in collegereadiness. The students grades are normally distributed around an average that we call mu.You compare their SAT scores to the national average of 500. What of the followingstatements is true?

Your null hypothesis is mu = 500. True Your null hypothesis is mu is not equal to 500. False Your alternative hypothesis is mu > 500. False

9) My friend, Bob, believes that his supermarket's prices are lower than mine. We construct alist of identical items and purchase them at our respective stores every week for two months.We realize that our items' prices are normally distributed. Then Bob wants to know if hishypothesis is supported. What should he use?

Bob has a directional hypothesis and should do a paired t-test.

True

Bob has a directional hypothesis and should do an unpaired t-test.

False

Bob has a non-directional hypothesis and should do an unpaired t-test.

False

Bob has a non-directional hypothesis and should do a paired t-test.

False

10) T-tests and other significance tests are frequently criticized. Overrepresentation of statisticalsignificance in research may result in…

Publication bias True Confused students False Researcher fatigue False Lost funding False

6) What is a common type of t-test?

One-sample t-test True Variable t-test False Null hypothesis t-test False

Part IV

Course (re)design and constructive alignment

13

Course alignment: Application and revision of amethod to align a course

Gabriela Alandia Robles

Department of Plant and Environmental SciencesUniversity of Copenhagen

Introduction: the importance of the course alignment -Brief theoretical background

The course alignment is important to set up the desired outcomes of teach-ing in terms of contents and also level of understanding that we want stu-dents to achieve (Biggs, 2003). The course alignment should use the ob-jectives or learning outcomes of the course as the driving force to planteaching, learning and assessment activities. The course alignment shouldhelp to guide students with a clear direction and to improve learning expe-riences. Considering constructive alignment, a good structure of the coursedepends on consistency between the objectives, content, learning activitiesand assessment methods. It can constitute a good tool of communicationwith students to clarify expectations, reduce anxiety and give more senseof the course into the program (Biggs, 2014; Blumberg, 2009).

Context of application

The experiment took place in a MSc course of 7.5 credits of the EuropeanSystem (ECTS) from the Department of Plant and Environmental Sciences(PLEN) of the University of Copenhagen. The course name is TropicalCrop Production (5440-B1-1E18) held for three months; every year fromSeptember until November. This course focuses different aspects of tropicalcrop production such as the properties, management of major tropical cropspecies; production constraints (biotic/abiotic stress); and optimization of

154 Gabriela Alandia Robles

small production systems. The course is given by different teachers (7 to 13teachers between 2013 and 2018) that hold expertise in the region and withthe topics of the course plan. Between 2013 and 2017, this course had from12 to 40 students with different background.

The problem

For three years (2015 - 2017) the lack structure of this course has beenan aspect coming out as an issue in the student evaluations of the course.Some comments about this matter are summarized in the following table toillustrate the problem:

13 Aligning a course 155

Table 13.1. Comments from students about the main problem in the Tropical CropProduction course between 2015 - 2017

Table 1 Comments from students about the main problem in the Tropical Crop Production course between 2015 – 2017

Comment to the evaluation question:

Suggestions for improvement?

Student

evaluation

year/

Evaluation

results

Element to target

“The course is simply unstructured. There is no

system in the order of topics, it is a complete mess

in its current form. It is possible that these are the

most important topics to cover, however the

lectures has to follow each other in a much much

more logical way.”

“Increased organization in the course objectives

and lecture topics. There was a lot of jumping

around from plant physiology topics to farming

system topics, making it difficult to make

connections.”

2015/

75%

positive

final

evaluation

of the

course

Lack of structure and need

of more logical order. More

organization and

connection of objectives

and lecture topics.

“This is probably one of the most disorganized

courses I have ever taken. We have had several

different lecturers in random orders with random

topics.”

“The different course components felt very

disconnected to the overall aims of the course. It

was often hard to follow the logic of what we were

supposed to learn week-to-week. I think more

connection between lecturers could give more sense

to the whole content.”

2016/

36.8%

positive

final

evaluation

of the

course

Randomized order of

lectures. Disconnection

among topics and aims of

the course.

“The general organization of the individual

lectures was in my opinion very illogical.”

“Develop a logical, linear plan for the course and

follow it.”

“Organize the course better and make the course

content more focused.”

2017/

45.5%

positive

final

evaluation

of the

course

Develop a more logical

organization of the course

plan with more focused

contents.

Justification of the intervention

The results of the student evaluations held for three years had recurrent observations and

suggestions to improve the structure and organization of the course. With this consideration in

2017, the course plan was changed and some topics were re-arranged according to calendar

possibilities of teachers involved and personal criteria of few teachers in charge of this task.

Justification of the intervention

The results of the student evaluations held for three years had recurrentobservations and suggestions to improve the structure and organization ofthe course. With this consideration in 2017, the course plan was changedand some topics were re-arranged according to calendar possibilities ofteachers involved and personal criteria of few teachers in charge of thistask.Nevertheless the good intentions, this organization didn’t have yetgood acceptation and response from students. Therefore a "Course Align-


ment" was proposed to improve the organization and structure of the Trop-ical Crop Production course plan.

Study question: Can the course alignment serve as a tool to align andhave a better linkage between different topics of the course?

Intervention - Methods

The objective of this intervention was to determine the course alignment ofa MSc course using the proposed method by Blumberg (2009) that takesinto account the course objectives, cognitive processes, the different typesof knowledge and assessment activities. This method was applied to twocourse plans (2017 and 2018) of the Tropical Crop Production course totest the course alignment of their structures. After testing, results werecompared with the final student evaluations to evaluate the final learningoutcomes.

The method proposed by Blumberg considers the course alignment anddelivery of the course contents by the arrangement of activities that startsfrom the objectives and ends in the assessment. It uses the concepts of ob-jectives taxonomy, cognitive processes and the different types of knowledgeto maximize learning (Engelhart et al., 1984; Krathwohl et al., 2001). Asthis method considers how learning occurs (through different processes andknowledge) it was considered as suitable to be applied in the course wherethe problem was identified. Blumberg method indicates the following steps:The first part corresponds to filling out a table listing the course objectives,the teaching and learning activities and the assessment exercises accord-ing to their corresponding cognitive process. When an objective does nothave any related teaching, learning or assessment activity, then the courseis considered misaligned; this can be visualized through a line drawn con-necting the cells in the columns, when the lines are straight and connectall elements, the course is aligned. If there are diagonals, then the course ismisaligned due to missing elements (Blumberg, 2009).

The course alignment was built considering the Tropical Crop Produc-tion course Intended Learning Outcomes which in the course descriptionwere described considering the different dimensions of knowledge accord-ing to Anderson (Krathwohl et al., 2001): knowledge (factual and concep-tual), skills (procedural) and competences (meta-cognitive).


Table 13.2. Main intended learning outcomes of course where Blumberg (2009)method was appliedTable 2 Main intended learning outcomes of course where Blumberg (2009) method was

applied

Main Intended Learning

Outcomes

To get a comprehensive understanding of properties of

selected tropical crop species and management in tropical

rainfed and irrigated agro-ecosystems. Focus on climate

related production constraints (biotic/abiotic stress) and

human endeavor to optimize crop production in small scale

for poverty alleviation and sustainable production.

Knowledge Skills Competences

Demonstrate knowledge on

principles for Tropical Crop

Production

Analyze and synthetize

information about tropical

crop production

Assess and formulate

agronomical components of

development support

programs

Understand the characteristics

of major tropical crops

Design cropping system

calendars for major tropical

crop species

Advise extension and research

institutions in tropical

countries

Demonstrate the

characteristics of tropical crop

production systems in relation

to agro-ecological and socio-

economic conditions

Develop tropical crop

production plans in relation to

their agro-ecological and

socio-economic conditions

Assess field and greenhouse

trials

Design, implement and

analyze research projects in a

tropical environment


The application of Blumberg method was done for all the objectives of the course. The

application of the method in the course plans of 2017 and 2018 turned to be the same since both

plans had the same objectives and activities and only differed in the activities arrangement.

Ninety percent (90%) of the course plan was aligned (9 out of 10 objectives). For the objective

that was not aligned (“Assess and formulate agronomical components of development support

programs”), adjustments were formulated by reducing activities that were not in line with this

objectives and giving more emphasis to the corresponding assessment exercise as suggested by

Blumberg (2009). This adjustment was not applied yet in the course plan, in order to be able to

compare the final course evaluation results. An example of the application is shown in Table 3

where the teaching, learning and assessment activities are aligned between them and with the

course objective.

The course structures of 2017 and 2018 were considered aligned according to the method

proposed by Blumberg (2009). There were different types of cognitive activities to support and

maximize the learning process. Nevertheless in the final student evaluation of 2017, only 45.5%


The application of Blumberg method was done for all the objectives of thecourse. The application of the method in the course plans of 2017 and 2018turned to be the same since both plans had the same objectives and activi-ties and only differed in the activities arrangement. Ninety percent (90%) ofthe course plan was aligned (9 out of 10 objectives). For the objective thatwas not aligned ("Assess and formulate agronomical components of de-velopment support programs"), adjustments were formulated by reducingactivities that were not in line with this objectives and giving more em-phasis to the corresponding assessment exercise as suggested by Blumberg(2009). This adjustment was not applied yet in the course plan, in order tobe able to compare the final course evaluation results. An example of theapplication is shown in Table 3 where the teaching, learning and assessmentactivities are aligned between them and with the course objective.

The course structures of 2017 and 2018 were considered aligned ac-cording to the method proposed by Blumberg (2009). There were different


types of cognitive activities to support and maximize the learning process.Nevertheless in the final student evaluation of 2017, only 45.5% of studentsshowed a positive final valuation and kept complaining about the coursestructure. In contrast, in 2018 after the new arrangement of activities 100%of the students expressed a positive valuation of this course with commentssuch as "The lectures were logically planned out....".

The course plan of 2018 considered an arrangement of the activities inrelation to the main objectives. As the factual and conceptual knowledgeare considered the foundation to build the other two types (procedural andmeta-cognitive); the objectives of the course describing the "knowledge ob-jectives" were used to identify the key elements, together with the main ob-jective of the course (Table 1). From here four modules were proposed: I:Principals of tropical crop production, II: Major tropical crops, III: Crop op-timization in the tropics and IV: Evaluation. The arrangement of the courseactivities responded to a better logic that considered the main aspects of thecourse objectives and not to the teachers’ time as it was mainly in previousyears. From this case study to value or assess the course alignment it seemsfundamental to consider the logic order of course activities. Therefore aninitial step in the course alignment determination is proposed to determinethe key elements that give a logic order to the course objectives, then enlistthem following that order and apply the rest of Blumberg method. With thisboth the alignment and logic sequencing of the objectives in relation to theteaching, learning and assessment activities can be verified.

Conclusions

The objective of this intervention was to apply Blumberg (2009) methodto assess a MSc course alignment. This exercise was applied to two courseplans of the same MSc course implemented in 2017 and 2018. Consideringthat the course objectives, the teaching, learning and assessment activitieswere the same and only the order of these differed; the course alignmentwas the same and showed that 90% of the course objectives were alignedwith the rest of the course activities. Nevertheless the final student evalua-tion of the course was positive for 45% of the students in 2017 compared to100% in 2018. The course alignment method proposed by Blumberg (2009)and applied to this case was a useful, but not sufficient. The applicationshowed that a good structure of a course that maximizes learning not onlydepends in the consistency between objectives, content, learning activities


and assessment methods, but also in a logic sequencing. With this consid-eration, an initial step to Blumberg’s method is recommended to identify alogic sequencing based on key elements of the course objectives.

References

Krathwohl, D. R., & Anderson, L. W. (2001). A taxonomy for learning,teaching, and assessing: A revision of Bloom’s taxonomy of educationalobjectives. Longman.

Biggs, J. (2003). Aligning teaching and assessing to course objectives.Teaching and learning in higher education: New trends and innovations,2(April), 13-17.

Biggs, J. (2014). Constructive alignment in university teaching: HERDSAReview of Higher Education, 1, 5–22.

Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1984).Taxonomy of educational objectives book 1: Cognitive domain . BS Bloom.

Blumberg, P. (2009). Maximizing learning through course alignment andexperience with different types of knowledge. Innovative Higher Educa-tion, 34(2), 93-103.


A

Table 13.3. Example of the method using one of the objectives of the courseTable 3 Example of the method using one of the objectives of the course

What level is

each of the

following?

Aspect

not

included

in the

course

1 Remember

(recognize,

recall)

2 Understand

(interpret,

exemplify,

classify,

summarize,

infer,

compare,

explain)

3 Apply (execute,

implement)

4 Analyze

(differentiate,

organize,

attribute)

5 Evaluate

(check,

critique)

6 Create

(generate,

plan,

produce)

Objective Demonstrate

knowledge on

principles of

Tropical Crop

Production

Teaching

learning

methods

The tropical

environment,

Highland

Tropics, Seed

biology, soil

fertility, crop

physiology and

stress (lectures)

Cooking activity

with tropical

products

Germination

exercise, soil

fertility, crop

physiology

and stress

(practical

exercises)

Germination and

soil fertility, crop

physiology

implementation of

experiments and

exercises

Analyzing

results in

germination,

soil fertility,

crop physiology

exercises

Course

discussions

Cooking

activity with

tropical

products

Crop

Production

calendar

Assessment

requirements

Quizes and

exam questions

Course quiz I

and II, exam

questions,

reports of

practical

exercises, final

report

Report with the

procedures of

execution of

practical

exercises.

Reports with

the analysis of

practical

exercises

Peer assess

other

students final

reports

Final report

Note: the vertical and horizontal blue lines show the alignment of the teaching, learning and assessment methods with the course objective (Blumberg 2009)


B

Figure 1 Final evaluation results of the Tropical Crop Production course (from eight

respondents out of twelve) with some pictures of the teaching and learning activities of this

course held at the University of Copenhagen (2018).

Fig. 13.1. Final evaluation results of the Tropical Crop Production course (fromeight respondents out of twelve) with some pictures of the teaching and learningactivities of this course held at the University of Copenhagen (2018).

14

Introducing elements of active learning in asmall course

Laura Mancinska

Department of Mathematical SciencesUniversity of Copenhagen

Setting the scene

Despite the modern consensus that traditional-style lectures do not promotedeep and longlasting learning outcomes, many mathematics courses are de-livered precisely in this manner.1 How does one incorporate elements ofactive learning in a pre-established course with traditional-style lectures?Are these additional activities introduced at the expense of covering lessmaterial and do more advanced students suffer from this style of teaching?If so, do the advantages outweigh the negative effects in the context of amaster’s level mathematics course?

Context of the study

This study will be conducted in the context of the 7.5 ECTS Master’s levelcourse “Introduction to Modern Cryptography” at the Department of Math-ematical Sciences (see Table 14.1 for a quick overview of the course). Theteaching format adopted in the previous years was traditional-style lecturesplus exercise classes which are normally used for going over assignment so-lutions or additional course material. This was the first time the course wasoffered as a graded rather than a pass/fail course. Therefore, to allow for

1 A typical mathematics course consists of lectures and exercise classes. It is thelecture component that is often delivered in the traditional “instructor-at-the-blackboard” manner. Most of this article concerns the format of the lecture ratherthan exercise-class component.

164 Laura Mancinska

more accurate individual assessment, I introduced a final exam. This wasdone in view of the fact that the students are encouraged to work on assign-ment problems collectively. Another change in comparison to the previousyears was that the deliverable component of the project was changed fromwritten to oral presentation. Oral presentations allow students to gain ex-perience and improve their presentation skills, as well as allowing them tolearn from each other.

Table 14.1. The course "Introduction to Modern Cryptography" at a glance.

Laura Mančinska Department of Mathematical Sciences

1

Introducing elements of active learning in a small course

1 Setting the scene

Despite the modern consensus that traditional-style lectures do not promote deep and long-lasting learning outcomes, many mathematics courses are delivered precisely in this manner1. How does one incorporate elements of active learning in a pre-established course with traditional-style lectures? Are these additional activities introduced at the expense of covering less material and do more advanced students suffer from this style of teaching? If so, do the advantages outweigh the negative effects in the context of a master’s level mathematics course?

1.1 Context of the study

This study will be conducted in the context of the 7.5 ECTS Master’s level course “Introduction to Modern Cryptography” at the Department of Mathematical Sciences (see Table 1 for a quick overview of the course). The teaching format adopted in the previous years was traditional-style lectures plus exercise classes which are normally used for going over assignment solutions or additional course material. This was the first time the course was offered as a graded rather than a pass/fail course. Therefore, to allow for more accurate individual assessment, I introduced a final exam. This was done in view of the fact that the students are encouraged to work on assignment problems collectively. Another change in comparison to the previous years was that the deliverable component of the project was changed from written to oral presentation. Oral presentations allow students to gain experience and improve their presentation skills, as well as allowing them to learn from each other.

Audience Study program Mathematics (majority), Computer Science, Statistics

Level Master’s (majority), Bachelor’s Size 9 students

Course Weekly contact hours 4 hours of lectures + 3 hours of exercise classes Assessment • 4 assignments

• Individual project presentation (15 min)• Written final exam (3 hours)

Credit 7.5 ECTS

Table 1. The course "Introduction to Modern Cryptography" at a glance.

To evaluate and reflect on the planned intervention we must consider them in the context of intended learning outcomes (ILOs). To comply with the Danish Qualifications Framework

1 A typical mathematics course consists of lectures and exercise classes. It is the lecture component that is often delivered in the traditional “instructor-at-the-blackboard” manner. Most of this article concerns the format of the lecture rather than exercise-class component.

To evaluate and reflect on the planned intervention we must considerthem in the context of intended learning outcomes (ILOs). To comply withthe Danish Qualifications Framework for Higher Education (Danish Min-istry of Higher Education and Scince, 2008), ILOs are specified in terms ofknowledge, skills, and competencies (see Table 14.2).

Table 14.2. Intended learning outcomes of the course "Introduction to ModernCryptography".


2

for Higher Education (Danish Ministry of Higher Education and Scince 2008), ILOs are specified in terms of knowledge, skills, and competencies (see Table 2).

Knowledge The students will have an understanding of the theoretical and mathematical basis of modern cryptographic systems.

Skills The students will be able to give rigorous security proofs of basic cryptographic systems and connect various cryptographic primitives with rigorous reductions.

Competencies Understanding theorems about theoretical cryptography; proving security reductions; reasoning about the limits of computationally-bounded adversaries.

Table 2. Intended learning outcomes of the course "Introduction to Modern Cryptography".

2 The planned intervention and its theoretical backing

In this section we describe the planned intervention which can be summarized as incorporating elements of active learning in traditional-style lectures of a pre-existing course. We include a description of the elements to be incorporated, give specific examples, explain the desired outcomes of each of the elements. We also provide theoretical backing of the intervention as a whole as well as its constituent elements.

2.1 Theoretical backing and motivation behind the intervention

Brown et al. define learning as “Acquiring knowledge and skills and having them readily available from memory so you can make sense of future problems and opportunities.” (Peter C. Brown 2014). According to the seminal work (Piaget 1978), the learner is viewed as a

goal-oriented agent who actively seeks information. In contrast, the traditional didactic lectures are one-way exchanges, where the knowledge is supposed to flow from the instructor to the students. In addition, the modern consensus is that the acquiring of usable knowledge and skills occurs when the learner examines different facets of the topic in question and connects it to already existing knowledge and skills (see e.g. (John D. Bransford 2000)). During a traditional-style lecture it is easy for a student to take the role of a passive listener and leave the auditorium without having connected the discussed topic with their previous knowledge and experiences. Active and student-centered learning is an approach that seeks to correct this by putting the focus on the student and placing them into situations where they are “forced” to take a more active role and form the desired links. Of course, not all the students require such forcing as they independently examine different aspects of the discussed topic and contrast it with their pre-existing knowledge via an un-coerced inner reflection. However, it is not this group of students2 that an instructor should focus on as they are bound to learn almost irrespective of the circumstances. Instead one should encourage the more passively-inclined students and design activities that would encourage them to take up a more active role therefore promoting the acquisition of deep, usable and long-lasting knowledge.

2 For the purpose of this article, let us briefly refer to this group as ``advanced students’’ or “advanced learners”.

14 Introducing elements of active learning in a small course 165

The planned intervention and its theoretical backing

In this section we describe the planned intervention which can be summa-rized as incorporating elements of active learning in traditional-style lec-tures of a pre-existing course. We include a description of the elements tobe incorporated, give specific examples, explain the desired outcomes ofeach of the elements. We also provide theoretical backing of the interven-tion as a whole as well as its constituent elements.

Theoretical backing and motivation behind the intervention

Roediger et al. define learning as “Acquiring knowledge and skills and hav-ing them readily available from memory so you can make sense of futureproblems and opportunities.” (Roediger et al., 2014). According to the sem-inal work (Piaget, 1978), the learner is viewed as a goal-oriented agent whoactively seeks information. In contrast, the traditional didactic lectures areone-way exchanges, where the knowledge is supposed to flow from the in-structor to the students. In addition, the modern consensus is that the acquir-ing of usable knowledge and skills occurs when the learner examines differ-ent facets of the topic in question and connects it to already existing know-ledge and skills (see e.g. Bransford et al., 2000)). During a traditional-stylelecture it is easy for a student to take the role of a passive listener and leavethe auditorium without having connected the discussed topic with their pre-vious knowledge and experiences. Active and student-centered learning isan approach that seeks to correct this by putting the focus on the student andplacing them into situations where they are “forced” to take a more activerole and form the desired links. Of course, not all the students require suchforcing as they independently examine different aspects of the discussedtopic and contrast it with their pre-existing knowledge via an un-coercedinner reflection. However, it is not this group of students2 that an instructorshould focus on as they are bound to learn almost irrespective of the circum-stances. Instead one should encourage the more passively-inclined studentsand design activities that would encourage them to take up a more activerole therefore promoting the acquisition of deep, usable and long-lastingknowledge.

2 For the purpose of this article, let us briefly refer to this group as “advancedstudents” or “advanced learners”.

166 Laura Mancinska

Description of the intervention

We now proceed to describe the elements of active learning introduced inthe lecture-component of the course “Introduction to Modern Cryptogra-phy”. We also describe what aspects of active learning each of these ele-ments is meant to address.

a) Short self- or peer-graded quizzes at the beginning of the lecture.Description: Start a lecture by asking the students to recall the pre-viously introduced concepts necessary for the upcoming lecture. Thestudents would be given a few minutes to write down their answerand then provided with a model solution along with a sample grad-ing scheme. Sometimes the students would grade their own solutionsand sometimes they would be asked to switch with a peer. The stu-dents’ performance in these quizzes had no effect on their final grade.However, similar style questions were asked in the first part of the fi-nal exam and I would remind the students of this in order to encouragemaximum effort.Intended outcome: Activate the students at the beginning of the lectureand refresh the concepts needed to place the forth-coming material inthe context of the previously covered topics. The purpose of the grad-ing scheme is to help the students identify the essential ingredients ofa correct answer. The intention behind peer-grading is to help the stu-dents recognize a correct and complete answer as we are often quickerto notice shortcomings of others’ work in comparison to our own.

Table 14.3. An example quiz question.


3

2.2 Description of the intervention

We now proceed to describe the elements of active learning introduced in the lecture-component of the course “Introduction to Modern Cryptography”. We also describe what aspects of active learning each of these elements is meant to address.

a) Short self- or peer-graded quizzes at the beginning of the lecture.Description: Start a lecture by asking the students to recall the previously introducedconcepts necessary for the upcoming lecture. The students would be given a fewminutes to write down their answer and then provided with a model solution alongwith a sample grading scheme. Sometimes the students would grade their ownsolutions and sometimes they would be asked to switch with a peer. The students’performance in these quizzes had no effect on their final grade. However, similar stylequestions were asked in the first part of the final exam and I would remind thestudents of this in order to encourage maximum effort.Intended outcome: Activate the students at the beginning of the lecture and refresh theconcepts needed to place the forth-coming material in the context of the previouslycovered topics. The purpose of the grading scheme is to help the students identify theessential ingredients of a correct answer. The intention behind peer-grading is to helpthe students recognize a correct and complete answer as we are often quicker to noticeshortcomings of others’ work in comparison to our own.

Question Define a private-key encryption scheme Answer A (simple) PK-encryption scheme 𝛱 over (𝒦,ℳ,𝒞) consists of three

probabilistic algorithms • Key generation algorithm Gen producing key 𝑘 ∈ 𝒦• Encryption algorithm Enc:𝒦 ×ℳ → 𝒞

𝑐 ← Enc/(𝑚) • Decryption algorithm Dec:𝒦 × 𝒞 → ℳ.

Scheme must satisfy correctness requirement Dec/1Enc/(𝑚)2 = 𝑚for any 𝑚 ∈ ℳ and any key output by Gen.

Grading scheme

[1pt] mention the three sets (𝒦,ℳ, 𝒞) [1pt] for a correct description of each of the 3 algorithms [1pt] Correctness requirement [1pt bonus] For “probabilistic” and quantifiers on m, k.

Table 3. An example quiz question. b) Guided interactive investigation of a topic.

Description: Rather than presenting the students with a ready-made theory andconcepts, sometimes I would attempt to develop the material together with them via aseries of leading questions. Precisely what is meant here is best understood via anexample (see Table 4).Intended outcome: Engage the students and allow them to actively discover thematerial themselves, promote deeper understanding, develop research skills.Further considerations: When employing this tactic, one should bear in mind that it israther time-consuming. It is a very engaging way to present new material and itundoubtedly activates students and leads to a deeper understanding of the topic.However, due to the associated time cost and the fact that not every topic lends itselfeasily to this approach, I did not use it in every single lecture.


b) Guided interactive investigation of a topic.Description: Rather than presenting the students with a ready-madetheory and concepts, sometimes I would attempt to develop the materialtogether with them via a series of leading questions. Precisely what ismeant here is best understood via an example (see Table 14.4).Intended outcome: Engage the students and allow them to actively dis-cover the material themselves, promote deeper understanding, developresearch skills.Further considerations: When employing this tactic, one should bearin mind that it is rather time-consuming. It is a very engaging way topresent new material and it undoubtedly activates students and leadsto a deeper understanding of the topic. However, due to the associatedtime cost and the fact that not every topic lends itself easily to thisapproach, I did not use it in every single lecture.

Table 14.4. An example of guided investigation.


4

Concept to be discovered

Message authentication codes (MACs) for long messages.

Question and answer sequence

Initial question (instructor): Now that we have seen fixed-length MACs, how could we use them to authenticate longer messages?

Answer (student): Split message into blocks and obtain the tag by concatenating tags for each of the blocks.

Instructor (writes on the board): Alright, so we consider a block-message 𝑚6𝑚7 …𝑚/ and then obtain its tag as 𝑡6𝑡7 … 𝑡/ , where

𝑡: = Mac/(𝑚:) Does anybody see a possible attack? (Give some time if no one raises their hand.) Student: We can request tags for two different messages and then produce a new valid message-tag pair.

Instructor: Formalizes the suggested answer on the board. Does anyone see how we could modify the previous idea to render this attack ineffective? (Give some time.) Students: … Instructor: Recall that we know from the last week that deterministic encryption schemes are insecure…

Student: Offers an idea that is later formalized by the instructor as a complete construction for MACs that can be used to authenticate long messages.

Table 4. An example of guided investigation. c) Short activation questions.

Description: While explaining the material pose quick questions to the audience. Forexample, the question could be drawing students’ attention to crucial or subtle aspectsof the notion currently being introduced or asking them to compare it to a relatedpreviously introduced concept. Other questions of this type would be to encourage thestudents to suggest a relatively easy-to-anticipate next step in a proof or perform asimple calculation. In order to avoid interrupting the flow of the exposition thesequestions are designed to be rather straightforward.Intended outcome: Activate the students that have slipped into passive listening,encourage the students to link the new material with the previously introduced one,give them a chance to gain further familiarity with the concept in question.

3 Evaluation and outcomes of the intervention

The intervention was evaluated via the following four methods E1. A standard course evaluation form (Anonymous, Response rate: 5/9). E2. Additional free-form question specifically regarding the small exercises and quizzes

during the lectures (Anonymous, Response rate: 5/9). E3. An individual, informal follow-up interview with select students. E4. Feedback from the peer-supervision group who observed two lectures.

168 Laura Mancinska

c) Short activation questions.Description: While explaining the material pose quick questions to theaudience. For example, the question could be drawing students’ atten-tion to crucial or subtle aspects of the notion currently being introducedor asking them to compare it to a related previously introduced con-cept. Other questions of this type would be to encourage the studentsto suggest a relatively easy-to-anticipate next step in a proof or per-form a simple calculation. In order to avoid interrupting the flow of theexposition these questions are designed to be rather straightforward.Intended outcome: Activate the students that have slipped into passivelistening, encourage the students to link the new material with the pre-viously introduced one, give them a chance to gain further familiaritywith the concept in question.

Evaluation and outcomes of the intervention

The intervention was evaluated via the following four methods:

E1. A standard course evaluation form (Anonymous, Response rate: 5/9).E2. Additional free-form question specifically regarding the small exer-

cises and quizzes during the lectures (Anonymous, Response rate: 5/9).E3. An individual, informal follow-up interview with select students.E4. Feedback from the peer-supervision group who observed two lectures.

The course evaluation E1 was generally very positive. For instance, allof the students agreed that “they have acquired the competencies describedin the course objectives”. The students evaluated this year’s course a bithigher than the previous year’s one in practically all the categories. Also,based on evaluation E1, the Teaching Committee at MATH categorized thiscourse as belonging to (the highest) category A. This category is describedas “Courses where the teaching has worked particularly well and can in-spire others.”

Of course, without a further follow-up E1 does not allow us to con-clude that the positive evaluation was due to the added elements of activelearning. In E2 the students explicitly mention that they found the smallexercises helpful and that they allowed them to discern the most importantaspects of definitions. However, most of the students did not like that someof the quizzes were peer-graded. Some mentioned feeling uncomfortablewhile others felt they could do just as well by grading their own work.


According to the feedback from the peer-supervision group who ob-served two of the lectures (E4), the students were generally engaged duringthe lectures and the interactive student-activation elements were perceivedas aiding active learning. However, according to E4, student engagementlowered while a longer proof was being presented on the board.

Finally, according to evaluation E3, students generally enjoyed theadded interactive elements. However, some of the more advanced studentsmentioned that the they found the pace of the course a bit slow. This wasalso echoed in one of the anonymous comments from E1.

Discussion of the outcomes and further improvements

In general, the introduced elements of active learning seem to have in-creased the student engagement and also their satisfaction with the course.However, the small number of involved students does not allow to concludewith certainty that the observed improvement is due to the intervention. Infact, to me the most convincing piece of evidence is my own empirical ob-servation of the student engagement during the lectures that suggests thatthe intervention worked well. However, the small exercises did take up timewhich slowed the pace of the lectures and reduced the amount of material Iwas able to cover.

I believe that the effects of the intervention varied between differentgroups of students. For instance, if the definition from the last week is freshin your memory, you are not gaining much by being asked to repeat it. Also,the time used for the small exercises is generally tailored to the average paceof the students. Therefore, I believe that the intervention might have causedsome negative effects for the learning outcomes of the more advanced stu-dents. In general, I find that it is challenging to design activities that areequally beneficial to all groups of students.

The evaluation shows that peer-grading was generally not perceivedwell. However, evaluations E1 and E3 show that this can be attributed tomiscommunication. The students seemed to believe that peer-grading wasused to promote fair assessment. However, this was not the intention be-hind it (see (a) Intended outcomes in Section 2.1). I believe the perceptionof peer-grading can be altered by better communicating the reasons behindit.

In the future I would like to experiment with incorporating elementsof active learning into the presentation of mathematical proofs. Proofs are

170 Laura Mancinska

important elements of virtually any mathematics course, as they form thefoundation of mathematical thinking. They are most often presented in themost elegant available form that highlights the crucial steps. This is usu-ally not the form the proof is first conceived. So often times the instructorpresents the proof without much interaction with the students. This canlower the student engagement. In fact, this was my experience during the“Introduction to Modern Cryptography” course and it was also pointed outby my peer-supervision group (evaluation E4).

References

Danish Ministry of Higher Education and Scince. (2008). "Danish Qualifi-cations Framework for Higher Education." Uddannelses og Forskningsmin-isteriet. ufm.dk.

Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn(Vol. 11). Washington, DC: National academy press.

Roediger, H. L., McDaniel, M. A., & Brown, P. C. (2014). Make it stick:The science of successful learning. Harvard University Press.

Piaget, Jean. (1978). Success and Understanding. Harvard University Press.

15

Implementing a student-centered exerciseformat in a PhD course in applied biostatistics

Signe Marie Jensen

Department of Plant and Environmental SciencesUniversity of Copenhagen

Introduction

Statistics is widely taught in different educations and levels due to its instru-mental role in other disciplines and its role in the development of criticalthinking (Batanero, 2004). Despite the fact that most university students inscience take at least one course in statistics during their education, statis-tics continuously seems to be a “foreign language” for many students whonever obtain the associated competences. Although many students may becapable of manipulating formulas and equations, they do not have the abil-ity to understand what statistical procedure to apply when facing a real dataset (Quilici et al., 1996).

Good teaching is teaching that makes it possible for all students to makeprogress. Accordingly, teaching should preferably be tailored to each indi-vidual student (Splittorff et al., 2009). The challenge is to reach students atall levels. One way of doing this could be to differentiate exercises, to avoidthat the tasks become too hard for some but still offer a challenge in orderfor each individual student to learn.

Part of learning is relating what is taught to what is already known (Il-leris, 2003). The teacher’s job is to help provide the link between the two.One way to do this is to put the methodology taught into a well-knowncontext for instance through example based exercises (Quilici et al., 1996).In the context of statistics, this may be done using data examples from theliterature within the research area of focus to the students.

Interest and mobilization of mental energy affects the students’ attitudesto the intended learning (Illeris, 2003). It is therefore beneficial to the learn-

172 Signe Marie Jensen

ing process if case studies and exercises appeal to the individual student. Incourses teaching cross-field disciplines, i.e. a PhD course in biostatistics,students may come with very diverse backgrounds and interests. In such acourse, providing examples that appeal to all is a challenge.

Differentiated instructions are well-used and examined in elementaryschool, but very few studies have been made in adult learning, even thoughadult students are perhaps even more diverse due to varying educationaland life experiences (Dosch et al., 2014). From the few studies performedat college level a common outcome is the students appreciation of havingchoices and that this allowed them to learn better (Dosch et al., 2014). Ad-ditionally a few studies have found a significant increase in the final scorefor the students taught using differentiated learning compared to the tra-ditional teacher-centered learning (Chamberlin et al., 2010; Dosch et al.,2014).

The present project implements a student-centered exercise format inthe PhD course: Applied biostatistics in biological sciences using R withfocus on applications in nutrition, physiology, and plant and environmentalsciences. The new format includes choice of exercises based on examplesfrom two different main areas, 1) nutrition and physiology and 2) plantand environmental sciences, as well as two levels of details provided in theexercises.

Methods

The course

Applied biostatistics in biological sciences using R with focus on applica-tions in nutrition, physiology, and plant and environmental sciences is arelatively new course, as 2017 was the first time we ran the course. Theoverall aim of the course is to give PhD students in applied biosciences asolid ground for analyzing and interpreting their own data.

The course is an intense one-week course with lectures and exercisesfollowed by three weeks to write a report on the PhD student’s own data.The lectures include theory but are primarily based on examples and hands-on instructions on how to do the appropriate analyses in the statistical pro-gramming software R. The assessment of the course is an evaluation of thereport assessed with passed/not passed. The students additionally get feed-back on what to improve to pass, if not passed in the first attempt, or ingeneral to improve the analyses and presentation.

15 Implementing a student-centered exercise format... 173

There are no specific prerequisites for the course in terms of othercourses, but there is an expectation that the students know the basic statis-tics.

The intended learning outcomes for the course is:Skills

• To understand key concepts and ideas underlying a wide range of sta-tistical methods

• To interpret output and results from a wide range of statistical analyses

Competences

• To identify and apply appropriate R functionality for data analysis andvisualization

• To define statistical challenges for given data sets and limitations in theintroduced statistical methodology

The students

Twenty-eight PhD students participated in the course in 2018. The classconsisted of PhD students with a very diverse geographical background,counting students from North America, Asia and Europe, including 12 PhDstudents with a Danish background. Most, but not all, were in a currentPhD position at a Danish University. The PhD students were involved inwide range of research areas including bioengineering, agronomy, biology,nutrition and physiology.

The level of the PhD students in terms of statistical background know-ledge and experience with the software used ranged from low to relativelyhigh, as judged by activity during lectures and performance in exercises.The big majority had one or more statistics courses “in their backpack”, butfor many of the participants the methods taught were well hidden or longforgotten or never grasped.

Student-centered exercises

Two changes to the exercises were tested this year:

1) Within each exercise session covering a specific topic, a range of ex-ercises from different research areas were provided. All exercises wereconstructed in a way that to a certain degree the same skills and com-petences were trained in all exercises.


2) All exercises were provided in two versions. Version A of an exerciseincluded a description of the data example and one or two researchquestions providing an open-ended exercise for experienced students.Version B additionally included a step-by-step guide on how to answerthe research question providing more help for those in need of it, butwithout giving away all the details.

Starting each exercise session, the PhD students were told to choosefreely between all exercises finding an exercise that they found interestingand perhaps even related to their own research area. Furthermore, the PhDstudents were encouraged to start out in version A of the chosen exerciseand only move to version B if they were stuck. Each time an exercise ses-sion started the PhD students were encouraged to spend a little extra timein version A, compared to the last session, before moving to version B ofthe exercise. Two teachers were available for help and discussion during allexercise sessions. In last exercise session, the students were encouraged toteam up with someone interested in the same type of assignment.

Evaluation of the new format

Evaluation of the new format of exercises were based on oral evaluationin plenum and an anonymous questionnaire focusing on the new format ofexercises sent to all participants at the course 2 weeks after the final courseday.

Statistical analysis

Questions from the questionnaire with only two possible answers (Yes/No)were analyzed using logistic regression and results were reported as per-centage with 95%-confidence intervals.

Time spend on the two different versions of the exercises were summa-rized to whether or not there was an increase in percentage time spend onversion A of the exercises from the first exercise session to the last.

Results

All course participants took active part in the plenum evaluation of thecourse and seventeen course participants answered the online question-naire. The outcome from the questionnaire and the oral evaluation is pre-sented below for the two changes separately.


Using example data from several research areas

Of the 17 answering the questionnaire, 88.2% (67.9-97.9) liked the use ofexercises inspired by data from different research areas. One person elab-orated this by saying “This was a very interesting part of the course forme”.

When asked whether they understand the statistical challenges better ifexercises are based on data close to their own research area 82.3% (60.4-95.3) answered yes. One elaborated this with saying: “I will be more mo-tivated if the data set related to my research area”, while one person dis-agreeing explains his/her answer by saying: “I think that it is very usefulto understand statistical challenges also if they are a bit far from your ownresearch area, and as long the design of the experiment is explained, it isokay”.

Finally, 94.1% (76.6-99.6) stated that we as teachers succeeded in pre-senting exercises that appeal to him/her.

Two levels of detail

All 17 of the 17 answering the online questionnaire liked the use of versionA and B for the exercises. One participant elaborated his/her answer saying:“Very nice to start without help - and then move on. If we only had versionA, I think we would have waited for help instead of figuring out ourselves”.Another wrote: “I thought I gained most from sticking to version A andasking for help. But thinking back it might have been the wrong approachcaused by my stubbornness. I think some of my questions could have beenfound in the version Bs”, while a third participant elaborated the answerwith: “I did not use it that much myself, but I think that it will be morehelpful for the ones that had more challenges with the statistical exercises”.

Of the 17 answering the online questionnaire, 1 did not provide timespend on the two versions, 8 spend less time, 4 spend the same amount oftime and 4 spend more time on version A on the last day than on the firstday.

Finally, the students were presented with the main purpose of runningthe exercises in two versions: To give all participants the opportunity totrain the following competences: 1) To be able to identify statistical chal-lenges for given sets. 2) To be able to identify which model is appropri-ate for given data sets. When asked whether they felt these goals wereachieved, no one answered “No”, 58.8% (35.4-79.7) answered “Yes” and41.2% (20.3-64.6) answered “To some degree”.


Discussion

Overall, the new format of exercises was a success and an improvementover the existing format of the exercises and I take notice that all studentsappreciate the two levels format of the exercises.

Based on my impression from the exercises sessions, I was expectingthat most students spend an increased percentage of time in version A of theexercises on the last day compared to the first day. However, this was notthe case. There may be several reasons why this was not the case, one be-ing that the level of the theory introduced in the lectures and subsequentlytrained in the exercises were much higher on the last days than in the begin-ning of the week where the material covered was repetition for a big part ofthe PhD students. For many of the PhD students the content of the last daysof the course was associated with understanding or at least accepting a con-siderably new way of thinking of experiments and the resulting challengesin terms of data analysis and this transcendent learning was not achievedby everybody, at least no during the course (Illeris, 2003).

Another reason may be the limited time and number of exercise sessionsthe participants had to get used to the format. While the format is supposedto gradually build up the competences to identify statistical challenges andappropriate model for a given data set, a competence in the definition asthe ability to perform adequately and flexibly in well-known as well asunknown situations takes time (Illeris, 2003). In that matter, the format maybe even more suited for courses that takes place over a longer period, i.e.block or semester courses, for the students to have more time to get familiarwith the exercise format.

The use of the two versions of exercises were supposed to make the PhDstudents take responsibility for their own learning. Taking responsibilityfor own learning should in general be expected for PhD students, however,statistics as a discipline is not the main interest for the participants on thiscourse coming from applied research areas.

The use of teams in the last exercise session gave a boost to the formatfrom my point of view. Most teams really used each other for competentpeer feedback on ideas on how to solve the exercises, trying of ideas oneach other before they asked one of the teachers, which again gave themmore faith in their own solutions (Rienecker et al., 2015; The University ofEdinburgh n.d.). The sparring gave everybody a lift that made the teachersredundant for a longer time. This again made it possible for us to take no-


tice of the different ways of approaching the exercises and facilitating thediscussions going on rather than just being problem solvers.

Conversations with students at bachelor and master level often revealthat they feel unfamiliar with the data examples used in the basic statisticscourses and that this makes it harder for them to understand the methodstaught and to see how it fits into their own study field. The examples used inthe student-centered exercises were chosen from the literature with the pur-pose of reflecting fundamental principles and ways of thinking that couldbe generalized for many research disciplines (Mørcke et al., 2015). To pro-mote thinking by analogy, providing examples close to the field of inter-est/research of the student may ease the recognition of similar problems, inorder to map the method, which in the end hopefully leads to abstraction(Quilici et al., 1996).

Perspectives

The student-centered exercise format has potential and could be further de-veloped for the course next year. In particular, I would expect a deeperintroduction to the format and how it fits into the learning outcomes of thecourse would benefit the outcome (Jørgensen, 2015). Teamwork will beencouraged from day one.

A third version of the exercises may additionally be introduced; a step-by-step solution on how to do the programming to get through the exer-cises. Students struggling with the programing part could use this versionto avoid spending too much time on this part but instead having more timeto focus on the interpretation of the results. Finally, the format should beimplemented into other courses, particularly courses involving more exer-cise sessions.

References

Batanero, C. (2004, July). Statistics education as a field for research andpractice. In Regular Lecture at the Tenth International Congress on Math-ematical Education, Copenhagen, Denmark.

Chamberlin, M., & Powers, R. (2010). The promise of differentiated in-struction for enhancing the mathematical understandings of college stu-


dents. Teaching Mathematics and Its Applications: An International Jour-nal of the IMA, 29(3), 113-139.

Dosch, M., & Margaret Z. (2014). “Differentiated Instruction in the Col-lege Classroom.” International Journal of Teaching and Learning in HigherEducation 26(3): 343–57.

Illeris, K. (2003). Towards a contemporary and comprehensive theory oflearning. International journal of lifelong education, 22(4), 396-406.

Jørgensen, Peter Stray. (2015). “Instructing Written Assignments.” In Uni-versity Teaching and Learning, 175–177.

Mørcke, Anne Mette, and Camilla Østerberg Rump. (2015). “UniversityTeaching and Learning - Models and Concepts.” In University Teachingand Learning, 93–104.

Quilici, J. L., & Mayer, R. E. (1996). Role of examples in how studentslearn to categorize statistics word problems. Journal of Educational Psy-chology, 88(1), 144.

Rienecker, Lotte, & Jesper Bruun. (2015). “Feedback.” In University Teach-ing and Learning, DK, Frederiksberg C: Samfunds litteratur, 259–280

Splittorff, K et al. (2009). Preparation Centered Teaching

The University of Edinburgh. “Enhancing Feedback; Involving Students inFeedback.” http://www.enhancingfeedback.ed.ac.uk/staff/resources/involvingstudents.html

This appendix contains examples of two exercises from different research areas used at the PhD course in 2018. Both exercises are provided in a version A and a version B.

Exercise 1 version A

Three standard cultivars (st, ci, wa) and 30 entries was laid out with 6 blocks (Pattersen, 1994). Only the three standards were repeated among the blocks; the other 30 cultivars were only observed once.

Data are available in the file: augmented.xlsx

Answer the following research questions:

1) What is the estimated mean yield for each cultivar?2) Which cultivar(s) is the best in terms of highest yield?

Exercise 2 version A

Relaxation techniques could lower the heart rate. In a randomized trial 28 recreational distance runners were randomly allocated to one of 3 treatments: control, meditation, or progressive muscular relaxation (PMR). The steady state heart rate during a standardized running test was taken before and after a two-week treatment period (Ashley et al., 1996).

Data are available in the file: RELAXATION.CSV

Answer the following research question:

1) Did any of the two relaxation techniques lower heart rate as compared to the control?


A Examples of exercises

Exercise 1 version B

An augmented design with three standard cultivars (st, ci, wa) and 30 entries was laid out with 6 blocks (Pattersen, 1994). Only the three standards were repeated among the blocks; the other 30 cultivars were only observed once.

Data are available in the file: augmented.xlsx

Answer the following research questions:

1) What is the estimated mean yield for each cultivar? To answer this question proceed through the following steps:

a. Import the data and make sure that the import was successful b. What is the random effects structure based on the experimental design? c. Fit a linear mixed model using the random effects structure found above (Hint: Use

lmer() from lme4) d. From the fitted model obtain estimated means and standard errors for each cultivar

2) Which cultivar(s) is the best in terms of highest yield? To answer this question proceed

through the following steps: a. Compare estimated yields between cultivars (Hint: use glht() in multcomp) b. Does the highest yielding cultivar differ significantly from all other cultivars or are

there any other cultivars that performs equally good

Exercise 2 version B

Relaxation techniques could lower the heart rate. In a randomized trial 28 recreational distance runners were randomly allocated to one of 3 treatments: control, meditation, or progressive muscular relaxation (PMR). The steady state heart rate during a standardized running test was taken before and after a two-week treatment period (Ashley et al., 1996).

Data are available in the file: RELAXATION.CSV

Answer the following research question:

1) Did any of the two relaxation techniques lower heart rate as compared to the control? a. Import the data and make sure that the import was successful b. What is the random-effects structure based on the experimental design? c. What is the fixed-effects structure based on the experimental design? d. Fit a linear mixed model using the random effects structure found above. e. From the fitted model obtain estimated means and standard errors for each of the

three treatments. f. Compare the two “active” treatments to the control treatment based on the fitted

model.

Part V

Strengthening peer feedback

16

Peer-feedback i holdundervisning og hvordandet påvirker de studerendes opfattelse afegenlæring på kurset Globale Geosystemer

Mikkel Fruergaard

Institut for Geovidenskab og NaturforvaltningKøbenhavns Universitet

Indledning og problemformulering

Dette projekt tager udgangspunkt i holdundervisningen på bachelorkursetGlobale Geosystemer (GGS), som afholdes på Institut for Geovidenskabog Naturforvaltning (IGN), Københavns Universitet (Bilag 1). På kurset,som er et obligatorisk 2.-års kursus i blok 2 på Geografi og Geoinformatik-uddannelsen, benyttes undervisningsformerne forelæsninger og holdunder-visning. På årets kursus (2018/19) er der ca. 75 tilmeldte studerende, somer inddelt i tre hold til holdundervisningen. For at opnå mest mulig aktivitetfor den enkelte studerende i holdundervisningen, arbejder de studerende igrupper med 3 til 5 studerende per gruppe.

Fra underviserteamets side været et ønske om, at de studerende skaltræne deres skriftelige kompetencer inden den skriftelige eksamen i GGSog inden de skal skrive deres 2. års-opgave i blok 3 og 4. På trods af det-te ønske har skriftlig afleveringsopgaver ikke været anvendt i kurset. Denovervejende årsag er manglende tid hos underviserne til rette og give feed-back på opgaverne. Dette er en udfordring som rækker ud over GGS og påuniversitets-uddannelser er det en generel udfordring, især på kurser medmange studerende, at det er tidskrævende at give studerende feedback (Ri-enecker et al., 2015). Da feedback netop har til formål at den studerendelærer, vil den manglende feedback betyde mindre læring (Rienecker et al.,2015).

Baggrunden for denne UP-opgave er altså et ønske om at udvikle un-dervisningen i GGS i en retning, hvor (1) de studerende træner deres skrif-

184 Mikkel Fruergaard

telige færdigheder igennem opgaveskrivning og (2) de modtager feedbackpå deres opgaver uden (3) undervisergruppen får betydelige mere forbere-delsestid.

På denne baggrund er der på årets kursus indført en skriftelig afle-veringsopgave samt peer-feedback til evaluering af denne opgave. Peer-feedbacken indføres som et element i undervisningen, som primært har tilformål at understøtte de studerendes læring og herved opfyldelse af kur-sets kompetencemål. Sekundært er målet med peer-fedback-øvelsen af destuderende bliver bekendt med og i stand til at give feedback til andre.

Problemformformuleringen lyder således: Hvordan påvirker anvendel-sen af peer feedback de studerendes opfattelse af deres læring?

Dette spørgsmål blev belyst ved at foretage en intervention i holdun-dervisningen på GGS, hvor de studerende både skulle give og modta-ge feedback på en mindre skriftelig opgave. Konkret blev peer feedback-formerne ”editing and redrafting” og ”co-revising” anvendt (The Univer-sity of Edinburgh, 2018). Disse er beskrevet nedenfor.

Interventionen og dokumentation

For at besvare problemformuleringen blev nedenfor beskrevne interventiondesignet. Interventionen blev diskuteret med både min fagvejleder og enanden fastansat kollega på vores institut inden den blev afholdt for de stu-derende på GGS. Som en del af interventionen blev de studerende sat tilskrifteligt at besvare en opgave (bilag 2). Opgaven er todelt bestående afen faglig del, hvor de studerende arbejder med en konkret case relevant forkursets kompetencemål, samt en evalueringsdel. De studerende fik én ugetil at lave opgaven.

Jeg startede holdundervisningen med at motivere henholdsvis den skrift-lige øvelse og feedback delen. Til den sidste del lagde jeg vægt på følgende:

1. Den læringsmæssige værdi at modtage og give feedback (jf. Rieneckeret al., 2015).

2. At (peer) feedback er en integreret del af de fleste jobs, som de stu-derende vil besætte efter studiet (Biggs et al., 2007; Brindley et al.,1998).

Herefter blev de studerende delt ind i grupper af 2 til 5 studerende. Påhvert undervisningshold var der 6 eller 8 grupper, som blev parret to og

16 Peer-feedback i holdundervisning... 185

to, så hver gruppe havde en feedback-gruppe. I opgavens første del – denfaglige del – blev de studerende bedt om at besvare en række spørgsmålom to forskelige geografiske områder. Hver gruppe skulle vælge ét af de toområder, men de parrede grupper skulle vælge forskellige områder. Efteren introduktion til den faglige del blev de studerende instrueret i opgavensanden del – peer feedback-delen (bilag 2).

Peer feedback-delen er også todelt da den består af både en skrifteligog en mundtlig del. Da det at give feedback ikke er en del af Geografiog Geoinformatik-uddannelsen var mit udgangspunkt at de studerende ikkehavde kendskab til, hvordan feedback skal gives. Så for at give kursisterneet fælles udgangspunkt for, hvordan konstruktiv feedback gives skrev jegen kort vejledning (bilag 3). I vejledningen fremhæver jeg at feedbacken,såvel om den er mundtlig eller skriftelig, skal være positiv og specifik, atde studerende skal stille åbne spørgsmål og udgå negativ sprogbrug samtgiv forslag til ændringer. Processen blev indledningsvis også skitseret påtavlen.

I den skriftlige del skulle de studerende kommentere og rette hinan-dens tekster (editing and redrafting). Kommenteringen foregik ved hjælpaf kommentarfunktionen og track changes i Word. Denne feedback er selv-generet, og de studerende kan umiddelbart ikke drage fordel af interaktionmed hinanden. Fordelen ved den individuelle arbejdsform er dog, at meretilbageholdende studerende får tid og mulighed for selvstændigt at arbejdemed evalueringen.

I den mundtlige del skulle de studerende gruppevis give hinanden feed-back (co-revising). Udgangspunktet for feedbacken var det samme som forden skriftelige feedback (se ovenfor), men hvor den skriftelige del var hjem-mearbejde, foregik den mundtlige del til holdundervisningen. Hver gruppehavde cirka 10 til 12 minutter til at give deres feedback. Inden feedback-session gik i gang tegnede jeg figuren med de fire kvadranter og pointeredeigen af feedback skal være positiv og specifik. Ved at denne del af interven-tionen foregik til selve holdundervisningen fik jeg mulighed for at fungerersom mediator, der kunne guide de studerende i deres evaluering samt at til-byde komplementerende feedback. Feedback-delen blev afsluttet ved at jegfra tavlen opsummerede resultaterne af øvelsen, samt evaluerede forløbet(se nedenfor).

Kvalitative metoder blev anvendt til at dokumentere interventionen.Umiddelbart efter at feedback-delen var afsluttet blev forløbet evalueret.Denne evaluering tog omkring 15 minutter og foregik i plenum, hvor jegstillede spørgsmål (bilag 4) om, hvordan de studerende umiddelbart havde


oplevet feedback-øvelsen. Svarende fra de studerende blev sammenholdtmed mine egne observationer og observationer gjort at min pædagogiske ogfaglige vejleder, som overværede interventionen. Endeligt er interventionendokumenteret igennem et interview foretaget med to studerende fra kurset.Interviewet tog udgangspunkt i spørgsmålene i bilag 4 og fokuserede deru-dover på om interventionen bidrog til at opfyld kursets målbeskrivelse.


På baggrund af mine egne observationer under interventionen, evaluerin-gerne i plenum efter interventionerne samt de to interviews kan dragesnogle mere eller mindre generelle betragtninger omkring den afholdte in-tervention og de studerendes opfattelse af interventionen i forhold til deresopfattelse af egenlæring. Adspurgt om hvordan de oplevede peer feedback-situationen svarede alle studerende at det havde været en positiv oplevelse.Årsagen til den generelt positive evaluering af interventionen er dog ikkeentydig og vil kræve et større kvalitativt datasæt. Dog går tre ting igen i destuderendes kommentarer:

1. Peer feedback-formen var en ny måde at behandle en faglig problem-stilling og det var en ny måde at få undervisning på hvilket blev tillagten positiv værdi ”. . . Jeg har ikke prøvet denne form for undervisningfør, altså hvor man skal give hinanden feedback på en opgave, hverkeni gymnasiet eller her på universitetet. Jeg synes det er forfriskende atdenne øvelse er anderledes” (citat fra studerende givet ved plenume-valueringen).

2. Som evalueringsform gav den individuelle peer feedback-del (editingand redrafting) bedre mulighed for fordybelse og at forstå fagstoffetbag øvelsen samt at løse øvelsen. Dette skal ses i modsætning til denevalueringsform de studerende har mødt tidligere på kurset, hvor engruppe af studerende i plenum fremlægger resultaterne af en øvelse forde resterende studerende på holdet. ”. . . Når jeg sad der hjemme oglæste de andres besvarelser, forstod jeg bedre svarende på spørgsmåle-ne. Når jeg sidder til en øvelse og høre en anden gruppe fremlægge, erdet nogen gange svært at følge med. Jeg tror det handler om, at jeg harmere tid til at arbejde med evalueringen når jeg sidder alene.” (citatfra interview med studerende).

3. De studerende fandt det generelt positivt både at modtage og givemundtligt feedback (co-revising) til medstuderende. Der var mindre


uro forbundet med denne feedback-form da det uddannelsesmæssig ni-veau er ens i modsætning til feedback mellem underviser og studeren-de. ”Når vi sad i grupper og snakkede om hinandens opgaver følte jeg,at det var mindre angstprovokerende at få feedback på vores opgavei forhold til når man får feedback af en underviser”. ”. . . på noglepunkter tror jeg, at det er lettere at forstå den feedback man får af enmedstuderende end fra en underviser. Det hænger nok sammen med atvi studerende bruger de samme ord og termer når vi taler sammen”(citat fra interview med studerende).

De kvalitative data peger altså på at de studerende er af de opfattelse afat feedbackøvelsen har givet dem et faglige udbytte, hvilket er i tråd medblandt en lang række af studer der dokumenterer de positive effekter af peerfeedback (Liu et al., 2006; Pearce et al., 2009; Topping, 2009; Van den Berget al., 2006).

På trods af den generelle positive evaluering af interventionen, kritise-rede flere studerende også feedback-øvelsen. En del af denne kritik rettedesig imod, at de studerende følte sig usikre på, hvordan denne for dem nyeundervisningsform skulle gribes an samt at det umiddelbart ikke var klart,hvordan feedback-øvelsen bidrog til at nå kursets målbeskrivelse. Dette un-derstreger, at det er vigtigt at forberede de studerende grundigt inden selvefeedback-øvelsen (Pearce et al., 2009). En anden kritik som blev fremført atenkelte studerende handlede om, at man som studerende vægter feedbackfra underviser højere end feedback fra med-studerende. Dette er også be-skrevet i Topping (2009) og det understreger at nogle studerende har en be-stemt opfattelse af, hvilke rolle underviseren burde indtage og hvilke rolleden studerende burde indtage (Pearce et al., 2009). Det er dog min forvent-ning, at jo flere gange de studerend deltager i peer feedback-sessioner, jomere bekendte vil de blive med denne undervisningsform og de fordele derknytter sig læringsmæssigt til at blive evalueret af ligemænd.

Konklusion og perspektivering

I dette projekt har jeg undersøgt hvordan peer-feedback i holdundervisnin-gen i kurset Globale Geosystemer påvirker de studerendes opfattelse af de-res egenlæring. Basseret på egne observationer, mundtlig evaluering og in-terviewes kan det konkluderes at de studerende generelt fandt at brugen afpeer feedback i undervisningen øgede deres egenlæring og derfor deres ev-ne til at opfylde kursets målbeskrivelse. Samtidig muliggjorde brugen af


peer feedback i undervisningen at de studerende kunne få feedback på enskriftelig opgave uden at dette betød en væsentlig større mængde forbere-delsestid for underviseren. Der er dog også en række udfordringer i forholdtil anvendelsen af peer feedback i undervisningen. I den nærværende in-tervention har den største udfordring været at sikre at de studerende giverhinanden korrekt feedback i forhold til det faglige indhold. Det vil sige ikkegiver feedback som er faktuelt forkert. Dette er svært at kontrollere da mansom underviser med peer feedback afgiver en del af ansvaret for undervis-ningen til de studerende. Samtidig blev det også fremhævet i de studerensevaluering af interventionen af nogle studerende ikke opfatter feedback fraen med-studerende som så vigtig, som hvis den samme feedback blev givetaf en underviser. Det er desuden vigtigt, at de studerende instrueres grun-digt inden feedback-sessions, således de føler sig ”klædt på” til at give ogmodtage feedback fra medstuderende.

Jeg mener at perspektiverne for anvendelse af peer feedback i undervis-ningen på universitetet er gode og jeg vil blive ved med at anvende denneevalueringsform i min undervisning. For den aktuelle øvelse, vil jeg dogvære mere opmærksom på at lave en klare rammesætning for feedback-øvelsen, samt forberede de studerende mere grundigt på, hvordan peer feed-back anvendes til at nå kursets målbeskrivelse.


Referencer

Biggs, J., & Tang, C. (2007). Teaching for quality learning at universityMaidenhead. Berkshire, UK: McGraw-Hill Education.

Brindley, C., & Scoffield, S. (1998). Peer assessment in undergraduate pro-grammes. Teaching in higher education, 3(1), 79-90.

Liu, N. F., & Carless, D. (2006). Peer feedback: the learning element ofpeer assessment. Teaching in Higher education, 11(3), 279-290.

Pearce, J., Mulder, R., & Baik, C. (2009). Involving students in peer review:Case studies and practical strategies for university teaching. Centre for theStudy of Higher Education, University of Melbourne.


Topping, K. J. (2009). Peer assessment. Theory into practice, 48(1), 20-27.

The University of Edinburg (2018). Involving students in feedback.http://www.enhancingfeedback.ed.ac.uk/staff/resources/involvingstudents.html

Van den Berg, I., Admiraal, W., & Pilot, A. (2006). Design principles andoutcomes of peer assessment in higher education. Studies in Higher educa-tion, 31(03), 341-356.


A Den officielle kursusbeskrivelse

Øvelse 3

Læringsmål: Efter øvelsen vil I: have forståelse for de vigtigste naturlige og menneskeskabte parametre der

kontrollerer den relative havspejlsudvikling samt sedimentationen i to typer afkystnære vådområder.

have kendskab til litteratursøgning igennem Web of Science og Scopus. Være i stand til at evaluere og give konstruktiv feedback på andres skriftlige

arbejde. Være i stand til at modtage konstruktiv feedback

Øvelsen er todelt: Del 1 Hver gruppe skal udarbejde en lille skriftlig rapport, hvor nedenstående spørgsmål ønskes besvaret. Opgaven afleveres til underviseren og en opponentgruppe. Format for aflevering er et word-dokument.

Del 2 Opponentgruppen giver skriftelig feedback på opgavebesvarelsen vha. kommentering og track changes-funktionen i Word.

Omfang: ca. 2 siders tekst + figurer og referencer.

Opgaven skal indeholde en kort opsummering af konsekvenserne af en havspejlsstigning for det pågældende kystnære vådområde:

Mississippi deltaet i USA Lagunen ved Venedig i Italien

Følgende spørgsmål skal besvares i opgaven: Hvilke parametre er tilsammen med til at give den relative havspejlsstigning i det

pågældende område? Hvad er havspejlsstigningen i det pågældende vådområde? Hvad er sedimentationsraten i det pågældende vådområde? Kan sedimentationsraten matche havspejlsstigningen? Er der på nuværende tidspunkt tegn på at havspejlsstigningen påvirker vådområdet?

Hvis ja, hvilke?


B Den skriftelige opgaveformulering

Hvilke parametre er med til at kontrollere sedimentationsraten i det pågældendevådområde?

Deadline Del 1 Hold 3 (mandagsholdet) torsdag d. 6. dec.; Hold 1 (tirsdagsholdet) fredag d. 7. dec.;

Hold 2 (fredagsholdet) 11. dec. Opgaven afleveres til underviseren og til en opponentgruppe via grupperummet på

Absalon

Del 2 Skriftelig evaluering og feedback skal være færdig, til vi mødes til øvelsesgang 4.

OBS Mindst 4 af spørgsmålene skal besvares ved hjælp af litteratur fundet via Web of

Science eller SCOPUS. I er meget velkomne til også at bruge GIS.

Øvelse 3

Vejledning til at evaluere og give feedback på skriftelige aflevering + til brug af Word ”registrer ændringer”-funktionen (track changes, eng.)

Feedback og evaluering

Feedback, om det er mundtlig eller skriftelig, skal være positiv og specifik. Stil åbne spørgsmål og udgå negativ sprogbrug. Giv forslag til ændringer.

Eksempel på en kommenteret tekst ”…den havspejlsstigning som finder sted i lagunen ved Venedig er resultatet af den globale havspejlsstigning (ref) og fordi de finkornede lagunesedimenter kompakterer og synker sammen (ref), (se figur/tabel xx).”

Giv feedback på følgende punkter:

Struktur


C Information om feedback på skriftelig aflevering

• Opsætning• Er figurer og tabeller kvalitetsmæssig i orden og med forklarende figurtekst. I grafer,

er der forklaringer til akserne med enheder.• Er referencelisten opsat korrekt, er den i alfabetisk rækkefølge. Er alle referencer i

teksten også i referencelisten og omvendt.

Indhold • Er der svaret fyldestgørende på spørgsmålene fra opgaveformulerings• Er der argumenteret for svarende med brug af referencer til videnskabelig tekster• Er der figurer og tabeller til at underbygge påstande og beskrivelser• Er der noget som I synes mangler eller har I forslag til andre vinkler på besvarelsen

Brug af Word’s registrer ændringer (track changes)-funktion og kommenter- funktion (comments).

Disse funktioner findes under fanebladet Review. For at skrive en kommentar til tekststykke markeres først teksten, hvorefter der trykkes på New Comment. For at lave track chages aktivers denne funktion først ved at trykke på Track Changes knappen. Til højre for Track Changes-knappen skal i fra drop-down menuen væge All Markup for at se jeres kommentarer.


D EvalueringsspørgsmålUniversitetspædagogikum 2018 Mikkel Fruergaard

14

Bilag 4 – Evalueringsspørgsmål 1. Hvordan oplevede du peer-feedback situation? 2. Hvordan oplevede du at skulle give feedback til dine medstuderende? 3. Hvordan oplevede du at modtage feedback fra dine medstuderende? 4. I forhold til dit faglige udbytte, hvordan vurderer du anvendelsen af peer-feedback i

undervisningen? 5. Var det læringsmæssige formål med peer-feedback øvelsen klar for dig? 6. Er peer-feedback en undervisningsform som du vil foretrække anvendt mere i din

universitetsundervisning – begrund dit svar?

17

Udvikling af et peer-2-peer feedback forløb

Sandra Gentin

Institut for Geovidenskab og NaturforvaltningKøbenhavns Universitet

Indledning og problemstilling

Som del af min undervisning bruger jeg peer2peer feedback. Det kan væresom midtvejsseminar i et bachelorforløb, eller som del af et eksamensopga-veforløb, hvor de studerende skal give formativ feedback på hinandens ar-bejde, eller når de studerende giver hinanden formativ feedback i de mangemindre og større mundtlige formidlingsopgaver de laver i løbet af studieti-den. Peer2peer feedback er altså noget de studerende møder gennem helederes uddannelse som Natur- og Kulturformidler. Den måde feedback givespå kan være noget de studerende kan være meget utilfredse med (Boud etal., 2013), mange uddannelser har derfor arbejdet med at forbedre ramme-sætningen af feedback til studerende (Nicol et al., 2014).

På Natur- og Kulturformidler uddannelsen er feedback seminarerne eroftest opbygget således at de studerende inddeles i grupper fremlægger enproblemstilling, spørgsmål ell. lign, og som de andre i gruppen efterføl-gende giver formativ feedback på. Såfremt det er nødvendigt, tilføjer jegafslutningsvis noget, ellers understøtter jeg blot den givne feedback. Inten-tionen er, at de studerende på den måde får flere og andre perspektiver påderes problemstilling end hvis de kun taler med underviserne. Mit eget ud-gangspunkt for at benytte mig af peer2peer feedback er at jeg ofte opleverde studerende tager den feedback de får af deres medstuderende mere tilsig, end hvis jeg sagde (præcis) det samme. Topping (1998) fremhæver, atstuderende ofte forstår den feedback de modtager af deres medstuderendebedre, end hvis en underviser giver feedback, ligesom der oftest gives me-re og mere varierende feedback af de studerende selv, end hvis det kun er

198 Sandra Gentin

underviserend der giver feedback. Derudover trænes de studerende også iat tænke kritisk, begrunde deres kritik og reflektere over andres (og eget)arbejde (Dochy et al., 1999; Nicol et al., 2014; Wanner et al., 2018).

Udfordringen er imidlertid at de studerende ofte vælger denne form forfeedback fra, og bliver væk fra undervisningen når der står ”peer2peer feed-back” på programmet, blandt andet fordi de som de selv siger ”ikke følerat de får noget ud af det” eller sat endnu mere på spidsen ” hellere vil havedin vejledning end den feedback jeg kan få fra mine medstuderende”. Detkan skyldes en for utydelig rammesætning af formativ feedback, det kanhandle om uvidenhed om hvad det er, hvordan man giver og modtager det,og også hvad man gør ved feedbacken efterfølgende. Undervisning i ellerafklaring af hvad formativ feedback er, hvordan det gives og hvordan manarbejder med den feedback man har fået heller er ikke en del af de forskelli-ge undervisningsplaner på uddannelsen (Københavns Professionshøjskole,2018). Det virker altså med andre ord sådan, at vi som undervisere forven-ter de studerende ved hvad feedback er og hvordan man giver og modtagerdet, mens de studerende måske ikke er helt klar over formålet og formen,og det videre arbejde med feedbacken.

Der er et stort læringspotentiale i et godt rammesat peer2peer feedbackforløb (Poulos et al., 2008). Det kræver dog at feedback-frekvensen skalvære tilstrækkelig ofte, feedbacken skal fokuseres på de studerendes læ-ring og de handlinger relateret til hvad de studerende kan gøre noget ved,frem for at fremføre personlig kritik; tids nok til at de studerende har hand-lemuligheder pba den givne feedback og passende til niveauet af opgaven(Gielen et al., 2010). En af årsagerne til at peer2peer feedback har et stortpotentiale for at øge de studerendes læring er at de studerende oftere dis-kuterer feedbacken med hinanden og indgår i dialog om fortolkningen afhvad der menes, sammenlignet med hvis feedbacken gives af underviserentil de studerende (Dochy et al., 1999). Derudover fremhæver Nicol et al(2014, p. 105) også at de studerende for en bedre forståelse af deres egenskrivestil (og udfordringerne i at formidle skriftligt) når de studerende læserog kommenterer andres tekster. Dette underbygges også af de studerendeder rent faktisk møder op til feedback-sessionerne, da de nærmest sammen-stemmende siger at de virkelig har fået meget ud af det, især når de har haften tydelig problemstilling de ønskede at diskutere med deres medstuderen-de – imens de studerende der har haft vage problemstillinger, ofte opleverpeer2peer sessionerne som ikke udbytterigt, forvirrende eller ligefrem de-motiverende.

17 Udvikling af et peer-2-peer feedback forløb 199

Problemformulering

Peer2peer feedback benyttes på mange kurser og i mange forskellige sam-menhænge på Natur- og Kulturformidler uddannelsen. Der er imidlertiduklart hvordan de studerende forberedes på at give peer2peer feedback, oghvordan de arbejder videre med den, og hvad udbytte de føler de får af atgive og modtage peer2peer feedback. Det er derfor interessant at undersøgehvordan man som underviser kan facilitere og forberede de studerende pået peer2peer feedback forløb, så de kan give hinanden feedback og arbej-de videre med den feedback de har fået, og derigennem opfatte peer2peerforløbet som udbytterigt.

Det er derfor formålet med denne afsluttende opgave at udvikle et un-dervisningsforløb der har til formål at rammesætte et peer2peer feedbackforløb hvor der arbejdes med mundtlig peer2peer feedback, evaluere detkvalitativt med de studerende der har deltaget og efterfølgende tilpasse un-dervisningsforløbet.

Følgende spørgsmål søges besvaret:

• Hvad er de studerendes erfaringer med peer2peer feedback?• Hvordan kan et peer2peer feedback forløb rammesættes og tilrettelæg-

ges så det giver mere mening for de studerende, og hvordan opfatter destuderende deres læring gennem peer2peer forløbet?

Feedback og peer2peer feedback

Denne opgave læner sig op ad Hattie & Timperleys (2007) definition affeedback: ”as information by an agent regarding aspects of one’s perfor-mance or understanding”. Feedback kan komme både fra undervisere, for-ældre, medstuderende mv. den største forskel mellem underviser-feedbackog studenter-feedback er, at de studerende ikke er de faglige eksperter, somunderviseren er. Derfor kan kvaliteten af p2p feedback variere – ikke destomindre kan p2p feedback stadig give et højt læringsudbytte (Gielen et al.,2010). En af årsagerne hertil kan være, at feedback fra underviserne kanvære sværere at forstå for de studerende, mens de har lettere ved at for-stå den feedback de modtager af deres medstuderende. Samtidig kan denusikkerhed der baserer sig på manglende viden om et emne i et p2p feed-back forløb reslutere i ”to search for confrimation by checking instructionmanuals, asking the teacher, and/or performing more self-corrections. As aresult the students acquired a deeper understanding of the subject.” (Gielen

200 Sandra Gentin

et al., 2010, p. 305). Samme søgen for løsninger på uklarheder gør sig ikkegældende når feedbacken gives af underviseren.

Ifølge Dochy, Segers & Sluijsmans (1999) øger peer2peer feedback læ-ringsudbyttet blandt de studerende på en række punkter: 1) de studerendefår større selvtillid ifht. til deres egen præstationsevne; 2) de studerende bli-ver mere opmærksomme på kvaliteten i deres eget arbejdet; 3) højere kva-litet i de studerendes eget arbejde; 4) de studerende tager ansvar for egenlæring; 5) større tilfredshed med opgaven og den feedback der gives. Der-for fremhæver Nicol et al. (2014) at vi som undervisere skal bevæge os vækfra de ”gamle feedback opfattelser” hvor det primært er underviser-drevenfeedback til en mere aktiv involvering af de studerende gennem p2p feed-back forløb. For at dette kan lade sig gøre, kræver det dog, at de studerendefår viden om hvordan man giver og modtager konstruktiv feedback.

Det meningsgivende feedbackforløb

De studerendes viden om hvad feedback er, hvad man skal gøre med den- og hvordan og hvornår det gives er spørgsmål man som underviser skalforholde sig til, ikke bare på uddannelsen generelt men også på fag-basis.Uden denne italesættelse kan man ellers ikke være sikker på hvorvidt der eren fælles forståelse for feedback i undervisningen (The University of Edin-burgh, 2010). Dette er også i tråd med Sadler (1989, p. 78) der fremhæver”It cannot simply be assumed that when students ’are given’ feedback theywill know what to do with it”. Samtidig kan feedback være en af de mesteffektive måder til at styrke de studerendes læring på (Nicol et al., 2014;Poulos et al., 2008). Dog afhænger denne positive effekt af feedback afflere faktorer – blandt andet bør de studerende spille en aktiv rolle i feed-back processen (Nicol et al., 2014), men placeringen af feedbacken er ogsåvigtig: ”the word effective in the context of feedback has been associatedwith feedback that is both appropriate and timely” (Poulos et al., 2008, p.143). For at øge de studerendes læringsudbytte skal placeringen af feedba-cken give mening i forhold til det produkt der skal gives feedback på. Herhar p2p feedback en særlig fordel, idet p2p feedback ofte placeres midt i etopgave-skrivnings forløb. Et eksempel er at studerende afleverer et halvfær-digt kapiteludkast til deres medstuderende som de modtager feedback på,og kan efterfølgende arbejde videre med kommentarerne og aflevere denfærdige opgave til underviseren. På den måde kan de studerende anvendeden modtagne feedback mere direkte på deres arbejde. Samme mulighedgør sig ikke altid gældende når de studerende modtager underviser feed-


back, da der er evidens for, at den feedback de studerende modtager afunderviserne ofte ikke kan anvendes direkte men kræver en vis bearbejd-ning da feedbacken kan være for kompleks når den gives af en underviser(Nicol et al., 2014).

For at imødekomme det meningsgivende feedback og tydeliggøre hvor-når der gives feedback på hvad er det vigtigt at man som underviser ram-mersætter den feedback de studerende kan få – det kan gøres ved at plan-lægge hele feedbackforløbet for hele kurset inden undervisningsstart og in-darbejde det i undervisningsplanen til de studerende (her kan det fx frem-gå hvem giver feedback og på hvad, og med hvilket fokus) (Rienecker etal., 2013). Derudover fremhæver Hounsell, McCune, Hounsell, & Litjens(2008) at det også er væsentligt at italesætte overfor de studerende, hvadder er ”høj kvalitet” i den opgave eller det produkt de i sidste ende skal af-levere. Dette skal gøres for at give en forståelse af hvad de studerende skalbedømmes og bliver bedømt på.

Hounsell et al. (2008) fremhæver at det kan være en fordel, at opfat-te feedback som et ”loop”, og at der i denne ”loop” er flere trin og faser,som man som underviser skal forholde sig til (se figur 17.1). I forbindelsemed opgaveskrivningen kan der allerede indledningsvis opstå udfordrin-ger, eftersom især nye studerende (1. semester studerende) måske manglererfaringer med de specifikke opgavetyper der skal afleveres. Dette kan imø-dekommes ved at lade de studerende se på tidligere opgaver, men også vedat være tydelig i kursets læringsmål. Endivdere kan de studerende føle deikke vil eller kan forstyrre vejleder og der dermed er relativ ringe adgangtil løbende vejledning og sparring. Her kan et p2p feedback forløb være gi-vende for den studerende. Når opgaven er afleveret, kan udfordringen være,at der enten er lidt eller ingen feedback på produktet – eller feedbacken ik-ke er konstruktiv. Det kan være demotiverende for den studerende. Somunderviser er det her væsentligt at man afsætter tilstrækkelig tid til at giveisær den formative feedback (også selvom karakteren er givet), for det erbegrundelserne og den konstruktive feedback der danner grundlag for ly-sten til at skrive den næste opgave (Hounsell et al., 2008; Rienecker et al.,2013).

202 Sandra Gentin

4

Figur 1: Feedback loopen - og de forskellige faser i feedbackprocessen. (Hounsell et al., 2008)

Læring I dette projekt forstås læring ikke kun som en kognitiv og individuel proces, men også i høj grad som social

proces. Dermed læner jeg mig op ad Illeris’ (2000) definition af læring, der ”altid på én gang er en kognitiv,

en psykodynamisk og en social, samfundsmæssig proces” (p. 19), og kan illustreres på føglende måde:

Figur 2: Læringstrekanten. Læring som kognitiv, psykodynamisk og samfundsmæssig proces. (Illeris, 2000, p. 19)

Ifølge Illeris (2000) omfatter læring tre forskellige dimensioner: 1) læring som kognitiv proces; 2) læring

som psykodynamisk proces; og 3) læring som social proces (se figur 2). De tre dimensioner indgår alle tre

som integreret form i al læring, der foregår, og kan ikke adskilles. Den kognitive proces udgøres af en

tilegnelse af færdigheder og forståelse af fagligt stof. Læring som psykodynamisk proces omfatter følelser,

Figur 17.1. Feedback loopen - og de forskellige faser i feedbackprocessen. (Houn-sell et al., 2008)

Læring

I dette projekt forstås læring ikke kun som en kognitiv og individuel proces,men også i høj grad som social proces. Dermed læner jeg mig op ad Illeris’(2000) definition af læring, der ”altid på én gang er en kognitiv, en psyko-dynamisk og en social, samfundsmæssig proces” (p. 19), og kan illustrerespå føglende måde:


4

Figur 1: Feedback loopen - og de forskellige faser i feedbackprocessen. (Hounsell et al., 2008)

Læring I dette projekt forstås læring ikke kun som en kognitiv og individuel proces, men også i høj grad som social

proces. Dermed læner jeg mig op ad Illeris’ (2000) definition af læring, der ”altid på én gang er en kognitiv,

en psykodynamisk og en social, samfundsmæssig proces” (p. 19), og kan illustreres på føglende måde:

Figur 2: Læringstrekanten. Læring som kognitiv, psykodynamisk og samfundsmæssig proces. (Illeris, 2000, p. 19)

Ifølge Illeris (2000) omfatter læring tre forskellige dimensioner: 1) læring som kognitiv proces; 2) læring

som psykodynamisk proces; og 3) læring som social proces (se figur 2). De tre dimensioner indgår alle tre

som integreret form i al læring, der foregår, og kan ikke adskilles. Den kognitive proces udgøres af en

tilegnelse af færdigheder og forståelse af fagligt stof. Læring som psykodynamisk proces omfatter følelser,

Figur 17.2. Læringstrekanten. Læring som kognitiv, psykodynamisk og samfunds-mæssig proces. (Illeris, 2000, p. 19)

Ifølge Illeris (2000) omfatter læring tre forskellige dimensioner: 1) læ-ring som kognitiv proces; 2) læring som psykodynamisk proces; og 3) læ-ring som social proces (se figur 17.2). De tre dimensioner indgår alle tresom integreret form i al læring, der foregår, og kan ikke adskilles. Denkognitive proces udgøres af en tilegnelse af færdigheder og forståelse affagligt stof. Læring som psykodynamisk proces omfatter følelser, holdnin-ger og motivation – her vil læringen både påvirke og påvirkes af den psy-kiske energi undervisningen mobiliserer. Læringsprocessen drives i detteperspektiv af de følelser og holdninger man har til opgaven (og undervise-ren). Læring som social og samfundsmæssig proces henviser til to dele aflæringen – den første: læring som social proces refererer til det mellemmen-neskelige sammenspil, mens referencen til det samfundsmæssige relaterersig til den enkeltes medvirken som samfundsborgere.

Læring som kognitiv proces forklarer Illeris (2000) med udgangspunkt iPiaget. Piaget’s udgangspunkt er ”den konstruktivistiske opfattelse der gårud på at mennesket gennem læring og erkendelse selv konstruerer sin for-ståelse af omverdenen” (cf. Illeris, 2000, p. 26). Det betyder samtidig atman som underviser ikke ”fylder på” eller overfører viden til sine studeren-de, men at de selv må arbejde med stoffet. For Piaget er det centralt at derforegår en assimilation – altså at den studerende bygger oven på sin allere-de eksisterende viden (kognitive skemaer), og tilføjer således noget til det

204 Sandra Gentin

allerede erkendte. Denne tilføjelse sker ved akkomodation, hvor den stude-rende ændrer sine allerede erkendte strukturer ifht. det nye stof. Det skeri en omstrukturering, hvor det nye stof indarbejdes i de allerede eksiste-rende strukturer. Akkomodative processer kan både være hurtige hvor manoplever at 25 øren falder, men kan også være længerevarende processer,hvor man ”tumler med” en problemstilling over længere tid. Da skemaerneer individ-afhængige, vil både assimilationerne og akkomodationerne væreforskellige selvom de studerende udsættes for samme undervisning. Medugangspunkt i Piaget og den akkomodative læring vil især refleksionen, be-grundelserne og den mundlige formulering af pointerne i en p2p situationbidrage til en yderligere bearbejdning af stoffet, og dermed øge de stude-rendes læringsudbytte.

Læring består ikke af en transmission af viden fra underviser til stude-rende men gennem kommunikative relationer mellem underviser og stu-derende. Kommunikation kan ifølge Mathiasen (2008) opfattes som treforskellige selektioner (se figur 17.3): 1) underviserens selektion af infor-mation og 2) underviserens valg af meddelelsen. Herefter foretages den 3.selektion: modtagerens forståelse. Mens de første to selektioner er synlige,er den tredje selektion usynlig, da forståelsen foregår internt i modtageren.Dermed er der ikke nødvendigvis overensstemmelse mellem afsenderensintention af information og meddelelsen og modtagerens forståelse af in-formationen (Mathiasen, 2008). Flere forskere peger på, at de studerendehar lettere ved at forstå den feedback de modtager af deres medstuderen-de, end af underviseren, hvilket kan tyde på at de studerende har nemmereved at forstå hinandens kommunikationsenheder i fx en feedbacksituation,sammenlignet med kommunikationsenheden underviser-studerende.

5

holdninger og motivation – her vil læringen både påvirke og påvirkes af den psykiske energi undervisningen

mobiliserer. Læringsprocessen drives i dette perspektiv af de følelser og holdninger man har til opgaven (og

underviseren). Læring som social og samfundsmæssig proces henviser til to dele af læringen – den første:

læring som social proces refererer til det mellemmenneskelige sammenspil, mens referencen til det

samfundsmæssige relaterer sig til den enkeltes medvirken som samfundsborgere.

Læring som kognitiv proces forklarer Illeris (2000) med udgangspunkt i Piaget. Piaget’s udgangspunkt er

”den konstruktivistiske opfattelse der går ud på at mennesket gennem læring og erkendelse selv konstruerer

sin forståelse af omverdenen” (cf. Illeris, 2000, p. 26). Det betyder samtidig at man som underviser ikke

”fylder på” eller overfører viden til sine studerende, men at de selv må arbejde med stoffet. For Piaget er

det centralt at der foregår en assimilation – altså at den studerende bygger oven på sin allerede

eksisterende viden (kognitive skemaer), og tilføjer således noget til det allerede erkendte. Denne tilføjelse

sker ved akkomodation, hvor den studerende ændrer sine allerede erkendte strukturer ifht. det nye stof.

Det sker i en omstrukturering, hvor det nye stof indarbejdes i de allerede eksisterende strukturer.

Akkomodative processer kan både være hurtige hvor man oplever at 25 øren falder, men kan også være

længerevarende processer, hvor man ”tumler med” en problemstilling over længere tid. Da skemaerne er

individ-afhængige, vil både assimilationerne og akkomodationerne være forskellige selvom de studerende

udsættes for samme undervisning. Med ugangspunkt i Piaget og den akkomodative læring vil især

refleksionen, begrundelserne og den mundlige formulering af pointerne i en p2p situation bidrage til en

yderligere bearbejdning af stoffet, og dermed øge de studerendes læringsudbytte.

Læring består ikke af en transmission af viden fra underviser til studerende men gennem kommunikative

relationer mellem underviser og studerende. Kommunikation kan ifølge Mathiasen (2008) opfattes som tre

forskellige selektioner (se figur 3): 1) underviserens selektion af information og 2) underviserens valg af

meddelelsen. Herefter foretages den 3. selektion: modtagerens forståelse. Mens de første to selektioner er

synlige, er den tredje selektion usynlig, da forståelsen foregår internt i modtageren. Dermed er der ikke

nødvendigvis overensstemmelse mellem afsenderens intention af information og meddelelsen og

modtagerens forståelse af informationen (Mathiasen 2008). Flere forskere peger på, at de studerende har

lettere ved at forstå den feedback de modtager af deres medstuderende, end af underviseren, hvilket kan

tyde på at de studerende har nemmere ved at forstå hinandens kommunikationsenheder i fx en

feedbacksituation, sammenlignet med kommunikationsenheden underviser-studerende.

Figur 3: Sammenhængen mellem tre selektioner i en kommunikationsenhed (Mathiasen, 2008).

Metode For at finde svar på spørgsmålene i problemformuleringen gennemføres der et fokusgruppeinterview med

studerende på 1. semester af Natur- og Kulturformidleruddannelsen. Fokusgruppeinterviewet skal give en

indsigt i de studerendes erfaringer med p2p feedback samt afdække hvad hvilke rammer de studerende

opfatter som meningsgivende i et p2p feedback forløb. Dette bruges, sammen med litteraturen om p2p

Figur 17.3. Sammenhængen mellem tre selektioner i en kommunikationsenhed(Mathiasen, 2008).


Metode

For at finde svar på spørgsmålene i problemformuleringen gennemføres deret fokusgruppeinterview med studerende på 1. semester af Natur- og Kul-turformidleruddannelsen. Fokusgruppeinterviewet skal give en indsigt i destuderendes erfaringer med p2p feedback samt afdække hvad hvilke ram-mer de studerende opfatter som meningsgivende i et p2p feedback forløb.Dette bruges, sammen med litteraturen om p2p feedback som afsæt til enintroduktion om p2p feedback i form af en halv times undervisning med re-fleksionsøvelser for de studerende. Formålet med undervisningsforløbet ogintroduktionen til p2p feedback er at rammesætte to p2p feedback sessioneri fagene ”Naturforståelse” og ”Kulturforståelse”.

Fokusgruppeinterview

Fokusgruppe interview med fire studerende (Peter, Hannah, Sofie og Sara)blev afholdt inden interventionen. De fire studerende blev valgt på bag-grund af deres tidligere erfaringer med p2p feedback, hvor nogle har megeterfaring, som tidligere RUC studerende, mens andre er kommet direkte fragymnasiet, og derfor kun har prøvet p2p feedback på deres nuværende ud-dannelse eller i gymnasiet.

Interviewet blev afholdt som et semistruktureret interview med en te-matisk interviewguide. Temaerne i fokusgruppeinterviewet tog afsæt i pro-blemformuleringen: 1) de studerendes opfattelse af hvad p2p feedback erog hvad formålet med det er; 2) de studerendes erfaringer med p2p feed-back – både de gode og de dårlige erfaringer og 3) hvordan et p2p feedbackforløb kunne rammesættes for at give mening.

Interventionen

Med afsæt i litteraturen og fokusgruppeinterviewet med de studerende blevder udviklet et peer-2-peer feedback forløb i to forskellige fag. Her beskri-ver jeg først mine indledende overvejelser, herefter oplægget til de stude-rende, efterfølgende de to peer-2-peer feedback forløb i fagene ”Naturfor-ståelse” og ”Kulturforståelse”.

206 Sandra Gentin

Indledende overvejelser om læringsrummet og peer2peer feedback

Jeg har aldrig været i tvivl om at dette hold ville være det helt rette hold forat afprøve et p2p forløb med. Det er socialt meget velfungerende, og rum-meligt – holdet bakker op om fagligt svagere studerende, og der er generelten god stemning på holdet. Dette bakkes også op af andre undervisere påårgangen.

Som nævnt tidligere er den sociale dimension ifølge Illeris (2000) en delaf læringstrekanten. Dette fremhæves også blandt andet af de studerende ifokusgruppeinterviewet: ”Jeg tænker og tror helt klart når man snakker omfeedback så blotter man sig, især når man fremlægger et ufærdigt produkt.Vi kunne se på Anders hvor svært de havde ved at fremlægge et ufærdigtprodukt. Og hvis ikke det sociale rum er godt, så har man og man udstillersig selv - rent fysisk - det fysiske rum spiller virkelig ind på alt hvad vi gør[. . . ] og det sociale er vigtigt når man kommer ud af sin comfort zone isådan en feedback situation” (Peter). Blandt andet fordi der er rigtig godstemning på holdet – det fungerer godt socialt, kunne jeg godt stille kravom hvordan vi leverer feedbacken, ligesom jeg også kunne udfordre dempå formen, fordi holdet som helhed er velfungerende.

Forberedelse af p2p feedback: Oplæg i undervisningen

Det var formålet med oplægget om p2p feedback at italesætte det overord-nede formål med peer-2-peer feedback. Derudover bidrog oplægget ogsåtil en fælles forståelse af reglerne for p2p feedbacken samt i det hele ta-get øge de studerendes bevidsthed om hvad intentionen med p2p feedbacker. Oplægget rammesatte dermed hele p2p forløbet. Læringsmålene for p2pfeedback forløbet var: 1) forstå rammen for feedback – herunder a) at kunnegive konstruktiv og begrundet feedback, og b) kunne reflektere over mod-taget feedback og identificere egne udviklingspotentialer.

Oplægget indledtes med de studerendes input til ”hvorfor p2p feed-back” her skulle de studerende reflektere over hvorfor p2p kan være me-ningsgivende. De studerende fremhævede her: 1) feedbacken gives af no-gen der står i samme situation som en selv, 2) de medstuderende kender enbedre end læreren og kan dermed målrette feedbacken bedre; 3) det er engod øvelse for den der giver feedback, for man kommer måske til at opdagenogle styrker og svagheder i sit eget arbejde når man spejler sig i andresarbejde; 4) det kan samtidig også tvinge modtageren til at reflektere me-re over sit eget arbejde, 5) det kan give en indsigt i hvad andre på holdet


arbejder med og 6) p2p feedback giver muligheden for at man kan lærerammerne for konstruktiv og begrundet feedback.

De studerendes argumenter blev herefter i klassen sammenlignet medNicol et al. (2014): 1) at feedbacken kan være lettere tilgængelig hvis denkommer fra en medstuderende sammenlignet med når den kommer fra un-derviseren; 2) at der kan gives mere nuanceret feedback når den kommerfra flere medstuderende, sammenlignet med når den kommer fra én under-viser, 3) at følsomme studerende får mere ud af den feedback de får af deresmedstuderende da den kan blive leveret mere omsorgsfuld.

Fokusgruppeinterviewet viste, at de studerende ikke er særlig bevidsteover deres eget læringsudbytte af at give p2p feedback, men at de derimodmener, de lærer noget af at se andres oplæg: – ”det er den måde vi lærer– hvordan man opbygger en opgave eller formidling, oplevelsen af andresopgaver ser jeg som en læring”. Dette fremhæves også i et studie af Nicolet al. (2014) der fremhæver, at læringsudbyttet blandt de studerende dergiver og modtager p2p feedback er størst (se figur 17.3).

7

kommer måske til at opdage nogle styrker og svagheder i sit eget arbejde når man spejler sig i andres

arbejde; 4) det kan samtidig også tvinge modtageren til at reflektere mere over sit eget arbejde, 5) det kan

give en indsigt i hvad andre på holdet arbejder med og 6) p2p feedback giver muligheden for at man kan

lære rammerne for konstruktiv og begrundet feedback.

De studerendes argumenter blev herefter i klassen sammenlignet med Nicol et al. (2014): 1) at feedbacken

kan være lettere tilgængelig hvis den kommer fra en medstuderende sammenlignet med når den kommer

fra underviseren; 2) at der kan gives mere nuanceret feedback når den kommer fra flere medstuderende,

sammenlignet med når den kommer fra én underviser, 3) at følsomme studerende får mere ud af den

feedback de får af deres medstuderende da den kan blive leveret mere omsorgsfuld.

Fokusgruppeinterviewet viste, at de studerende ikke er særlig bevidste over deres eget læringsudbytte af at

give p2p feedback, men at de derimod mener, de lærer noget af at se andres oplæg: – ”det er den måde vi

lærer – hvordan man opbygger en opgave eller formidling, oplevelsen af andres opgaver ser jeg som en

læring”. Dette fremhæves også i et studie af Nicol et al (2014) der fremhæver, at læringsudbyttet blandt de

studerende der giver og modtager p2p feedback er størst (se figur 3).

Figur 4: Eksempel på læringsudbytte fra p2p feedback fra Nicol et al. (2014)

Derfor fokuserede jeg på hvad studerende kan få ud af at give p2p feedback: læringen i at analysere andres

tekster, diagnosticere problematiske forhold i tekster samt foreslå løsninger til problemerne. Dette

relaterede jeg til at de studerende gennem p2p feedback får mulighed for at producere bedre tekster selv,

da de begrundelser og kommentarer vedr. andres styrker og svagheder kan bruges i deres eget arbejde.

Her diskuterede vi i plenum, hvorvidt netop den spejling af andres tekster i eget arbejde er noget de

studerende på holdet benytter sig af. Det var den generelle opfattelse på holdet, at de ikke havde været

bevidst over denne spejling, men at de fremadrettet ville være mere bevidste om denne relation – og at de

ikke havde været opmærksomme på deres eget læringsudbytte på eget arbejde gennem p2p feedback.

Denne diskussion blev understøtte af citater fra Nichol, Thomson & Breslin ”you’ve got what you’ve done in

the back of your mind whilst you’re going over theirs so you see where you’ve gone wrong without anyone

pointing it out so you learn it yourself” (p. 111)

Figur 17.4. Eksempel på læringsudbytte fra p2p feedback fra Nicol et al. (2014).

Derfor fokuserede jeg på hvad studerende kan få ud af at give p2p feed-back: læringen i at analysere andres tekster, diagnosticere problematiskeforhold i tekster samt foreslå løsninger til problemerne. Dette relateredejeg til at de studerende gennem p2p feedback får mulighed for at produ-

208 Sandra Gentin

cere bedre tekster selv, da de begrundelser og kommentarer vedr. andresstyrker og svagheder kan bruges i deres eget arbejde. Her diskuterede vi iplenum, hvorvidt netop den spejling af andres tekster i eget arbejde er no-get de studerende på holdet benytter sig af. Det var den generelle opfattelsepå holdet, at de ikke havde været bevidst over denne spejling, men at defremadrettet ville være mere bevidste om denne relation – og at de ikkehavde været opmærksomme på deres eget læringsudbytte på eget arbejdegennem p2p feedback. Denne diskussion blev understøttet af citater fra Ni-chol, Thomson & Breslin ”you’ve got what you’ve done in the back of yourmind whilst you’re going over theirs so you see where you’ve gone wrongwithout anyone pointing it out so you learn it yourself” (p. 111).

Herefter gennemgik jeg rammesætningen for den feedback de stude-rende skal give hinanden på film de har produceret i faget “Naturforståel-se”. De studerende skulle, med afsæt i faglige fokuspunkter give hinandenfeedback på følgende: 1) ændringer eller tilføjelser og begrunde hvorfor deforeslåede ændringer er vigtige; 2) fremhæve de dele af filmen som kunnehave været uddybet mere og forklar hvorfor, samt 3) Identificer og foreslåen tilføjelse/ændring sok kunne have forbedret produktet og begrund svaret.

De studerende i fokusgruppeinterviewet havde fremhævet hvordan derkan være usikkerhed om hvordan og på hvad der gives feedback: ”Hvis derer en gruppe der giver knap så meget ros, så tager jeg den stilhed som ros –så tænker jeg det er vel fint nok, hvis de ikke siger noget” (Sofie). Det lederdog til følgende refleksion hos Hannah: ”Men det kræver jo at det er aftalt –sådan at man ikke tager personligt – så man aftaler på forhånd, at hvis derer noget vi ikke kommer ind på så er det fordi det er godt. Vi skal heller ikkekun rose hinanden, det er vigtigt vi får konstruktiv kritik – så vi kan forbedreos.” Dette finder genklang hos de andre deltagere i fokusgruppeinterviewet.Derfor aftalte vi rammerne til hvordan man formidler feedbacken – efterhot-dog modellen. Derudover måtte de studerende kun tage udgangspunkt idet de så/det der sker (Hvad ser jeg og hvad hører jeg), og de budskaber derblev leveret skulle være præcise baseret på iagttagelsen og en beskrivelseuden bedømmelse (Christiansen et al., 2008).

I faget ”Kulturforståelse” blev de studerende instrueret til rammesæt-ningen af feedbacken via ”IntraPol”1. De studerende skulle give hinandenfeedback på et mundtligt oplæg, og her fokusere på samme tre fokuspunktersom i ”Naturforståelse”, dog i tilpasset form: 1) Kan du/I foreslå ændrin-ger/tilføjelser og begrunde hvorfor det er vigtigt – både i forhold til pro-

1 Københavns Professionshøjskoles pendant til KU’s Absalon.


blemstilling og handleforslag; 2) Fremhæv de dele af handleforslaget sommåske kunne have været udfoldet på anden vis eller mere og forklar hvor-for; 3) Identificer og foreslå en tilføjelse/ændring som kunne have forbedretproduktet (handleforslaget) og begrund dit svar. For at øge de studerendesfokus på eget udbytte af den modtagne feedback blev de spurgt, hvad detager med sig af den modtagne feedback.

Det praktiske forløb – hvad skete der ved feedbacksessionerne?

I dette afsnit vil jeg redegøre for hvad der faktisk skete på feedbackfor-løbene i hhv ”Naturforståelse” og ”Kulturforståelse”, og afslutte afsnittetmed de studerendes kommentarer og refleksioner om Feedback på Feed-back samt opsamling på feedback – hvad tager vi med os – hvad kan sådanet struktureret peer2peer feedback forløb bibringe os/mig?

Feedback forløbet i faget ”Naturforståelse”

De studerende var på forhånd inddelt i opponentgrupper, og havde på denbaggrund en film de specifikt skulle give feedback på. Hver enkelt feedbackseance blev indledt med at vi i fællesskab så den film der skulle modtagefeedback, så alle havde et kendskab til filmen. Efterfølgende gav oppo-nent gruppen feedback, hvorefter andre studerende kunne byde ind medyderligere kommentarer. Afslutningsvis gav fagets underviser feedback påfilmen, hvorefter jeg med fokus på den givne feedback gav feedback påfeedbacken.

På trods af, at vi havde talt om hot-dog modellen, og der var givet in-struktion i at de studerende skulle give hinanden begrundelser for den kon-struktive kritik var det svært for de studerende at bevæge sig væk fra kunat give ros. Et eksempel herpå var en feedback på en film, hvor enkeltemedlemmer af opponentgruppen havde fremhævet at den speak der havdeværet i filmen havde fungeret godt. Det sidste medlem af gruppen frem-hæver også speaken i sin feedback: ”I eksperimenterer med speaken – ogman bliver fanget af de sindsstemninger, det overrasker, I taler i metaforerog det gør man bliver mere opmærksom på speaken – jeg tror også det erderfor det bliver nævnt af de to andre”. Grunden til at jeg fokuserede såmeget på begrundelserne i feedbacken var, at gøre det synligt for de stu-derende at de også bidrager med en faglighed, som kan være nemmere at

210 Sandra Gentin

forstå for medstuderende end en undervisers feedback, jf. Topping (1998).Derudover ønskede jeg også at træne deres kritiske tænkning og reflektereover andres og eget arbejde på et højere niveau (Dochy et al., 1999; Wanneret al., 2018). Derfor gav jeg de studerende flere benspænd undervejs, for atflytte dem væk fra kun at rose uden begrundelser. Dette bevirkede, at derfremkom flere begrundelser for hvad der fungerede: ”Det virkede godt atI havde redskaber med som viste hvad i talte om – det viser konkret hvadi taler om, så man har noget at kigge på mens man lytter og ser og detfanger godt.” Derudover viste det sig, at det også var udfordrende at givebegrundet konstruktiv feedback, derfor talte vi om at man med fordel kunnebruge ordet ”fordi” i sin feedback, for at tvinge sig selv til at begrunde sinemeninger: ”Det er en sjov film, og vi kender konceptet [om Dr. Pjuskebusk,Ketil og Martin], og det er nogle genkendelige rammer der er med i fortæl-lingen, men i mangler den faglige rammesætning – hvad er det for en skov?I siger er det ikke en fantastisk skov – men hvilke naturtyper er der? Detville have givet en bedre faglig dybde – hvis I havde taget jer mere tid tilat forklare, for det behøver ikke at være overfladisk når man formidler tilbørn – det tager bare mere tid at forklare.” Jo længere vi kom hen i feed-back forløbet, jo tydeligere blev de studerende også i spejlingen af den setefilm i deres eget produkt: ”Musikken fungerede godt – den fadede ind ogud – det skulle vi have tænkt mere over”.

Feedback forløbet i faget ”Kulturforståelse”

Feedback sessionen i faget ”Kulturforståelse” fungerede som en slags midt-vejsseminar på de studerendes eksamensopgave. Ved feedbacksessionenskulle de studerende fremlægge deres problemformulering og et såkaldt”Handleforslag” – et forslag til en handling der kunne løse eller bidrage tilen løsning af det problem de studerende arbejdede med i deres problem-formulering. Formålet med dette forløb var at give input til de studerendesvidere arbejde med eksamensprojekterne. Derfor var der her ekstra fokuspå at give begrundelser for den feedback man ønskede at give. Vi aftaltedog, at hvis man ikke lige helt kan begrunde, så må man gerne bede omhjælp ifht. at finde begrundelser i plenum, så man ikke mister lysten til atsige noget.

En anden ting der blev indarbejdet i dette feedbackforløb var et spørgs-mål om ”hvad tager jeg med mig” da dette blev fremhævet af de studerendei fokusgruppeinterviewet – og at feedbacken på den måde bliver mere brug-bar, hvis man som gruppe får mulighed for at reflektere over den modtagne


feedback: ”jeg tænker også hvis man skal kunne bruge den faglige kritik –så kan man skabe et rum for at evaluere feedbacken i modtagergruppen – atman lige får to minutter til at reflektere over, hvad tager jeg/vi med mig/osher?”.

Rent praktisk forløb denne session på den måde at de studerende igenvar inddelt i opponentgrupper, hvor opponenterne havde ”førsteret” til at gi-ve feedback til oplægsholderne. Dette feedback forløb lå omkring 1½ ugeefter forløbet i ”Naturforståelse” og var dermed det andet struktureret fe-edbackforløb de studerende skulle igennem. Derfor var der i dette forløbmeget fokus på begrundelser og brugen af ordet ”fordi”, ifht. den ros derblev givet: ”Jeg kan godt se jeres videnshul – fordi der er lavet mange un-dersøgelser der underbygger det [druk blandt unge på Roskilde og grænserder bliver overskredet] – hvis man skal gøre noget ved det tal er der jo no-gen der skal stoppe med at drikke. Der skal være et sted hvor man kan tageden snak, det er nogle af de snakke fx i Orange Together, hvor I vil bydeind med et dilemmaspil”. Et andet eksempel på begrundet ros: ”Jeg tænkerHorsens og Slagelse er gode eksempler – fordi de kan sammenlignes, dettaler for at I benytter de byer ifht. at konkretisere handleforslaget – kan ibruge jeres case eksempler på tilsvarende byer hvor der er en negativ fraf-lytning – fordi så vil det blive mere konkret, og I har mulighed for at trækkepå jeres empiri”. Mens et eksempel på en konstruktiv begrundet feedback:”I har ikke sådan en rigtig problemformulering, vi skal vide helt grundlæg-gende hvorfor vi skal have kulturarv og dermed parforce landskabet – fordidet legitimerer hele opgaven”. For nogle studerende var det svært at be-grunde feedbacken – men de blev hjulpet af både underviseren på faget, ogde andre studerende: ”. . . Fordi. . . . Hvis jeg nu siger fællesskab, hvad sigeri så? - sanseligt – æstetisk – byrum – identitet mv.”, på den måde sikredevi stadigt et trygt læringsrum hvori man kunne få hjælp til at begrunde sinfeedback.

På tværs af alle feedback sessionerne tegnede sig et billede af, at det destuderende tog med sig var at være mere konkrete, at være mere specifikke,at tænke mere over hvordan de med et specifikt afsæt kunne gøre opgavenmere håndterbar: ”Vi skal ændre problemformuleringen – fordi Anna Møl-ler [et skib på et museum i Holbæk] skal gøre det mere konkret”. Mensen anden gruppe fokuserede på ”Vi har et godt handleforslag, men vi skaljustere på det og fokusere på en aktivitet, og ikke bruge alle skovene menkun en – det skal justeres lidt – så opgaven ikke er så uoverskueligt”.

212 Sandra Gentin

Afrunding af interventionen

Efter de to p2p feedback sessioner i klassen opsummerede vi udbyttet, førstmed en summeøvelse og efterfølgende delte klassen tankerne i plenum –her diskuterede vi både udbyttet i forhold til læringsmålene, men også iforhold til den opgave de skal skrive i faget ”kulturforståelse” som feedba-cken var rettet i mod.

Her var der enighed om at forløbet ved at fremhæve og kræve begrun-delsen for at give feedback kunne virke ”lidt kasset”, og meget struktureret.Men strukturen omkring forløbet havde også bidraget til at forkorte denfeedback der blev givet, som ellers kan blive lang og til dels ubrugelig,særligt når den er ubegrundet. Med begrundelserne blev ”tydeligheden påden måde fremtvunget”. En anden studerende fremhævede at begrundelser-ne havde alligevel tvunget de studerende til at tage stilling til hvorfor demente som de gjorde: ”Feedbacken bliver ikke omkostningsfrit hvis mantvinges til at sige fordi, ellers bliver det meget hurtigt meget langt”, Sam-tidig har den kraftige rammesætning og fokus på feedbacken været nogetsom de studerende har opfattet positivt: ”det har været fedt med feedbackpå feedbacken – det har gjort at jeg selv har gjort mig mere umage medat levere feedbacken til mine medstuderende. Ellers kan det hurtigt blivesådan bla-bla-agtigt”.

Mange af feedback punkterne kredsede om det samme emne – at pro-blemstillingen og handleforslaget skulle konkretiseres mere, for at blivemere håndterbar for de studerende i opgaven. Dette blev også bemærketaf en studerende ved afrundingen: ”Jeg er begyndt selv at tænke over minegen opgave – mens jeg hørte på de andres oplæg – gør jeg det samme? Harjeg gjort noget lignende?”. Dette blev understøttet af en anden studerendeder synes læringsudbyttet havde været stort fordi hun både havde lyttet tilandres kommentarer, og noteret kritikpunkter på andres arbejde, hvor kri-tikken passede til hendes eget projekt. På den måde læner de studerendesrefleksioner sig meget godt op ad Nicol et al. (2014) studie, der har under-søgt studenterperspektivet i p2p feedback, hvori det fremhæves ”I had achance to see others pople’s work and aspects of their work that I felt werelacking in my work – this helped me to improve my work”.

Jeg diskuterede som nævnt hele interventionen i en fælles plenumøvel-se i klassen efter det sidste p2p feedback forløb i faget ”Kulturforståelse”.Selvom diskussionen og opsummeringen blev indledt med en summeøvelsetil at give de studerende rum for at formulere deres tanker i mindre grupperkan der stadig være aspekter af forløbet som de studerende ikke ønsker at


fremlægge i plenum. Det kan fx være specifikke tanker om nogens feedba-ckform eller andre mere ømtålelige emner.

Konklusion

Det er formålet med denne afsluttende opgave at udvikle et undervisnings-forløb der havde til formål at rammesætte et peer2peer feedback forløb hvorder arbejdes med mundtlig peer2peer feedback, evaluere det kvalitativt medde studerende der har deltaget og efterfølgende tilpasse undervisningsfor-løbet.

Følgende spørgsmål søges besvaret:

• Hvad er de studerendes erfaringer med peer2peer feedback?• Hvordan kan et peer2peer feedback forløb rammesættes og tilrettelæg-

ges så det giver mere mening for de studerende, og hvordan opfatter destuderende deres læring gennem peer2peer forløbet?

De studerendes erfaringer med p2p feedback er meget svingende – af-hængig af deres uddannelsesmæssige baggrund. De studerende der har skif-tet uddannelse, eller som eksempelvis læst på RUC har stor erfaring medp2p feedback, mens de studerende der kommer direkte fra gymnasiet ikkehar meget eller slet ingen erfaring med hverken at give eller modtage feed-back. Dog har de alle stiftet bekendtskab med at både at give og modtagefeedback på mindre og større formidlingsopgaver på Natur- og Kulturfor-midleruddannelsen. Hvorvidt disse forløb var tydeligt rammesat var der dogikke enighed om blandt de studerende i fokusgruppeinterviewet. Derudovervar der i de tidligere feedbackforløb ikke eksplicit fokus på at give begrun-det feedback.

Et p2p forløb kan rammesættes ved først at introducere rammerne forfeedbacken for de studerende. Det kan eksempelvis være at der skal gi-ves begrundelser for den feedback man giver, samtidig med der udleveresspørgsmål eller lign som de studerende i deres feedback til hinanden skalforholde sig til. Spørgsmålene kan dermed guide de studerende til hvad dekan/skal fokusere på i deres feedback. Da de studerende i denne interven-tion ikke tidligere havde reflekteret over at der ved at give feedback foregåren spejling af eget arbejde op mod andre studerendes arbejde kan det medfordel italesættes som et læringsudbytte. For yderligere at tydeliggøre eget

214 Sandra Gentin

læringsudbytte ved at modtage feedback, kan man lade de studerende re-flektere over hvad de tager med sig af den givne feedback, og hvordan (oghvad) de vil arbejde videre med fremadrettet.

Denne rammesætning (ved at fokusere på begrundelser på den givnefeedback, og give de studerende spørgsmål de skal komme ind på i feed-backen) bidrager til at de studerende gør sig mere umage med at leverefeedbacken. Ved at italesætte læringsudbyttet ved at give feedback hjælperman de studerende til at opdage deres eget udbytte af feedbacken, idet detbliver tydeligt for de studerende at de spejler sig i andres arbejde, og dennespejling kan bidrage til at de opdager både styrker og svagheder ved dereseget arbejde.

Perspektiver – hvordan kan man implementere en godfeedbackkultur på en uddannelse?

Afslutningsvis vil jeg kort komme ind på hvordan man fortsat kan arbej-de med feedback kulturen på holdet – og dermed skabe rammerne for atde studerende anerkender hinanden fagligt – og får øjnene op for, at p2pfeedback er både lærerigt og givtigt.

For at skabe en forankring af en god feedback kultur på holdet er detvigtigt at de andre undervisere ved hvad der er foregået på dette 1. semesterhold af Natur- og Kulturformidlere, og der dermed fortsat fokuseres på denfeedback-kultur denne intervention har skabt grobund for: ”Jeg vedhæfteret skema, som I skal skrive jeres område ind i. Jeg har også skrevet, at Ikan skrive en person på, som I tænker, I vil gi og ha feedback af i perio-den. Jeg tænker, I er blevet så gode til at gi feedback, at dette ville væremeget nyttigt at bruge under eksamensskrivning.”2 Derudover kan vi un-dervisere på et undervisermøde, tale om feedbackkulturen på uddannelsengenerelt, og hvordan vi som undervisere fremmer den gode feedback kultur– fx ved en tydelig rammesætning af p2p feedback gennem fokusspørgs-mål og begrundelser for hvorfor lige netop denne feedback gives, men ogsåved at fokusere mere på spørgsmål som ”Hvad tager du med dig videre fradenne feedback session?”. Dette kan understøtte at de studerende opleverbåde at give og modtage feedback som meningsfuldt, hvilket kan fremmederes fremmøde til p2p sessionerne. Derudover skal det diskuteres hvornår

2 Email til de studerende om eksamensopgave i faget ”Naturforståelse”.


de studerende modtager p2p feedback, så det fremmer oplevelsen af detmeningsfulde feedback.

Ved at være tydelig i sin rammesætning af feedback kan man som un-derviser fremme de studerendes motivation for at give (og modtage) godfaglig feedback, og dermed fremme de studerendes mulighed for kritisk ogrefleksivt tænke over deres eget arbejde ved at spejle sig i den feedback degiver til deres medstuderende.

Referencer

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for lear-ning: the challenge of design. Assessment & Evaluation in Higher Educa-tion, 38(6), 698-712.

Christiansen, M. & Rosenkvist, G. (2008). Voksen undervisning – formid-ling i praksis. 2. udgave, 4. oplag. Hans Reizels Forlag.

Dochy, F. J. R. C., Segers, M., & Sluijsmans, D. (1999). The use of self-,peer and co-assessment in higher education: A review. Studies in Highereducation, 24(3), 331-350.

Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Im-proving the effectiveness of peer feedback for learning. Learning and in-struction, 20(4), 304-315.

Hattie, J., & Timperley, H. (2007). The power of feedback. Review ofeducational research, 77(1), 81-112.

Hounsell, D., McCune, V., Hounsell, J., & Litjens, J. (2008). The quality ofguidance and feedback to students. Higher Education Research & Devel-opment, 27(1), 55-67.

Illeris, K. (2000). Læring: aktuel læringsteori i spændingsfeltet mellem Fi-aget, Freud og Marx.

Københavns Professionshøjskole, & Københavns Universitet. (2018).Studieordning - Natur og kulturformidler - Metropol.https://www.phmetropol.dk/uddannelser

Mathiasen, H. (2008). Is There a Nexus between Learning and Teaching?:Communication as a Facilitator of Students’ Knowledge Construction. In

216 Sandra Gentin

Understanding Learning-centred Higher Education. Copenhagen BusinessSchool Press.

Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practi-ces in higher education: a peer review perspective. Assessment & Evalua-tion in Higher Education, 39(1), 102-122.

Poulos, A., & Mahony, M. J. (2008). Effectiveness of feedback: The stu-dents’ perspective. Assessment & Evaluation in Higher Education, 33(2),143-154.


The University of Edinburgh. (2010). Enhancing Feedback. http://www.enhancingfeedback.ed.ac.uk/staff/resources.html

Topping, K. (1998). Peer assessment between students in colleges and uni-versities. Review of educational Research, 68(3), 249-276.

Wanner, T., & Palmer, E. (2018). Formative self-and peer assessment forimproved student learning: the crucial factors of design, teacher participa-tion and feedback. Assessment & Evaluation in Higher Education, 43(7),1-16.

18

MSc students’ experience with peer feedbackduring a written task

Andy Gordon Howe

Department of Geosciences and Natural Resource ManagementUniversity of Copenhagen

Introduction and research question

Master’s courses at University of Copenhagen are conducted within a 9week period during which a vast quantity of material is addressed. Whilesome topics are explored at depth, others may be afforded less time andthereby less-detail. To support deeper learning opportunities, group-basedessay assignments conducted in parallel to coursework can allow studentsto explore in depth, analyse, reflect and report on topics of own choice.

On the MSc course Biodiversity in Urban Nature (‘BUN’; n = 40 stu-dents), participants write group essays which contribute to the final exam,where a given student presents a self-defined topic from within the scopeof a group essay, however essays themselves are not graded. Two studentchallenges associated with the essay writing process are investigated here:

1. BUN participants are from diverse educational and cultural back-grounds, yet encouraged to write essays in interdisciplinary groups. Stu-dents have different essay writing experience, and when group work isobligatory, a complex essay writing process can lead to student frustrationand anxieties. Thus, a common understanding of the essay writing processand how to define essay topics, may support students meet course require-ments;

2. prior to the essay deadline, there is limited time for formative feed-back from teachers on group essays. Students would obviously benefit fromopportunities for formative feedback prior to submitting essays.

In this University Pedagogy intervention conducted in Sept-Oct. 2017I investigated whether peer feedback could facilitate the initial essay writ-

218 Andy Gordon Howe

ing process, specifically students’ understanding of and definition of essayproblem statements, as well as guiding the group-forming and essay topicselection processes. Peer feedback (PFB) or “peer assessment”, definedby Topping (1998), is “an arrangement in which individuals consider theamount, level, value, worth, quality, or success of the products or outcomesof learning of peers of similar status.” The following research question wasinvestigated:

How do students experience peer feedback in relation to exploringpotential topics for the compulsory group essay?

Brief literature review

Reasons for incorporating formative PFB in tertiary education range fromcognitive and metacognitive aspects (Topping, 1998), fostering transfer-able skills such as reflection, critical thinking, negotiation and diplomacy- important for students’ own lifelong learning (Topping, 1998; Dochy etal., 1999; Adachi et al., 2018, and refs therein). Overarching is evidencePFB promotes active learning and formative reflection (e.g. Li et al., 2010),where students become active participants of FB, rather than passive recip-ients (Price et al., 2006), which can promote students’ capacity to monitorown learning (e.g. Carnell, 2016). Literature suggests the role of peer as-sessor often supports greater learning opportunities compared to the roleof assessee (Rienecker et al., 2013; Cho et al., 2011), however both rolescan promote active learning (Adachi et al., 2018). However, PFB is stillnot widespread in academic teaching (Taras, 2006; Mulder et al., 2014),despite that Danish students demand more feedback during their education(in Rienecker et al., 2013 ref. to Aarhus University, 2011 p 260), and thatPFB is identified as a valuable means to address these desires (Jensen 2011in Rienecker et al. 2013, p 261). Part of this may be challenges relating toimplementing PFB including time, effort and costs for teachers, students’motivation to engage with PFB, and lack of feedback literacy, e.g. students’empathy and human communication skills (Adachi et al., 2018).

Through structured formative PFB staring during the 2nd course week,this intervention prompted students to enter the essay writing processthrough exploration of their understanding of and criteria for PFB, fol-lowed by two iterations of giving and receiving oral PFB on individualwriting tasks. In addition, supporting information (i.e. scaffolding) on theintention of the course essay, formal essay structure and scope of topics

18 MSc students’ experience with peer feedback during a written task 219

were actively included. While written products of the intervention were notnecessarily used in submitted group essays, the process underlying the in-tervention was directly applicable to the subsequent essay writing process.For example, the PFB iterations related to important early stages of the es-say writing process, namely identification of topic/problem contexts anddefinition of the guiding essay problem statement (Rienecker & Jørgensen,2014). The underlying motivation with the intervention was thus to supportstudents explore topics for group essays, familiarise them with formal essayrequirements and to provide insight into the larger essay writing process.

Intervention and documentation (method)

The intervention is briefly described here, but for more detailed informationsee Appendix A. Following a group discussion where PFB was explored(Interview 1), students wrote individual problem statements and 5-10 linesof text briefly describing a problem context of own interest (Essay task1). Students then undertook PFB session 1 in randomly assigned groupsusing criteria described during Interview 1. Subsequently, Essay task 2saw students refine their statements, revise 10 -20 lines of text and outlinepotential essay topics to address. PFB session 2 was actively directed withstudents grouped according to their chosen topics. Finally, a second groupdiscussion aimed to explore student reflections on the process (Interview2).

Qualitative data was collected through written exercises, two master-class interviews (n= 25 - 30 students each time) and use of tag clouds(Appendix B,C). Written exercises entailed Group discussion 1 wherebystudents wrote their input on A1 posters to two questions (What is your un-derstanding of PFB? What do you need to consider when giving/receivingPFB to other students?). Posters were hung on a classroom wall and stu-dents asked to visit all posters. Subsequently, Interview 1 sought to iden-tify themes reflecting collective understanding of PFB, identify criteria forPFB and a code of conduct. As part of Essay task 2, students were askedto briefly state how they incorporated PFB from PFB session 1. In Groupdiscussion 2 students revisited their posters and added input (in a differ-ent colour) based on reflections of their PFB experience (e.g. Appendix D).Subsequently, Interview 2 explored student reflections on the process, aswell as their constructive input regarding ways to improve the process. Fi-nally, using tag clouds, students were asked to describe their experience


of PFB and most important learning experience from the intervention (Ap-pendix B,C).

Analysis + discussion

Students experienced the PFB intervention in relation to Essay tasks aslargely positive with “useful”, “constructive”, “helpful” and “clarification”reported most often by students (Appendix B). In Essay task 2, studentreflections generally revealed that PFB session 1 had been useful for struc-turing, refining problem statements and finding alternative sources: “I evenchanged the focus on the problem statement, because after the feedbackI realized that my essay had to be more focused in a more specific direc-tion. . . ” and “. . . the FB helped make my problem statement a bit more fo-cused, helping me choose the right direction.” Similar reflections providedevidence the PFB was being implemented in order to enhance an indivi-dual’s work. In the following, four themes (there were more) drawn fromall data forms are explored.

18.0.1 Understanding and co-creating PFB criteria

Exploration of students’ inputs and understandings led directly to a com-mon set of criteria for the PFB sessions. Groups suggested similar criteriawhich indicate students had prior experience with PFB and that severalcommon understandings existed. For example, “constructive criticism” ap-peared on all six posters (Appendix D). Other common criteria included theneed to be specific, to give examples and avoid being superficial. For exam-ple, “. . . be completely neutral. . . ” and “. . . when criticising. . . back up withexamples. . . to help the person move on. . . can you help me understandingthis part. . . ”

Cho & Cho (2011) suggest that instructors provide students with themethodology of PFB, e.g. question prompts to focus on particular aspectsof a written piece or a prescribed review model. In the present intervention,the criteria discussed during Interview 1 were not exhaustive; however, Ifelt the discussion was valuable for students in terms of co-determining thetype of PFB they should aim for and how to achieve this. In future iterationsI would develop the PFB model further. Although this is a time consumingaspect of implementing PFB, the development of criteria and how theserelate to a given course assessment, may help students decode intendedlearning goals (Adachi et al., 2018).


Benefits of being assessor/assessee

Students made reference to the value of PFB for both assessor and assessee:“PFB is a learning process both for the receiver and the giver.” Studentsidentified open-mindedness when receiving feedback as an important partof learning from others’ ways of working and thinking. This links to senti-ments relating to the value of receiving “multiple views” and that own re-flections on peers’ feedback are part of a student’s own learning process,as was evident from Interview 2: “Most PFB lived up to the standards(criteria). . . .we realised, in addition to multiples views, we also had newinformation/references from our feedbackers. . . gave us ideas for more ref-erences we could use. . . links to people having different backgrounds andtaken different courses, and have realised an article which you haven’t hadyourself. . . that was nice.” The intervention clearly created multiple learn-ing opportunities beyond those associated with instructor-based FB whichcorroborates with (Nortcliffe, 2012) who identified increased opportunitiesin PFB for learning from “a larger number of voices”.

Evidence of transferable skills through PFB

When students were asked the most important learning experiences formthe PFB exercise “reflection” was resoundingly reported, followed by “crit-ical thinking”, “tolerance” and “listening” (Appendix C). Other commentssuch as “preparation”, “re-analyse”, “constructive feedback”, “critical read-ing”, “exchanging ideas” and “being critical” point towards student expe-rience with the very skills associated with feedback itself (from literatureabove). This suggests that although a specific essay-based task was the fo-cus of PFB tasks an array of transferable skills are activated through thePFB process, and importantly, these are recognised by students.

Vulnerability/anxiety during PFB It surprised me that personal vulner-ability was expressed by several groups regarding the social/psychologicalside of PFB – which clearly denotes potential pitfalls of PFB. On posters,“brutal”, “honest”, and reference to “defence of one’s self” were statedand were subsequently discussed in plenum. These sentiments were ac-tively added to the PFB criteria stating a need for “empathy” and “respect”and “nurturing”.

The need to address vulnerability during discussion of criteria wasbrought up during Interview 1 - “If you are allowed to defend yourself, youforget to listen. . . try to listen first. . . and then you can say. . . ” Similarly,


yet more vulnerable sentiment, “It (PFB) only works if you don’t feel at-tacked”. Thus, important criteria for students were to listen actively, reflectin real-time, assess the feedback and discuss it collectively – but to avoidbeing defensive. In contrast, during Interview 2 one student offered “. . . interms of receiving feedback, if you find yourself defending it. . . it could bean indication that something you’ve written hasn’t been communicated ina clear way. . . ”. This student obviously reflected on previous (own) expe-riences which points to student’s awareness of the role that reflection andamendment of behaviour may have on influencing one’s learning process.

Both assessor and assessee may experience anxiety during PFB (Top-ping, 1998), exemplified by social embarrassment when a student’s weak-nesses are identified by peers. Mulder et al. (2014) uncovered anxieties in50% of students’ pre-PFB expectations relating to attaining the best toneto balance positive and negative feedback and the quality of their feed-back when assessing peers in connection with a writing task. However,this did not concur with post-PFB experiences, and Topping (1998) sug-gests that overall PFB reduces student anxieties. BUN students were askedfor three words to describe the PFB experience; “energy-draining”, “shy-ness” and “chaotic” reveal that some students experienced forms of anxiety(Appendix B). Masatoshi (2013) highlights the importance of positive so-cial relationships between students as an integral aspect of successful peerfeedback - future PFB interventions could explore this aspect of PFB.

Conclusion

Student reflections on the value of PFB for this early-stage exercise in con-nection with the compulsory group essay were largely positive, e.g. “. . . Ifeel like we agreed that a lot of experience was very positive FB and any-thing technically critical, was questions/clarification of the topic, but byhaving to clarify the points, it helped yourself structure the topic. . . ”. Assuccessful PFB relies on motivated students who see meaning in why theyare asked to undertake PFB (Adachi et al., 2018), scaffolding and a struc-tured PFB framework, employed in this intervention, undoubtedly con-tributed to this. Additionally, the plenum discussions and co-creation ofPFB criteria were, I feel, equally important for fostering ownership of thetask, group motivation and establishing a common willingness among stu-dents to participate in the intervention. However, there were a few somestudents who felt anxiety during the process.


Perspective and limitations

While PFB offers students opportunities for formative feedback which ulti-mately enhances students’ submitted works, this intervention did not eval-uate the impact of PFB on the quality of the students’ final essays (e.g.Mulder et al., 2014). Interestingly, no voluntary attempts were made toemploy PFB by groups subsequent to the intervention. Time constraintsexperienced by students were evident during the intervention and couldin part have contributed to quelling initiatives to embrace PFB voluntar-ily, e.g. “The time factor. . . in our group we didn’t make it to the criti-cal/improvement part.” Adachi et al. (2018) identified instructor frustra-tions associated with “non-completion of feedback loops” whereby stu-dents, for undisclosed reasons, do not enhance future work by incorporatingFB, despite teachers’ best intentions. Future attempts at incorporating PFBare likely to benefit from formal scaffolding and adequate time to achievethe benefits of PFB- perhaps PFB tasks should even be obligatory.

Mapping the causes of student anxiety associated with PFB requiresgreater attention. Student anxieties are probably influenced by a combi-nation of previous cultural experiences with PFB, a student’s personalitytraits, and the learning environment of a given course. In the present inter-vention, the learning environment is the most malleable aspect a teachercan influence. Future improvements to the learning environment could beexplored; for example, during development of PFB criteria, it might bevaluable to ask students to anonymously suggest approaches they feel mayreduce anxieties associated with PFB. Furthermore, increasing the time al-located to PFB could support a relaxed atmosphere, whereby students canfocus on providing good PFB and are not (also) pressured by time con-straints. Finally, in the BUN context, extending PFB as an obligatory partof the group essay, thereby focussing FB at the group-level, rather than stu-dent level (as in the intervention), might reduce anxiety further- as well asclosing feedback loops.

The PFB model employed here was based on student input and internetliterature (e.g. Oxford Brooks University, Herriot Watt University, Univer-sity of Edinburgh). It would be interesting to use a PFB model which aimedto enhance the quality of PFB by, prior to giving FB, formally distinguish-ing the potential scope of students’ feedback comments between surface,mico- and macro-meaning levels. This approach could promote assessors’awareness to differentiate between types of FB (Li et al., 2010), while as-sist the assessee to understand the orientation of a peer’s FB (e.g. Cho et al.,


2011). In this connection, exploring the value of incorporating co-creationof criteria with students in order to foster ownership, engagement and moti-vation for participating in the PFB process would be worth pursuing, sincethere are concerns reduced engagement in PFB contributes to superficiallearning (Adachi et al., 2018).

References

Adachi, C., Tai, J. H. M., & Dawson, P. (2018). Academics’ perceptions ofthe benefits and challenges of self and peer assessment in higher education.Assessment & Evaluation in Higher Education, 43(2), 294-306.

Carnell, B. (2016). Aiming for autonomy: formative peer assessment in afinal-year undergraduate course. Assessment & Evaluation in Higher Edu-cation, 41(8), 1269-1283.

Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments.Instructional Science, 39(5), 629-643.

Dochy, F. J. R. C., Segers, M., & Sluijsmans, D. (1999). The use of self-,peer and co-assessment in higher education: A review. Studies in Highereducation, 24(3), 331-350.

Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: Howstudent learning improves by giving and receiving peer feedback. Britishjournal of educational technology, 41(3), 525-536.

Sato, M. (2013). Beliefs about peer interaction and peer corrective feed-back: Efficacy of classroom intervention. The Modern Language Journal,97(3), 611-633.

Mulder, R., Baik, C., Naylor, R., & Pearce, J. (2014). How does studentpeer review influence perceptions, engagement and academic outcomes? A


case study. Assessment & Evaluation in Higher Education, 39(6), 657-677.https://www.tandfonline.com/doi/abs/10.1080/02602938.2013.860421

Nortcliffe, A. (2012). Can students assess themselves and their peers?: afive year study. Student Engagement and Experience Journal, 1(2).

Price, M., & O’Donovan, B. (2006). Improving performance through en-hancing student understanding of criteria and feedback. Innovative assess-ment in higher education, 100-109.

Rienecker L, Bruun J. (2013) Feedback in Universitetspædagogik edited byRienecker L, Jørgensen PS, Dolin J, Ingerslev GH. Samfundslitteratur

Taras, M. (2006). Do unto others or not: equity in feedback for undergrad-uates. Assessment & Evaluation in Higher Education, 31(3), 365-377.

Topping, K. (1998). Peer assessment between students in colleges and uni-versities. Review of educational Research, 68(3), 249-276. Internet feed-back homepages: http://www.enhancingfeedback.ed.ac.uk/staff/resources.html


A

8

8. Appendices

Appendix A. Overview of steps involved in peer feedback (PFB) intervention aimed at enhancing students’ experience of the initial essay writing process during Biodiversity in Urban Nature, 2017, within the course’s timeframe (other feedback opportunities highlighted).

Course week

Intervention step Qualitative data collected

2* In class: Group discussion 1 on understanding of PFB → A1 posters → Interview 1: Exploration of PFB and PFB criteria → Introduction to Essay task 1

- 6 posters- 1st masterclassinterview

4 Outside class: Students upload individual Essay task 1

Students assigned random PFB groups (3 - 4 students) → PFB session 1 (in class)

5 Outside class: Students upload individual Essay task 2

In class: Students assigned PFB groups based on topics (3 - 6 students) → PFB session 2 (in class)

Group discussion 2 on reflections on PFB experience → A1 posters revisited → Interview 2: Experience of PFB, ways of improving the intervention → tagcloudScheduled opportunity for Essay feedback by teacher

- Statement onhow studentsincorporatedPFB in EssayTask 2- 6 posters(revisited)- 2nd masterclassinterview- Mentimeter tagclouds

6-78

Opportunities for informal teacher feedback Scheduled opportunity for teacher feedback

9 Essay deadline 11 Post-exam summative feedback * Essay process started 4 weeks earlier than previous courses


B

9

Appendix B – Mentimeter 1: Provide 3 words which you feel describe your experience with PFB during essay tasks 1 and 2

Appendix 3 – Mentimeter 2: What are the most important learning experiences you take with you fromthis PFB exercise?


C

9

Appendix 2 – Mentimeter 1: Provide 3 words which you feel describe your experience with PFB during essay tasks 1 and 2

Appendix C – Mentimeter 2: What are the most important learning experiences you take with you from this PFB exercise?


D

10

Appendix D - Photo of a student poster

Part VI

Conceptual understanding and metacognition

19

Improving Mathematics Understanding in thePharmacy Undergraduate Program

Kathryn Browning

Department of PharmacyUniversity of Copenhagen

Introduction

Numeracy is an important component of the pharmacy career path. Errorsin routine calculations such as drug dosages and dilutions can lead to dis-astrous consequences. Whilst studying the concentration of morphine in-fusions for a neonatal critical care unit prepared by in-house pharmacists,Parshuram et. al. found errors of greater than 10% in in 65% of samples(Parshuram et al., 2003). 6% of these samples contained two-fold errors orgreater (Parshuram et al., 2003).

It is critical that the Pharmacy education effectively prepares studentsto be confident and precise in calculation and lab skills. It has been notedby course leaders that undergraduate students throughout the program arelacking the mathematics skills necessary for successful completion of thepharmacy education. The mismatch between teacher’s expectations and thestudent’s maths abilities is a complaint that is neither new or confined tohealth sciences (Cox, 2001). Upon entering university education, the levelof experience varies dramatically with some students unfamiliar with sim-ple rearrangement of equations and percentage calculations whilst othersare already comfortable with the majority of the first year curriculum. Thequality and level of teaching of mathematics prior to university level hasbeen found to be well-correlated to the success of the Pharmacy student(Conn et al., 2018). Within the Department of Pharmacy at KU previousresearch found that mathematics grades upon admission was the key com-petency that predicted the success of the student at the end of the first year(Bendahl et al., 2004). As the Danish mathematics high school curriculum

234 Kathryn Browning

is something that we, as university teachers, cannot easily affect, require-ments on teaching are greatly complicated especially in first year courses.Devising the most appropriate level to begin teaching is vital to strike a bal-ance between catering to the students that are struggling with the materialwhilst ensuring the advanced students are challenged. This, coupled withteachers who are not specialised maths lecturers, leads to a discrepancy inthe teacher’s expectations and the student’s actual mathematical abilities(Batchelor, 2004).

A decline in student’s mathematical abilities entering university levelhas been well documented (Hawkes et al., 2000; Lawson, 2003; Lawsonet al., 2003) and in some cases has led to a reduction in the mathematicalcontent of university courses (Norris, 2012). Furthermore, students risk dis-engagement from the scientific topics due to a lack of confidence in theirmaths abilities, especially when comparing themselves to students withhigher mathematical competence (Lawson et al., 2003). Anecdotally, thisleads to an overreliance on computer algebra systems (such as Maple) inthese less confident students and a reduction in critical assessment whenanalysing computed answers. Other universities have tried to fix this in anumber of ways from small scale project such as the preparation of studyguide leaflets (Study guides, 2018) to university wide centres of mathematicsupport (Lawson et al., 2003; Croft et al., 2014). In this project the aim isto assess the student’s mathematical skills upon entering the pharmacy pro-gram and point them in the direction of learning which can strengthen theirweaker areas. This was achieved by preparation of two online self-assessedquizzes, one, which contained familiar algebra style questions, and a sec-ond that focussed on routine calculations from the degree.

Description of the course

The pharmacy education consists a three-year bachelor course followed bya two-year masters course, completion of both elements qualifies the stu-dent as a Pharmacist. At Copenhagen University the admission require-ments for the bachelors program are: Danish A, English B, Mathematics Aand one of the following combinations: Physics B and Chemistry B, PhysicsB and Biotechnology A, Chemistry B and Geoscience A or Chemistry B,Biology A and Physics C. In addition in 2018, an entrance exam score of8.2 for quota 1 and 7.3 for quota 2.The bachelors program is taught be-tween two departments, Institut for Farmaci (IF) and Institut for Lægemid-

19 Improving Mathematics Understanding in Pharmacy Program 235

deldesign og Farmakologi (ILF), with IF taking many of the courses re-lated to physics and traditional pharmacy and ILF holding courses on bio-logy and organic chemistry. Previously there was a dedicated mathematiccourse, however, this was recently removed, and the content integrated intoall courses within the first year.

Intervention

In order to provide a self-assessment exercise for the first year BSc students,two quizzes were prepared as part of the Kemiske Principper (SFABIF117U)course. These quizzes were designed to test the student’s maths abilities intwo main areas, traditional maths and pharmaceutical maths. The first quizcontained more traditional, algebra based questions including addition offractions, rearranging algebraic equations, differentiation and integration.Due to limitations of the Absalon quiz structure, it was hard to input for-mulae as answers, therefore, multiple choice questions (MCQ) were used.The second quiz aimed to test the students application of the tradition mathsto pharmaceutical, real-life, situations, for example dilution of a stock so-lution, calculation of concentration, mole fractions etc. Only the first ques-tion, a question on rounding and significant figures (Appendix A, 1b) wasmultiple choice, all other questions were constructed response questions(CRQs) consisting of numerical answers. After completing each quiz, theanswers were provided with a detailed description (in English) of how toobtain the correct answers, a link to the relevant pages of the maths text-book for the course was also provided (Monk et al., 2010). The test wasnot mandatory, however, a deadline was set one week after the introductorylecture to encourage the students to complete the tests before becoming toodistracted by course material.


Results and discussion

Description of the course

The pharmacy education consists a three-year bachelor course followed by a two-year masters course, completion of both elements qualifies the student as a Pharmacist. At Copenhagen University the admission requirements for the bachelors program are: Danish A, English B, Mathematics A and one of the following combinations: Physics B and Chemistry B, Physics B and Biotechnology A, Chemistry B and Geoscience A or Chemistry B, Biology A and Physics C. In addition in 2018, an entrance exam score of 8.2 for quota 1 and 7.3 for quota 2.The bachelors program is taught between two departments, Institut for Farmaci (IF) and Institut for Lægemiddeldesign og Farmakologi (ILF), with IF taking many of the courses related to physics and traditional pharmacy and ILF holding courses on biology and organic chemistry. Previously there was a dedicated mathematic course, however, this was recently removed, and the content integrated into all courses within the first year.

Intervention

In order to provide a self-assessment exercise for the first year BSc students, two quizzes were prepared as part of the Kemiske Principper (SFABIF117U) course. These quizzes were designed to test the student’s maths abilities in two main areas, traditional maths and pharmaceutical maths. The first quiz contained more traditional, algebra based questions including addition of fractions, rearranging algebraic equations, differentiation and integration. Due to limitations of the Absalon quiz structure, it was hard to input formulae as answers, therefore, multiple choice questions (MCQ) were used. The second quiz aimed to test the students application of the tradition maths to pharmaceutical, real-life, situations, for example dilution of a stock solution, calculation of concentration, mole fractions etc. Only the first question, a question on rounding and significant figures (Appendix 1b) was multiple choice, all other questions were constructed response questions (CRQs) consisting of numerical answers. After completing each quiz, the answers were provided with a detailed description (in English) of how to obtain the correct answers, a link to the relevant pages of the maths textbook for the course was also provided.12 The test was not mandatory, however, a deadline was set one week after the introductory lecture to encourage the students to complete the tests before becoming too distracted by course material.

Results and discussion

Figure 1 Test score data for the two online quizzes, left, data for the algebra test (n=213) and, right, data for the

pharmaceutical calculations test (n=190).

0 1 2 3 4 5 6 7 8 9 10 11 120

5

10

15

20

25

30

35

40

Num

ber o

f stu

dent

s

Algebra test score0 1 2 3 4 5 6 7 8 9 10

0

5

10

15

20

25

30

35

40

Num

ber o

f stu

dent

s

Pharmaceutical calculation test score

Fig. 19.1. Test score data for the two online quizzes, left, data for the algebra test(n=213) and, right, data for the pharmaceutical calculations test (n=190).

Results from the online quizzes were collated in Absalon and the result-ing test scores presented in figure 19.1. The mean test score for the algebratest was 66% compared to 32% for the pharmaceutical calculations. Thedistribution of test scores for the algebra test shows a traditional Gaussiandistribution indicating that the level of the test was well pitched to be chal-lenging yet still achievable for the mathematically confident students. Theresults for the pharmaceutical calculations (figure 19.1, right) were surpris-ing, many students (33) failed to score above zero and the number of stu-dents obtaining higher grades was much lower, as seen by a drop in meantest score of 34%. Figure 19.2 shows the scores for both tests for each stu-dent after removal of students who only completed one of the two tests(n=187). The size of the bubble indicates the amount of students with thesame score pairing. A student scoring highly in the algebra exam will notnecessarily succeed at the pharmaceutical calculation test, in fact, it is clearthat almost all students scored worse on the pharmacy test compared to thealgebra test.


Results from the online quizzes were collated in Absalon and the resulting test scores presented in figure 1. The mean test score for the algebra test was 66% compared to 32% for the pharmaceutical calculations. The distribution of test scores for the algebra test shows a traditional Gaussian distribution indicating that the level of the test was well pitched to be challenging yet still achievable for the mathematically confident students. The results for the pharmaceutical calculations (figure 1, right) were surprising, many students (33) failed to score above zero and the number of students obtaining higher grades was much lower, as seen by a drop in mean test score of 34%. Figure 2 shows the scores for both tests for each student after removal of students who only completed one of the two tests (n=187). The size of the bubble indicates the amount of students with the same score pairing. A student scoring highly in the algebra exam will not necessarily succeed at the pharmaceutical calculation test, in fact, it is clear that almost all students scored worse on the pharmacy test compared to the algebra test.

0 2 4 6 8 10 12

0

2

4

6

8

10

Phar

mce

utic

al c

alcu

latio

ns te

st s

core

(out

of 1

0)

Algebra test score (out of 12)

1 4 7 10

Figure 2 Comparison of test scores of the two quizzes, larger bubbles indicate more students with the same score in both tests

The hypothesis upon devising the study was that mathematics was too abstract for many of the pharmacy students, as the course provides a high content of applied science, and therefore questions that were more closely related to real life maths applications would be easier to answer. However, having recently left high school education, the more traditional algebra style questions seem to be easier. There are a number of other reasons why the test scores may differ between the tests, the most striking is the difference in answer format. The nature of the algebra test answers (the need to input formula) necessitated the use of MCQs whereas the pharmaceutical test contained CRQs. MCQ tests suffer from a ‘guessing effect’, where scores are artificially inflated due to the student assessment of the most likely answer. However, Triantis et. al. found that two similar tests could be compared by using a scoring rule which corrected a considerably smaller difference between MCQ and CRQ tests (k parameter of 0.3).13 Therefore, there must be other factors affecting the difference in scores between the two tests.

Another possible difference between the two tests is language, although the introduction of the test was translated to Danish, the questions (and answer explanations) were written in English. Due to the applied nature of the questions in the pharmaceutical test, the average number of words per question was 24, excluding values, formulas and units, compared to only 5 words for the algebra

Fig. 19.2. Comparison of test scores of the two quizzes, larger bubbles indicatemore students with the same score in both tests.

The hypothesis upon devising the study was that mathematics was tooabstract for many of the pharmacy students, as the course provides a highcontent of applied science, and therefore questions that were more closelyrelated to real life maths applications would be easier to answer. However,having recently left high school education, the more traditional algebrastyle questions seem to be easier. There are a number of other reasons whythe test scores may differ between the tests, the most striking is the dif-ference in answer format. The nature of the algebra test answers (the needto input formula) necessitated the use of MCQs whereas the pharmaceuti-cal test contained CRQs. MCQ tests suffer from a ‘guessing effect’, wherescores are artificially inflated due to the student assessment of the mostlikely answer. However, Triantis et. al. found that two similar tests could becompared by using a scoring rule which corrected a considerably smallerdifference between MCQ and CRQ tests (k parameter of 0.3) (Triantis etal., 2014). Therefore, there must be other factors affecting the difference inscores between the two tests.


Another possible difference between the two tests is language, althoughthe introduction of the test was translated to Danish, the questions (andanswer explanations) were written in English. Due to the applied natureof the questions in the pharmaceutical test, the average number of wordsper question was 24, excluding values, formulas and units, compared toonly 5 words for the algebra test. As English is not the main language ofinstruction in the course, this may have added to the lower average scorefor the pharmaceutical test. In future iterations of this quiz, all questionswill be translated to Danish; however, the feedback will be kept in Englishas it refers to an English textbook.

Finally, each student was able to attempt the test 3 times, with feedbackgiven after the first attempt on how to attempt the question. As the phar-maceutical test was the second test, the students may have become wise tothis feedback and are not applying themselves in the first attempt, ratherjust waiting to read the feedback and attempting it on the second try. Infact, 128 students attempted the pharmaceutical quiz a second time and themean test score increased to 84% (from 34%). In comparison, 96 studentsretook the algebra test and the mean test score increased from 66% to 91%.

test. As English is not the main language of instruction in the course, this may have added to the lower average score for the pharmaceutical test. In future iterations of this quiz, all questions will be translated to Danish; however, the feedback will be kept in English as it refers to an English textbook.

Finally, each student was able to attempt the test 3 times, with feedback given after the first attempt on how to attempt the question. As the pharmaceutical test was the second test, the students may have become wise to this feedback and are not applying themselves in the first attempt, rather just waiting to read the feedback and attempting it on the second try. In fact, 128 students attempted the pharmaceutical quiz a second time and the mean test score increased to 84% (from 34%). In comparison, 96 students retook the algebra test and the mean test score increased from 66% to 91%.

Figure 3 Percentage of correct answers for each questions in, left, the algebra test (n=213) and, right, the pharmaceutical calculations test (n=190).

The results, figure 3, were further analysed to assess particular concepts that the students struggle with, shown as the percentage of correct answers for each question in Appendix 1. In the algebra test, there are a number of questions that scored below 60% correct, indicating that students have problems with rearranging equations especially when they contain natural logs and integration. Two questions scored over 90 percent indicating good knowledge of the addition of powers and differentiation. The results for the pharmaceutical test show that all concepts need to be improved, however, there are 5 questions which scored particularly badly with less than 30% correct. Suggesting the students struggle with units, significant figures, using equations and calculating the gradient of a graph. Interestingly 62% of students were able to answer question 2 correctly, calculating the amount of a drug in an aliquot of stock solution. The second highest score (48%) was for question 7, calculation of a percentage yield.

The multiple choice answers to the algebra test were chosen from simple mistakes e.g. order of rearrangements, miscalculation of positive and negative numbers etc. For the pharmaceutical calculation test, it was not clear which mistakes would be most common. As a CRQ style was used for this test, the most common wrong answers were recorded and can now be used to develop a MCQ test with realistic answers in the coming years.

As discussed by Batchelor, the best improvement in mathematical competence is obtained when self-assessment tests are coupled with a focussed lecture addressing the major issues of the

1 2 3 4 5 6 7 8 9 10 11 120

20

40

60

80

100

Cor

rect

ans

wer

(%)

Algebra test question number1 2 3 4 5 6 7 8 9 10

0

20

40

60

80

100

Cor

rect

ans

wer

(%)

Pharmacy calculation question number

Fig. 19.3. Percentage of correct answers for each questions in, left, the algebra test(n=213) and, right, the pharmaceutical calculations test (n=190).

The results, figure 19.3, were further analysed to assess particular con-cepts that the students struggle with, shown as the percentage of correctanswers for each question in Appendix A. In the algebra test, there are a


number of questions that scored below 60% correct, indicating that stu-dents have problems with rearranging equations especially when they con-tain natural logs and integration. Two questions scored over 90 percentindicating good knowledge of the addition of powers and differentiation.The results for the pharmaceutical test show that all concepts need to beimproved, however, there are 5 questions which scored particularly badlywith less than 30% correct. Suggesting the students struggle with units,significant figures, using equations and calculating the gradient of a graph.Interestingly 62% of students were able to answer question 2 correctly, cal-culating the amount of a drug in an aliquot of stock solution. The secondhighest score (48%) was for question 7, calculation of a percentage yield.

The multiple choice answers to the algebra test were chosen from sim-ple mistakes e.g. order of rearrangements, miscalculation of positive andnegative numbers etc. For the pharmaceutical calculation test, it was notclear which mistakes would be most common. As a CRQ style was usedfor this test, the most common wrong answers were recorded and can nowbe used to develop a MCQ test with realistic answers in the coming years.

As discussed by Batchelor, the best improvement in mathematical com-petence is obtained when self-assessment tests are coupled with a focussedlecture addressing the major issues of the cohort directly (Batchelor, 2004).From the results gathered from these tests, an additional lecture starting in2019 will be added to the curriculum in the second week of the semesterdirectly after the deadline, which will be tailored to focus on the issues ofthe incoming cohort.

Conclusion

In conclusion, two tests were developed to assess the mathematics abili-ties of incoming pharmacy students. Results showed that the algebra testwas a useful refresher for students with a normal distribution in test resultsand an average score of 66%. The pharmacy test was less successful (aver-age score 32%) and highlighted a number of issues in the understanding ofmaths when applied to ‘real-life’ situations. Key problems were discoveredin the use of units, significant figures and rounding errors. More emphasisin the first year of the degree should be placed on improving student’s con-fidence in their mathematics abilities and how to use mathematics as a toolfor pharmacy. The incorrect answers gathered for the pharmaceutical testwill be used to develop a multiple choice test so that the two tests can be


better compared in future years. Due to the findings of this research a 45minute lecture will also be added to the chemical principles course in 2019.Specific problems identified from the tests will be the focus whilst attempt-ing to ground the content in everyday occurrences in the pharmaceuticalcareer track, i.e. dilutions of drugs to correct dosages or preparation of ac-curate documentation for FDA approval.


References

Batchelor, H. (2004). The importance of a mathematics diagnostic test forincoming pharmacy undergraduates. Pharmacy Education, 4(2).

Bendahl, L., Jensen, H., Larsen, S. W., Thomsen, A. E. & Østergaard, J.(2004). Undersøgelse af farmaceutstuderendes grundlæggende kemiske ogmatematiske færdigheder efter 1 . år.

Conn, K. M., Birnie, C., McCaffrey, D., & Brown, J. (2018). The rela-tionship between prior experiences in mathematics and pharmacy schoolsuccess. American journal of pharmaceutical education, 82(4), 6257.

Cox, W. (2001). On the expectations of the mathematical knowledge offirst-year undergraduates. International Journal of Mathematical Educa-tion in Science and Technology, 32(6), 847-861.

Croft, A. C., Lawson, D. A., Hawkes, T. O., Grove, M. J., Bowers, D., &Petrie, M. (2014). sigma–a network working. Mathematics Today, 50(1).

Hawkes, T., & Savage, M. D. (2000). Measuring the mathematics problem.London: Engineering Council.

Lawson, D. (2003). Changes in student entry competencies 1991–2001.Teaching mathematics and its applications, 22(4), 171-175.

Lawson, D., Croft, A. C., & Halpin, M. (2003). Good practice in the provi-sion of mathematics support centres. LTSN Maths, Stats & OR Network.

Norris, E. (2012). Solving the maths problem: international perspectiveson mathematics education. Royal Society for the encouragement of Arts,Manufactures and Commerce.

Monk, P., & Munro, L. J. (2010). Maths for chemistry: a chemist’s toolkitof calculations. Oxford University Press.

Parshuram, C. S., Ng, G. Y, Ho, T. K., Klein, J., Moore, A. M., Bohn,D., & Koren, G. (2003). Discrepancies between ordered and deliveredconcentrations of opiate infusions in critical care. Critical care medicine,31(10), 2483-2487.

Study guides, Learning, University of Kent. Available at: https://www.kent.ac.uk/learning/resources/study-guides.html?tab=pharmacy-calculations. (Ac-cessed: 31st December 2018)

Triantis, D., Ventouras, E., Leraki, I., Stergiopoulos, C., Stavrakas, I., &Hloupis, G. (2014, April). Comparing Multiple-Choice and Constructed


Response Questions Applied to Engineering Courses. In International Con-ference on Computer Supported Education (pp. 130-140). Springer, Cham.

Appendix 1a Test Questions – Algebra

1. Simplify the following to solve for x.

𝒚𝒚 = 𝟓𝟓𝟓𝟓 − 𝟒𝟒𝟓𝟓 + 𝟕𝟕𝟓𝟓 + 𝟏𝟏

a. 𝑥𝑥 = 𝑦𝑦+18

b. 𝑥𝑥 = 𝑦𝑦−18

c. 𝑥𝑥 = 𝑦𝑦8

d. 𝑥𝑥 = 𝑦𝑦 − 8

2. What is the value of: 𝟏𝟏𝒂𝒂

+ 𝟏𝟏𝒃𝒃 ?

a. 𝑎𝑎𝑏𝑏𝑏𝑏𝑎𝑎

b. 𝑏𝑏+𝑎𝑎𝑎𝑎𝑏𝑏

c. 1𝑎𝑎𝑏𝑏

d. 1𝑎𝑎2𝑏𝑏2

3. Multiply out the brackets and solvefor x.

𝒚𝒚 = 𝟒𝟒(𝒄𝒄 − 𝟓𝟓𝟐𝟐)

a. 𝑥𝑥 = 4𝑐𝑐 − �𝑦𝑦

b. 𝑥𝑥 = �𝑦𝑦−4𝑐𝑐4

c. 𝑥𝑥 = �𝑐𝑐−4𝑦𝑦

d. 𝑥𝑥 = �𝑐𝑐 − 𝑦𝑦4

4. Multiply out the brackets in thefollowing equation.

𝒂𝒂𝒃𝒃(𝟓𝟓 − 𝒚𝒚)

a. 𝑎𝑎𝑥𝑥 − 𝑏𝑏𝑦𝑦

b. 𝑎𝑎𝑥𝑥 + 𝑏𝑏𝑥𝑥 − 𝑎𝑎𝑦𝑦 − 𝑏𝑏𝑦𝑦

c. 𝑎𝑎𝑏𝑏𝑦𝑦 + 𝑎𝑎𝑏𝑏𝑥𝑥

d. 𝑎𝑎𝑏𝑏𝑦𝑦 − 𝑎𝑎𝑏𝑏𝑥𝑥

5. Factorise the following equation.

𝟓𝟓𝟐𝟐 + 𝟒𝟒𝟓𝟓 + 𝟑𝟑

a. (𝑥𝑥 − 3)(𝑥𝑥 − 1)

b. (𝑥𝑥 + 3)(𝑥𝑥 + 1)

c. (𝑥𝑥 − 4)(𝑥𝑥 − 3)

d. (𝑥𝑥 + 4)(𝑥𝑥 + 3)

6. Reduce the equation to the formy=mx+c.

𝟒𝟒𝒚𝒚 = −𝟒𝟒𝟓𝟓 + 𝟏𝟏𝟐𝟐

a. 𝑦𝑦 = −𝑥𝑥 + 12

b. 𝑦𝑦 = −𝑥𝑥 + 3

c. 𝑦𝑦 = 𝑥𝑥 − 3

d. 𝑦𝑦 = 𝑥𝑥 + 4

7. Calculate the slope of a linebetween the following co-ordinates.

(−𝟓𝟓,𝟒𝟒) 𝒂𝒂𝒂𝒂𝒂𝒂 (𝟕𝟕,𝟏𝟏𝟐𝟐)

a. 4


A

b. 14

c. 1,5

d. 23

8. Simplify the following equation.

𝟏𝟏𝟏𝟏𝟑𝟑𝟓𝟓𝟏𝟏𝟏𝟏𝟓𝟓

a. 108

b. 19

c. 1008

d. 1015

9. Isolate x.

𝒚𝒚 = 𝐥𝐥𝐥𝐥 𝟓𝟓𝟐𝟐

a. 𝑥𝑥 = �ln𝑦𝑦

b. 𝑥𝑥 = √𝑒𝑒𝑦𝑦

c. 𝑥𝑥 = 𝑒𝑒√𝑦𝑦

d. 𝑥𝑥 = �ln �1𝑦𝑦�

10. Simplify the following equation.

𝐥𝐥𝐥𝐥 𝟓𝟓𝟓𝟓 − 𝐥𝐥𝐥𝐥𝟓𝟓

a. ln 1𝑥𝑥

b. 𝑒𝑒5𝑥𝑥−5

c. ln 25𝑥𝑥

d. ln 𝑥𝑥

11. Differentiate the followingequation.

𝒚𝒚 = 𝟓𝟓𝟑𝟑 + 𝟒𝟒

a. 2𝑥𝑥2 + 3

b. 𝑥𝑥2 + 43

c. 3𝑥𝑥2

d. 3𝑥𝑥4 + 4

12. Integrate the following equation.

𝒂𝒂𝒚𝒚𝒂𝒂𝟓𝟓

= 𝟐𝟐𝟓𝟓𝟓𝟓

a. 26𝑥𝑥6

b. 𝑥𝑥6

3+ 𝑐𝑐

c. 10𝑥𝑥4 + 𝑐𝑐

d. 10𝑥𝑥6 + 5

e. 2𝑦𝑦 ∙ 𝑥𝑥5 + 5𝑥𝑥

Appendix 1b Test Questions – Pharmaceutical Calculations

1. A chemist makes 0,37g ofpotassium chloride. The molarmass of KCl is 74,5 g mol-1. Whatis the amount of KCl made (mols)?Please provide answer in scientificnotation.a. 4.966 x 10-3 molb. 0,005c. 4,9 x 10-3 mold. 5,0 x 10-3 mol

2. If a 300 mL bottle of a productcontains 15 g of ingredient A, howmuch is there in 5,0 mL?(Hint: 𝑚𝑚1

𝑉𝑉2= 𝑚𝑚2

𝑉𝑉2)

3. How much glucose is needed tomake up 750 mL of a 5% w/vconcentration?(Hint: 𝑚𝑚 = 𝑐𝑐

𝑉𝑉)

4. How much of ingredient A shouldyou add to 100 mL of a 10% v/v

solution to increase it in strength to a 20% v/v solution?

5. During the course of a reaction,the entropy S changes from -15,1J K-1 mol-1 to -32,5 J K-1 mol-1.Calculate the numerical (no units)value of ΔS.

6. The molar mass of sulphuric acidis 98 g mol-1, and the molar massof water is 18 g mol-1. What is thetotal combined mass of 2 moles ofsulphuric acid dissolved in 12moles of water?

7. A chemical reaction occurs. Theinitial amount of the limitingreagent is 1,25 mol. After reactionand purification, the amount ofproduct is 0,85 mol. What is thepercentage yield?

8. We define the gradient (slope) of agraph as:

𝑔𝑔𝑔𝑔𝑎𝑎𝑔𝑔𝑔𝑔𝑒𝑒𝑔𝑔𝑔𝑔 = 𝑦𝑦2 − 𝑦𝑦1𝑥𝑥2 − 𝑥𝑥1

Calculate the slope that passes through the following co-ordinates, (4,1 , 3,0) and (5,5 , 12,0)

9. The ideal gas equation:

𝑝𝑝𝑝𝑝 = 𝑔𝑔𝑛𝑛𝑛𝑛

relates volume, V, of an ideal gas to temperature, T, the amount of gas, n, and the pressure, p. R is the gas constant.

Rearrange the equation to make n the subject and find the amount of moles of gas in a 24,0 L container at 298 K and 5 x atmospheric pressure (5,00 atm).

Hint: The gas constant, R, is 0,082057 L atm K-1 mol-1

10. The mole fraction, x, of asubstance is defined as:

𝑥𝑥 =𝑎𝑎𝑚𝑚𝑎𝑎𝑎𝑎𝑔𝑔𝑔𝑔 𝑎𝑎𝑜𝑜 𝑥𝑥(𝑔𝑔𝑔𝑔 𝑚𝑚𝑎𝑎𝑚𝑚𝑚𝑚)𝑔𝑔𝑎𝑎𝑔𝑔𝑎𝑎𝑚𝑚 𝑔𝑔𝑎𝑎𝑚𝑚𝑏𝑏𝑒𝑒𝑔𝑔 𝑎𝑎𝑜𝑜 𝑚𝑚𝑎𝑎𝑚𝑚𝑚𝑚

A mixture contains 3 components, A, B and C. The mixture comprises 4,5 mol of A, 3,2 mol of B and 11,6 mol of C. What is the mol fraction of B, xb.

20

Hvem har Ansvaret? - Om studerendes ogunderviseres forståelse af begrebet ‘vejledning’på kurset Applied Environmental and NaturalRessource Economics

Ayoe Hoff

Institut for Fødevare- og RessourceøkonomiKøbenhavns Universitet

Hvem har Ansvaret?

Om studerendes og Underviseres forståelse af begrebet ’vejledning’ på kurset Applied Environmental and Natural Ressource Economics

Ayoe Hoff Institut for Fødevare- og Ressourceøkonomi

Introduktion

Kurset ’Applied Environmental and Natural Ressource Economics’ (AE-NE) er et obligatorisk fag på Kandidat uddannelsem i Miljø og Naturressource-

økonomi. Formålet med kurset, der løber over en blok, er at give de stude-rende mulighed for at formulere, udføre og afrapportere et kortere projektinden for Miljø og Naturressource økonomi. De studerende vejledes i denneproces af en af kursets undervisere. Det bliver i starten af kurset gjort klartfor de studerende at vejledningen primært omfatter skriveprocessen, og kun

248 Ayoe Hoff

i mindre grad det videnskabelige indhold. Denne præmis for kurset er nød-vendig, givet at kurset normalt har 2-3 undervisere, der ikke tilsammen kanhave detail-kendskab til ethvert af de emner de studerende vælger, selvomunderviserne naturligvis alle har en solid baggrund inden for naturressourceøkonomi.

Kurset får generelt ros i evalueringerne. Mange studerende finder at kur-set er en god træning, inden specialet, i at arbejde med et selvvalgt projekt.Dog er nogle studerende utilfredse med at der kun vejledes i skriveproces-sen. En studerende udtaler: "I think it is a waste of time write a projectabout a random topic, which is only marginally known by the professor”,og en anden studerende siger "I don’t agree that the advisors should on-ly help with the writing process. . . I would appreciate some advice on thecalculations and the sources of our data so that I can be confident in theeconomics as well as in the writing". Disse udtalelser peger på to centraleproblemstillinger i en vejlednings-dynamik: (i) kan der altid opnås enighedi den forventningsafstemning der bør ligge til grund for vejledningen? Og(ii) har underviser og studerende forskellige opfattelser af begrebet ’vejled-ning’? Der må antages at svaret på spørgsmål (i) i høj grad må afhænge afsvaret på spørgsmål (ii): Hvis underviser og studerende er enige om hvadbegrebet ’vejledning’ dækker over bør der også kunne opnås enighed i for-ventningsafstemningen, mens der omvendt kan opstå konflikter hvis der eruenighed om hvad ’vejledning’ dækker over.

I denne opgave undersøges hvilken opfattelse hhv. studerende og under-visere på kurset AENE har af begrebet ’vejledning’. Dette med henblik påat sikre alignment mellem kursets læringsmål og de studerendes forvent-ninger til kurset i fremtiden. I undersøgelsen afprøves et ’forventningsaf-stemningsskema’ (Rienecker et al., 2015), og et delmål med opgaven erat evaluere i hvor høj grad brugen af et sådant skema kan være nyttigt ifremtidig forventningsafstemning på kurset. Studerende der har taget kur-set i 2018, plus undervisere på kurset, er blevet bedt om at besvare skemaet.Herudover er 4 studerende og 2 undervisere blevet interviewet med henblikpå en kvalitativ uddybelse af deres svar. Opgaven afsluttes med en eva-luering og opdatering af forventningsafstemningsskemaet med henblik påfremtidig brug i kurset.

20 Hvem har Ansvaret? Forståelse af begrebet ‘vejledning’ 249

Baggrund

Den Danske Ordbog definerer det at ’vejlede’ som ’Hjælpe nogen som ermindre vidende eller erfaren end en selv på et bestemt område, fx ved atgive oplysninger eller vise hvordan noget skal udføres’1. Som sådan ervejledning et relativt løst begreb idet det kan strække sig fra at ’give op-lysninger’, der kan leder den vejledte på vej, f.eks. gennem henvisning tillitteratur, til at ’vise hvordan det skal gøres’.

Derfor kan ’vejledning’ i universitets sammenhæng være et vidt begreb,og det kan dermed være svært præcis at fastslå hvad processen indebærer.Dette kan ifølge Wichmann-Hansen et al. (2007) netop være grunden tilat begrebet ’vejledning’ ikke er formaliseret i Universitetsloven (2018).2

Dette selvom ordene ’vejlede’ og ’vejledning’ f.eks. indgår 113 gange iStudieordningen for the Naturvidenskabelige fakultet (2018), hvor ordenebruges i forbindelse med at beskrive de rent administrative forpligtelser derligger i en vejlednings situation på Københavns Universitet.

Wichmann-Hansen et al. (2007) præsenterede et litteraturstudie af deninternationale forskning der op til 2007 var blevet udført vedrørende vejled-ning på universiteter. De konkluderede at dette stadig i 2007 var et relativt’ungt’ forskningsområde, med de fleste af de analyserede studier publicereti de 10 år ledende op til 2007. Der kan derfor være sket meget siden 2007 iforskningen inden for universitets-vejledning, men idet f.eks. Rienecker etal. (2015) inddrager denne artikel i konklusioner vedrørende vejledning an-tages i denne opgave at konklusionerne fra Wichmann-Hansen et al. (2007)stadig er relevante. Hovedkonklusionen i Wichmann-Hansen et al. (2007)er at en vellykket vejlednings proces i høj grad afhænger af om der er engod relation mellem underviser og studerende, og at det er af sekundærbetydning for de studerende om vejlederen er ekspert i deres valgte emne.Desuden understreges at konflikter opstår når vejleder og studerende haruafklarede forventninger til hinanden, noget der også fremhæves af Kiley(2003).

Et første skridt på vejen til den gode vejledningsproces må derfor være,som også understreges i Rienecker et al. (2015), at underviser og studeren-

1 https://ordnet.dk/ddo/ordbog?query=vejlede2 Wichmann- Hansen et al. Henviser til Universitetsloven af 2003, men en hur-

tig søgning gennem Universitetsloven (2018) viser at denne stadig ikke specifiktangiver hvad en vejlednings situation bør omfatte, kun vejlederens formelle ad-ministrative pligter.

250 Ayoe Hoff

de sammen afdækker og diskuterer eventuelt divergerende forventninger tilhvad en vejlednings proces indeholder. Et udgangspunkt for denne diskus-sion kan være at bruge et forventningsafstemningsskema (Rienecker et al.,2015), der kan udfyldes af både vejleder og den studerende enten inden ellerved det første vejledningsmøde. Gennem sammenligning af svar kan even-tuelt divergerende meninger vedrørende den kommende vejledningsprocesafdækkes, og en fælles forståelse og aftale indgås om hvordan processenskal forløbe.

Dataindsamling

Et forventningsafstemningsskema (Appendiks A), baseret på eksemplet gi-vet i Rienecker et al. (2015, side 335), er blevet konstrueret med specielthenblik på kurset AENE. Skemaets spørgsmål er udvalgt med henblik på atadressere den kritik der har været i evalueringerne vedrørende at der kun imindre grad vejledes fagligt. Som sådan er der i skemaet lagt vægt på (i)den praktiske vejlednings proces (hvem bør arrangere møder, hvor megetbør vejleder læse af opgaven), og (ii) at vejledningen primært er orienteretmod skriveprocessen (bør vejleder have detail-kendskab til den studerendesemne, hvem sætter ambitionen for opgaven, hvem har ansvaret for kvalite-ten af den færdige opgave). 13 studerende og 4 undervisere (3 nuværendeog en kommende) har besvaret skemaet.

Med henblik på opfølgning og uddybning er 4 studerende og 2 lærereefterfølgende blevet interviewet med basis i de følgende spørgsmål:

1. Hvordan opfatter du begrebet ’vejledning’?2. Bør underviser have detaljeret kendskab til det emne der vejledes i?3. Hvem sætter ambitionsniveauet for opgaven?4. Bør vejleder læse hele opgaven?

Resultaterne af både besvarelser og skema er løbende blevet diskuteretmed min faglige pædagogikum-vejleder.

Resultater

Besvarelse af forventningsafstemningsskema

Resultaterne af besvarelsen af forventningsafstemningsskemaet er vist i fi-gur 20.1. Her ses for det første at studerende og undervisere er relativt enige


når det kommer til spørgsmålet om hvem der skal vælge emne i opgaven(spørgsmål 1), hvor begge parter mener at det bør den studerende selv. Li-geledes er der stor enighed om at det er den studerende selv der bestemmerhvornår der er brug for møder med vejlederen, og den studerende der tageransvar for at arrangere disse (spørgsmål 5). Videre er der også nogenlundeenighed om at den endelige kvalitet af opgaven er den studerendes ansvar(spørgsmål 6), selvom en mindre del af de studerende (17%) mener at denendelige kvalitet er lige dele den studerendes og vejleders ansvar.

252 Ayoe Hoff

Hvem har ansvaret? Ayoe Hoff

3

Figur 1. Resultater af svar på forventningsafstemningsskema vedrørende kurset AENE. Skemaet er besvaret af 13 studerende og 4 undervisere på kurset.

1. It is the supervisor’s responsibility to select a topic for the student to work with

It is the student’s responsibility to select a topic to work with, either based on the list of suggested topics, or another topic of her/his choice

2. The supervisor should only accept to supervise students if the supervisor has specific knowledge about the subject the student is working with

The supervisor can accept to supervise a student, even though the supervisor does not have specific knowledge about the subject the student is working with

3. It is the supervisor that determines which theoretical framework is the most appropriate to use in the student’s assignment

It is the student that determines the appropriate theoretical framework for her/his work

4. The supervisor sets the level of ambition for the student

The student sets her/his level of ambition and the supervisor should adapt her/his supervision according to this

5. It is the supervisor’s responsibility to arrange frequent meetings with the student

It is the student’s responsibility to determine when she/he needs to meet the supervisor, and arrange these meetings

6. The supervisor has direct responsibility for the final standard (format and scientific content) of the assignment

The supervisor’s role is to advise, and the final standard of the assignment is ultimately the student’s responsibility

7. The supervisor should read and comment drafts of every section of the assignment

The supervisor can read parts of the assignment and give constructive criticism of these that can be used generally for the whole assignment

Figur 20.1. Resultater af svar på forventningsafstemningsskema vedrørende kursetAENE. Skemaet er besvaret af 13 studerende og 4 undervisere på kurset.

Modsat disse er der større uenighed om de øvrige spørgsmål. For detførste mener 54% af de studerende at vejleder kun må acceptere at vejlede


studerende hvis vejleder har specifik viden inden for det emne de studeren-de har valgt (spørgsmål 2), mod at 75% af underviserne mener at man godtkan vejlede uden at have specifik viden inden for det valgte område. Lige-ledes mener 58% af de studerende at vejleder bør bestemme den teoretiskeramme der bør bruges i arbejdet med det valgte emne (spørgsmål 3), mensunderviserne er mere i midten (75%) eller mener at den studerende selv børvælge den teoretiske ramme (25

Tilsammen peger spørgsmålene 2 og 3 på uoverensstemmelse mellemunderviseres og studerendes forventninger til hinanden, og måske på en delaf forklaringen på de kritiske kommentarer i kursus evalueringen. De stude-rende mener at vejleder bør have kendskab til og tage stilling til det fagligei lige så høj grad som til skriveprocessen. Underviserne mener at de, givetkursets præmis, primært skal vejlede i det sidste. Man kan overveje grundentil denne uoverensstemmelse, der kan tolkes som en vis faglig usikkerhedhos de studerendes. Disse står lige for at skulle starte deres speciale, mensvarene på spørgsmål 2 og 3 tyder på at de stadig gerne til en vis grad vilkunne støtte sig til en vejleder hvad angår fagligt indhold.

Videre mener alle undervisere at den studerende selv sætter sit ambi-tionsniveau (spørgsmål 4) mens de studerende ligger mere spredt og oghen mod at underviserens også har ansvar for at sætte ambitionsniveauet.Denne uoverensstemmelse skyldes forskellige opfattelser af spørgsmålet,hvilket uddybes herunder.

Og endelig mener alle undervisere at vejleder bør læse hele opgaven(spørgsmål 7), mens de studerende igen spreder sig mere ud, og 59% faktiskmener at underviser ikke bør læse det hele.

Resultater af interview

Appendiks B giver et overblik over de vigtigste pointer fra interviewene,der opsummeres og diskuteres herunder.

På spørgsmålet ’hvordan opfatter du begrebet vejledning’ er der en visadskillelse mellem de studerendes og undervisernes meninger. De inter-viewede studerende er til dels enige om at der bør være et fagligt elementi vejledningen (studerende nummer 2 og 3 i højere grad end studerende #1og #4). Vejleder bør ’guide’, være en ’man kan spørge til råds’, bør ’kunnesætte en retning’/’direct in the right way’, bør kunne vejlede i ’the scientificstuff’, og kunne give ’critical reflection’ på indholdet. Studerende #4 menerdog videre at ’I don’t think it should be too detailed in terms of contents asit really should be our report’. Underviserne mener omvendt at fokus bør

254 Ayoe Hoff

være på at ’den studerende får en god proces’, at de lærer at ’struktureretanker på et stykke papir’ og at en stor del af vejledningen er at ’hjælpedem igennem i detaljer med en problemformulering’. Disse svar reflektererden konflikt der kan ses mellem forventninger i spørgsmål 2 og 3 i forvent-ningsafstemningsskemaet. Dvs. at mange af de studerende forventer fagligsnarere end procesvejledning på kurset, mens kursets undervisere i høj gradfokuserer på at vejlede i proces.

På spørgsmålet ’ Bør underviser have detaljeret kendskab til det em-ne der vejledes i?’ er der delte meninger blandt de studerende. Studerende#2 og #3 mener vejleder behøver ’more than just broad knowledge aboutthe general topic of resource economics’ og at vejleder ‘to give some criti-cal reflection on the content has to know something about the content youare working with’. Studerende #1 mener at idet det var ‘formålet med fa-get’ at være ’proces og skrive rapport’ så ’kan man godt blive vejledt udenat vejleder ved alt om emnet’, men dog ’have lidt kendskab til det mansnakker om’. Studerende #4 starter med at sige ’I guess it is actually notneccesary’, men fortsætter så ’I guess it feels like you are kind of in thedarkness if someone does not know what you are writing about’, dvs. erlidt uenig med sig selv. Underviser #1 mener derimod at ’det er jo igen detproces vejledning. . . så detail sektor kendskab har vi ikke brug for’, og at’økonomer bruger den samme begrebsramme inden for. . . mange forskelli-ge research områder’. Og endelig mener underviser #2 at ’der er jo ingentvivl om at hvis du fik en der vidste noget specifikt inde for det præciseområde, så får du en bedre vejledning. . . men derfor kan man godt vejlede,specielt processen’. Svarene på dette spørgsmål understøtter dermed til envis grad svarene på spørgsmålet ’hvordan opfatter du begrebet vejledning’,altså at de studerende vil have en vejleder der ved noget specifikt om det delaver, mens underviserne mener de kan give fyldestgørende vejledning vedat støtte op om processen og gå ind med generel viden om naturressourceog miljø økonomi.

På spørgsmålet ’Hvem sætter ambitionsniveauet for opgaven’ viste detsig at undervisere og studerende for en stor del så dette spørgsmål forskel-ligt. Alle studerende lægger vægt at de selv sætter ambitionen for den en-delige karakter gennem deres arbejdsindsats. Men derudover forventer stu-derende #1, 2 og 3 at vejleder tager ansvaret for at opgaven ikke bliverfor ambitiøs, givet den begrænsede tid der er til at lave den. En studerendesiger ’ Man kan også have et meget stort ambitionsniveau og gerne villenå alt for meget, og så synes jeg det er helt vildt flot hvis vejlederen siger’jeg tror du skal tænke lidt anderledes, at du skal gøre det lidt anderledes


hvis du vil gøre det godt”. Ingen af underviserne, inklusive mig selv, havdeset spørgsmålet på denne måde. Så først adspurgt siger en underviser ’ Detkommer an på hvad man tænker om level of ambition, der forstår jeg atdet er deres karaktermæssige ambitionsniveau’. Men da de to interviewe-de undervisere bliver gjort opmærksom på de studerendes synsvinkel er deenige; ’Hvis det er med hensyn til afgrænsning, så er det klart, der skal vitage et ansvar’ og ’Men det er klart det der med at få fokuseret opgavenind i en problemformulering, det er altså ret vigtigt’. Denne uoverensstem-melse mellem initiale opfattelser af spørgsmålet har været interessant og ervigtig at tage med når skemaet bruges i fremtiden.

På spørgsmålet ’Bør vejleder læse hele opgaven’ afspejles resultaternefra skemaet i de svar der gives. En studerende mener at ’of course I wanthim to the entire paper to some extent’ mens de to øvrige studerende ermere ydmyge mht. hvad de kan forvente af deres vejleder: ’jeg ville da heltklart synes det var rarest, jeg tror ikke jeg ville forvente at vejlederen vilgøre det’ og ’It would be nice if the supervisor is ready to do that. . . yes - Ido not expect that the professor spends that much time doing that’ og ’ yes Iwould like that but I guess, if two pages are not read it is not a catastrophy’.Omvendt er underviserne enige om at på dette kursus skal hele opgavenlæses, ud fra den præmis at der skal være enighed om tilgangen til opgaven,dvs. hvis en vejleder læser det hele så gør alle vejlederne det. De er ogsåenige om at grunden til at læse det hele på dette kursus er at kurset primærter proceslære; ’. . . men det fungerer jo meget godt i sådan en sammenhæng,og det er jo en læringsproces i forhold til at styre processen selv’.

Diskussion og konklusion

Resultaterne af denne lille undersøgelse belyser en række vigtige problem-stillinger og pointer for kurset ’Applied Environmental and Natural Res-source Economics’. For det første ses en vis uoverensstemmelse mellemhvordan studerende og undervisere på kurset opfatter hvad begrebet ’vej-ledning’ betyder på dette kursus; De 4 interviewede studerende vil gerneguides også i den videnskabelige del af deres arbejde på kurset, hvor kursetsgrundpræmis omvendt er proces vejledning. Umiddelbart er der ikke nogendirekte løsning på dette problem, idet kurset, som nævnt i indledningen,har 2-3 undervisere der ikke tilsammen kan have detail kendskab til allede emner de studerende vælger. Det er dog gået op for mig, i arbejdet meddenne opgave, at vi undervisere på kurset ved den indledende forelæsning

256 Ayoe Hoff

kun kort fortæller de studerende at der vejledes i ’skriveprocessen’ udenat uddybe hvad vi mener med dette. Og i mine interview er det kommetfrem at mange studerende derfor tror at vejleder primært retter kommafejlog sprog. Derfor mener jeg at en del af dette problem kan løses gennem enlangt tydeligere forventningsafstemning i den indledende forelæsning. Or-det ’skriveproces’ skal erstattes af ordet ’proces’ og det skal gøres klart forde studerende at den ’procesvejledning’ der tilbydes, omfatter vejledning iproblemformulering og afgrænsning, disposition af både det skrevne og aftid, og i opsætning af et videnskabeligt arbejde. Disse ting er alle noget destuderende efterspørger og i høj grad har værdsat i løbet af kurset, specieltvejledningen i afgrænsning. Gennem at gøre dette klart i starten af kursetmener jeg at de studerende i mindre grad vil være utilfredse med kurset frastarten, og mere fokuserede på at søge og modtage den faktisk omfattendeprocesvejledning der tilbydes.

Videre er det imponerende at se hvor selvstændige, og i høj grad ogsåydmyge, de studerende er i forholdet til deres vejledere. De studerende me-ner det i høj grad er deres eget ansvar at holde processen gående gennemselv at opsøge kontakt med deres vejleder når det er nødvendigt, og de me-ner ikke de kan forvente at deres vejleder læser hele deres opgave, selvomde vil føle sig mere sikre hvis dette sker. Derfor har kurset i sin nuværendeform en stor fordel idet underviserne er enige om at læse hele opgaven forde studerende de vejleder. Dette bør derfor fremhæves ved den indledendeforelæsning, idet det viser at selvom de studerende til en vis grad er ’ale-ne’ når det kommer til det faglige, holder vejlederen et sikkerhedsnet underdem ved at læse den samlede opgave.

På baggrund mine interview med de studerende og underviserne viljeg konkludere at forventnings-afstemnings skemaet kan være et værdifuldtredskab på kurset AENE idet det kan skabe dialog mellem studerende ogvejleder vedrørende vores forventninger til hinanden. Hidtil har hver un-derviser på kurset haft en kort snak ved første vejledningsgang med deresstuderende, hvor vi har gjort klart hvad vi kan tilbyde og hvad vi forven-ter af den studerende (f.eks. at de selv tager kontakt når de har behov forvejledning). Vi har også spurgt de studerende hvad de forventer af os, menjeg mener at dette måske kan være svært for de studerende at overskueog (turde) formulere i deres første møde med deres vejleder. Hvis beggeparter har udfyldt et forventningsafstemningsskema inden første møde, vildenne dialog blive mere målrettet og de studerende kan på en mere trygmåde give udtryk for deres mening, idet denne allerede er angivet på ske-maet. Givet at jeg, gennem mine interviews, har fundet ud af at specielt


spørgsmålet vedrørende ’level of ambition’ kan være en smule misvisendehar jeg udfærdiget et revideret forventningsafstemningsskema (AppendiksC), hvilket jeg og de øvrige undervisere er enige om at bruge på det næsteAENE-kursus. ’Level of Ambition’ spørgsmålet er delt op i 2 del spørgs-mål, et der vedrører hvem der har ansvar for at begrænse det valgte emnehvis det er for omfattende, og hvem der har ansvar for den endelige karak-ter. Resten af skemaet har jeg beholdt i sin oprindelige form, idet jeg menermine interviews med studerende og undervisere har vist det fungerer godt,hvis det netop bruges som et dialogværktøj.

Som sådan håber jeg at vi på det næste AENE-kursus er bedre klædt påtil at komme de studerende i møde, både ved den indledende forelæsningog i vejledningen, og derigennem forebygge den frustration over kursetspræmis der kom til udtryk i kursets evaluering i forsommeren 2018.

Afsluttende vil jeg sige at jeg har haft stor glæde af at lave denne op-gave. Jeg har fået en dybere forståelse for hvem de studerende på kursetAENE er, og hvordan vi forhåbentlig kan hjælpe dem til at få en bedreproces på det næste kursus gennem målrettet dialog vedrørende forvent-ningsafstemning. Jeg har løbende diskuteret opgaven med min faglige pæ-dagogikum vejleder Frank Jensen, og de angivne fortolkninger af resultaterherover, specielt på spørgsmål 2 og 3 i forventningsafstemningsskemaet,afspejler i høj grad vores diskussioner.

Referencer

Kiley, M. (2003). Conserver, Strategist or Transformer: The experiences ofpostgraduate student sojourners. Teaching in Higher Education, 8(3), 345-356.


Studieordningen for the Naturvidenskabelige Fakultet (2018). Den fællesdel af bachelor- og kandidatstudieordningerne for uddannelserne ved det

258 Ayoe Hoff

Natur- og Biovidenskabelige Fakultet, Københavns Universitet 2018.https://www.science.ku.dk/studerende/studieordninger

Universitetsloven (2018). Bekendtgørelse af lov om universiteter.https://www.retsinformation.dk/Forms/R0710.aspx?id=198434

Wichmann-Hansen, G., Eika, B., & Mørcke, A. M. (2007). Hvad findesder af litteratur om vejledning?–Litteratursøgning med fokus på publicere-de, evidensbaserede studier. Dansk Universitetspædagogisk Tidsskrift, 2(3),11-19.


A Forventningsafstemningsskema brugt i undersøgelsenHvem har ansvaret? Ayoe Hoff

8

Appendiks 1: Forventnings afstemnings skema brugt i undersøgelsen Please read the conflicting statements in the table below. You might not agree fully with either of the

statements. Therefore, please estimate your position and mark it on the scale with an ‘x’ below your choice.

I.e. if you fully agree with the left statement put an ‘x’ under ‘1’, if you fully agree with the right statement

put an ‘x’ under ‘5’ and if you are not in full agreement with either, put an ‘x’ under ‘2’, ‘3’ or ‘4’ depending

on the degree you either agree with the left or right statement.

‘The supervisor’ referred to here is one of the supervisors on the course ‘Applied Environmental and Natural

Resource Economics’, i.e. with an in-depth knowledge about general resource economics.


1

2

3

4

5



1

2

3

4

5



1

2

3

4

5


4. The supervisor sets the level of ambition for the student

1

2

3

4

5

The student sets her/his level of ambition and the supervisor should adapt her/his supervision according to this


1

2

3

4

5


6. The supervisor has direct responsibility for the final standard (format and scientific content) of the assignment

1

2

3

4

5

The supervisor’s role is to advise, and the final standard of the assignment is ultimately the student’s responsibility


1

2

3

4

5


May I contact you for a short interview regarding your answers? Yes No

Hvordan opfatter du begrebet ’vejledning’:

Studerende #1 Det indebærer vel at få en eller anden form for hjælp og sådan guiding til sin opgave…at man kan spørge til råds om nogen ting, og man kan få hjælp til at sætte en retning på det

Studerende #2 Of course, I want the supervisor to comment on general stuff in the way I write. But at least for me personally that is not where I feel that I need supervision. I need supervision with the scientific stuff, because I'm just the student and my supervisor is an expert.

Studerende #3 For me the supervision should be focused mainly on helping you content wise by suggesting some things or giving critical reflection on what you have done content wise, and maybe set some deadlines to help.

Studerende #4 I think it is directing you in the right direction kind of, and I don't think it should be too detailed in terms of contents because it really should be our report…to say to someone 'remember to be critical' is not so much like telling you 'you should use this model' - I mean it is also quite a broad direction.

Underviser #1 Man skal ikke være så fokuseret på at den studerende skal klare sig godt til eksamen, det er den studerendes eget ansvar, men man skal lede de studerende så de får et eller andet ud processen … jeg synes at i stedet for at fokusere på alle skal have 12, - så synes jeg vi i stedet skal fokusere på at vejlede dem så de få en god skriveproces og en god oplevelse ud af at skrive det…og at de lærer at strukturere tanker på et stykke papir.

Underviser #2 Man skal styre processen en lille smule, det er den studerendes eget ansvar men man prøver at hjælpe dem lidt ved at fastsætte en mødeplan og deadlines… at vejlede i det her kursus, det ligger i at… det at hjælpe dem igennnem, i detaljer, med en problem-formulering…Når de så kommer i gang med at skrive går vejlednings opgaven også ud på at læse igennem og given nogle få kommentarer hist og her.

260 Ayoe Hoff

B Vigtigste pointer fra Interview

Bør vejlederen have specifikt kendskab til det emne der vejledes i?

Studerende #1 Den opgave vi skrev den handlede også meget om proces og om at skrive en rapport, så det synes jeg sagtens man kan blive vejledt på uden at vejlederen ved sådan alt om det emne… også fordi at det var - som jeg forstod var det formålet med faget…selvfølgelig skal vejlederen have lidt kendskab til det man snakker om - men man behøver ikke at vide alt.

Studerende #2 Of course, there are limits to how much you can…but since you are always focusing on this small little specific thing you kind of need more than just broad knowledge about the general topic of resource economics.

Studerende #3 To give some critical reflection on the content the supervisor has to know something about the content you are working with. It does not have to be that that is like the main role of your supervisor, that that person is completely specialized in that field, but at least know a little bit about it…to help the student along the way of finding what to write about exactly.

Studerende #4 We are only talking about this course right? So I guess it is actually not necessary. I think I was thinking about my master’s thesis. Because this course was also set up as kind of a test-run for your master’s thesis…yes, so for this course I think it is fine I guess, in the sense that you are not really guiding us on the scientific stuff of it. …. I guess it feels like you are kind of in the darkness if someone does not know what you are writing about at all and you are still judged on the content of the paper and you are not just judged for, you know, format.

Underviser #1 Det er jo igen det der med proces vejledning, det er lige meget om man er fysiker eller humanist eller det ene eller andet, i kraft af den viden du har erhvervet dig gennem dit arbejde… så det der detail sektor kendskab det har vi slet ikke brug for. Det vi sidder og laver herinde, det økonomer sidder og laver, det er at de bruger den samme begrebs ramme inden for et hav af forskellige emner, og også inden for en masse forskellige research områder.

Underviser #2 Der er jo ingen tvivl om at hvis du fik en der vidste noget specifikt inde for det præcise område, så får du en bedre vejledning. Men derfor mener jeg godt man kan vejlede -specielt processen. Og så ved vi jo også godt noget om…kernen i miljø økonomi og ressource økonomi…på generelt plan. Og egentlig lidt af vejlederrollen, det er sådan set også at spille dum, få dem til at forklare tingene noget mere hvis det sådan er uklart, og stille spørgsmål hele tiden.

Hvem sætter ambitionsniveauet for opgaven

Studerende #1 Jeg synes at man selv skal sætte sit ambitionsniveau, men jeg synes godt man kan lytte hvis vejlederen siger at det er et meget svært emne. Man kan også have et meget stort ambitionsniveau og gerne ville nå alt for meget, og så synes jeg det er helt vildt flot hvis vejlederen siger 'jeg tror du skal tænke lidt anderledes, at du skal gøre det lidt anderledes hvis du vil gøre det godt', og det synes jeg er fint hvis man får at vide at ambitionsniveauet er for stort.

Studerende #21 -

Studerende #3 We kind of had a good notion of how many hours you were supposed to spend in the project, but the question was more how much can you do with those hours. And you cannot everything you want to in those hours, so that is why you have to limit yourself until a certain part and that where I think the help of the supervisor can come in.

Studerende #4 I mean in terms of level of ambition I am thinking grades, and of course I think you should just talk with your supervisor about that, robably in the beginning so that they know how to guide you.

Underviser #1 Det kommer an på hvad man tænker om level of ambition, der forstår jeg at det er deres karaktermæssige ambitionsniveau. Men…de er nødt til at få afgrænset deres emne, en god karakter er ikke en opgave der kommer omkring 100 emner. Hvis det er med hensyn til afgrænsning, så er det klart, der skal vi tage et ansvar. Vi skal ikke tage det fulde ansvar, for hvis de nægter at afgrænse så kan vi jo ikke gøre noget ved det, men vi kan prøve at guide dem igennem, uden at styre hvad de skal skrive om.

Underviser #2 Jeg skal ikke styre om de studerende skal have 2 eller 12, det er deres egen indsats. Men det er klart det der med at få fokuseret opgaven ind i en problemformulering, det er altså ret vigtigt, og det er en kunst som er ret svær når man ikke har prøvet det før.

Note: 1 Dette spørgsmål blev ikke stillet til Studerende #2 – desværre en simpel forglemmelse

Bør vejleder læse hele opgaven

Studerende #1 Optimalt ville jeg synes det var helt vildt rart hvis vejlederen læste det hele. Men jeg tror bare det var sådan at - altså hvis vejlederen ikke har tid til det. Jeg ville da helt klart synes det var rarest, jeg tror ikke jeg ville forvente at vejlederen vil gøre det. Men jeg synes også godt at man kan tage nogle generelle rettelser og så kan man selv tage det videre.

Studerende #2 We can use those comments as the general thing - but of course I want him to read the entire paper to some extent.

Studerende #3 If there is the time – yes, but I just do not think that is a reasonable expectation. For a project that could be in the end 20 pages, If the supervisor is expected to read all 20 pages, and that for 10 to 15 students in a course of 6-7 weeks, I think that is quite extensive. It would be nice if the supervisor is ready to do that…yes - I do not expect that the professor spends that much time doing that.

Studerende #4 I guess that is maybe a dream scenario - but I think I was also thinking like for my masters thesis - where I do want someone to look through my thesis before I hand it in. But, lets stick to this cours, yes I would like that but I guess, if two pages are not read it is not a catastrophy.

Underviser #1 Man kan godt bruge 10-15 minutter på at læse det igennem, men jeg synes man skal passe på med de detail kommentarer i meget stort omfang, for hvis det bliver alt for detaljerede kommentarer så overtager du sådan set ansvaret for den studerendes karakter.

Underviser #2 Det vi har vedtaget her, og det jeg bruger i det her kursus. Jeg læser det hele hvis de vil, og hvis de ikke sender det så er det jo deres egen skyld, men de har mulighed for at jeg læser det hele…man er nødt til at gøre det ens (mellem underviserne – red.), men det fungerer jo meget godt i sådan en sammenhæng, og det er jo en læringsproces i forhold til at styre processen selv.

264 Ayoe Hoff

C Udkast til endeligt forventningsafstemningsskema tilfremtidig brug på kurset AENEHvem har ansvaret? Ayoe Hoff

13

Appendiks 3: Udkast til endeligt forventnings afstemnings skema til fremtidig brug på kurset AENE Please read the conflicting statements in the table below. You might not agree fully with either of the

statements. Therefore, please estimate your position and mark it on the scale with an ‘x’ below your choice.

I.e. if you fully agree with the left statement put an ‘x’ under ‘1’, if you fully agree with the right statement

put an ‘x’ under ‘5’ and if you are not in full agreement with either, put an ‘x’ under ‘2’, ‘3’ or ‘4’ depending

on the degree you either agree with the left or right statement. Your supervisor will also fill out the scheme

and talk with you about it at your first meeting.

‘The supervisor’ referred to here is one of the supervisors on the course ‘Applied Environmental and Natural

Resource Economics’, i.e. with an in-depth knowledge about general resource economics.


1

2

3

4

5



1

2

3

4

5



1

2

3

4

5


4. The supervisor is responsible for helping the student narrowing the chosen subject down to something that can be researched and reported within the timeframe of the course.

1

2

3

4

5

The student is responsible for narrowing the chosen subject down to something that can be researched and reported within the timeframe of the course.

5. The supervisor has direct responsibility for the final standard (format and scientific content) and grade of the assignment

1 2 3 4 5 The student sets her/his level of ambition (regarding grade) and the supervisor should adapt her/his supervision according to this


1

2

3

4

5



1

2

3

4

5


Del VII

Improving practical work and exercise classes

21

Helping less to strengthen active student-centredlearning in exercise classes

Mette Frahm Olsen

Department of Nutrition, Exercise and SportsUniversity of Copenhagen

Introduction

There is increasing focus on student centred and student activating learningin university teaching as an approach to enhance deeper and more mean-ingful learning among students (Prince, 2004). Many courses include a mixof lectures and exercise classes, where students get a chance to solve prob-lems during exercises related to theoretical concepts presented in lectures.Although these exercise classes are student activating in nature, large diver-sity of the students often make it extremely challenging to find a level ofquestions that does not leave behind the weaker students, while still beingrelevant for the stronger students in the class.

I have been involved as a teacher in exercises about nutritional epidemi-ology as a PhD student, about 10 years ago, and again recently as a PostDoc. Several minor changes to the approach have been made in the mean-time, but the exercise classes are still not working well and suffer fromproblems arising from a huge variation in student background, experienceand skills.

The exercises are part of an MSc course and build on basic statisti-cal and data handling skills that the students are expected to have attainedduring their BSc programme. However, many students struggle with thesebasic skills and have difficulties doing the calculations required to be able tointerpret and discuss the epidemiological concepts. Over time, the approachfrom myself and other teachers involved has been to increase teacher-directed input in terms of including more detailed step-by-step guides inexercise instructions as well as having three teachers present to help solve

268 Mette Frahm Olsen

students’ questions during class. However, classes have continued to bedominated by frustrated students who get stuck in analyses and need helpfrom a teacher to move forward. The last 35-40 minutes of classes are spendin plenary discussion of the exercises and the challenge of covering allneeds in this discussion further highlights the issue of student diversity.

With the additional help offered, these exercises have become increas-ingly teacher-directed, instead of student-centred, with many of the studentsrequiring 1:1 input to be able to solve questions. With 60-70 students inclass, this is clearly not feasible and rather than being activated, many stu-dents spend the majority of time waiving their hand in the air and waitingpassively for help from a teacher.

During the University Pedagogy course, I was part of a pre-project,which looked at drivers of students’ transformation from a passive mode oflearning to a more proactive and self-driven mode of independent learning.After interviewing PhD students and supervisors, our main conclusion wasthat the independent learning mode can be promoted when students feelownership and take initiative, and that supervisors may facilitate this by be-ing less controlling, giving less detailed instructions and not spoon-feedinganswers, but instead providing resources and autonomy for students to findthe solution themselves. In other words, the opposite of what we have beendoing in my exercise classes. . .

Consequently, the teachers involved decided to implement some moreradical changes to the exercise classes this year: We removed presence ofall teachers during the main part of exercise classes and all teacher-directedhelp in the written instructions. Instead, we provided alternative online re-sources for students to see if this would push them toward more self-drivenactive learning.

Context: course, students and previous attempts toimprove learning in exercises

The exercises are part of the course “Evidence, Diet & Health” which isa compulsory 7.5 ECTS course placed in block 2 on first year of the twoMSc programmes: “Human Nutrition” and “Clinical Nutrition”. About athird of the students have a BSc in “Food Science” or “Exercise and SportScience” from University of Copenhagen, another third have a Bachelor(i.a. Professionsbachelor) in “Nutrition and Health” from University Col-lege Copenhagen and the last third are international students with various

21 Helping less to strengthen active student-centred learning... 269

backgrounds. The course includes lectures, journal clubs and a total of fourexercise classes, of which I am responsible for three.

Each of the exercise classes has a duration of three hours and studentswork in groups of four. Groups are made by the course responsible, whoaim to combine students with different types of background to get theweaker and stronger students working together. They are given a datasetand questions relating to a topic that has been presented in a lecture earlierin the course, e.g. “sample size and power”. Students can use whicheverstatistical software they prefer for analysing data.

The intended learning outcomes (ILOs) of the course include overallcompetences of assessing the validity of nutritional studies and evaluat-ing evidence of relationships between dietary components and health. Toachieve this, it is essential that students develop a good understanding of theepidemiological concepts that are discussed in exercise classes. The compe-tences are assessed in the final exam, where students are asked to criticallyassess nutritional studies reported in scientific papers. Course ILOs also in-clude more concrete knowledge and skills outcomes on data managementand analysis, such as “describe the structure of a data set and types of vari-ables” and “demonstrate ability to handle a data set with nutritional vari-ables and to assess for confounding and effect modification”. These ILOsare closely related to the content of my exercises and they are assessed sep-arately in a statistical assignment during the course, which must be passedbefore the exam can be taken.

One of the admission criteria of both MSc programmes is 7.5 ECTScredits of statistics, but in practice, many students are not confident withbasic statistics and they have no common software experience. This chal-lenge, along with the general diversity of student backgrounds and aca-demic skills, result in limited learning outcomes for the students. Much ofthe time is spent on struggling with statistical software, rather than dis-cussing the relevant epidemiology, which students generally find difficult.After the course, only few students ever use the same statistical softwareagain, so little is gained from these efforts.

The exercise classes require a lot more teacher resources than what isallocated in terms of UAT. The assisting teachers are often PhD students,who change from year to year, and each year we therefore need to findnew teachers and introduce them to the exercises. However, I think themain issue with these challenging exercise classes is that many studentsget immensely frustrated and stressed and the resulting atmosphere isn’tvery conducive for anyone’s learning.


Previous attempts to solve the problem have included a one week in-troduction to basic statistics and data analysis using R (stat. software). Theidea was for weaker students to catch up, but we found that it was notenough for these students, while others were bored and complained thattime was spent on topics that had been covered on their BSc programmes.Also, it left too little time for the actual course. The most recent attempt hasincluded providing hints for R coding in the exercise instructions. However,this hasn’t worked well either, as only students with good data handlingskills are able to read and use these hints and teachers end up handing outentire R scripts for the majority of students, when they get stuck.

New approach: Reducing teacher-directed input andproviding alternative resources

In collaboration with the course responsible and two other teachers involvedin the course, we decided to try out a new approach this year, where noteacher will be present during the first two hours of exercise classes. I re-arranged my exercise instructions to clearly divide questions into “calcula-tion” and “interpretation” sections. At the same time, I removed all hintsand help for solving the calculation questions, including equations, codingof analyses, sample size tables and other concrete input that was previ-ously provided in the instructions. I gave an introduction to the exercises(in the written instructions and orally when meeting the students), in whichI explained that the aim of the classes was not to teach statistics or use ofsoftware and that we would spend time in plenary discussing the purpose ofthe analyses and the implications of their results, rather than how the resultswere achieved.

An online toolbox for statistics was developed and set up as a separatesite on Absalon called “Introduction to statistics”, which included help forchoosing and installing statistical software, getting an overview of data, ba-sic data handling functions such as generating new variables and stratifyingdata, choosing a relevant test and conducting selected analyses. Some ofthe help modules are general (e.g. about choosing between tests) while oth-ers provide specific help for one of two selected statistical software (R andSPSS). The toolbox also includes links to additional online resources (e.g.an online sample size calculator and Youtube videos with instructions ofhow to load and open data in R).


The students were encouraged to look at the exercises before class and,if finding them difficult, to familiarize themselves with the content of thetoolbox, find their notes from previous courses, discuss with their group,read a book, ask a classmate, or consult google for a way forward.

I prepared two detailed answer files for the exercises, which includedcoding and output of analyses in R and SPSS, respectively. These weremade available on Absalon immediately after the exercise classes.

The aim of this new approach to the exercise classes was to redirectstudent attention to the concepts of epidemiology rather than the tools andanalyses involved. With less direct teacher input and more general resourcesprovided for software and analyses issues, we hoped to enable students tofind solutions on their own, rather than waiting passively for a teachersdirections.

Plan for evaluating the new approach

This was the fifth time I taught the exercises and I was therefore familiarwith how students usually respond during class. Part of the evaluation of thenew approach was therefore just based on my own subjective comparisonwith previous years of how many questions students manage to complete,where they get stuck, what type of questions they have, as well as an as-sessment of the general frustration level in class.

I joined the classes after two hours and spend about 20-25 minuteswalking between groups before the discussion in plenary. When talking toindividual groups, I asked students how far they had come, what they foundwas easy or difficult, how they had made the calculations and analyses, andwhat resources they had used to solve them.

I ended the plenary discussion with a short oral evaluation, asking stu-dents if they found the exercises were useful, if they had found sufficientresources for working on the questions, and if time was well spend on groupwork and plenary discussion.

The course is still ongoing at the time of writing this assignment. Afterits completion, it may also be relevant to compare students’ evaluations andexam results, including outcomes of the statistical assignment, to previousyears. Several other factors may play into this, of course, but it might stillbe useful for assessing if students’ learning outcomes have improved afterchanging the approach to exercise classes.


Findings

While moving between the groups two hours into the exercise classes, itwas clear that many students found the exercises difficult and the majorityhad only been able to complete a few of the questions. This was not verydifferent from previous years, but there was a striking difference in the gen-eral atmosphere among students. Whereas these classes are usually chargedwith frustration and accusations of teachers being too few and too slow torespond to requests for help, students were now using their energy on try-ing to find solutions(!) Not all students had used the toolbox we made, butnearly everyone was doing something to find a solution. Approaches mainlyincluded googling and using notes and other material from their BSc courseon statistics. The students with stronger data analysis skills were very activein explaining and helping group mates as well as other groups.

In the oral evaluations, many said that they found exercises were diffi-cult but that they were able to find useful resources. Some students said thatboth group work and plenary discussions were useful for learning, whileothers asked for less time working in groups and more time discussing inplenum, since they had not been able to solve the majority of questions ontheir own. Some also asked for a teacher to be present during group work,so they could solve their questions faster. However, after discussing the aimof the classes, there was better support for the approach.

Discussion and conclusion

I found the reduction of teacher-directed help in exercise classes was ex-tremely conducive for improving independent and active student-centredlearning. I must admit that the idea came from others and that I was hes-itant to try it at first. Teaching a class of frustrated students can be quiteintimidating and my own intuitive approach, up until now, has been to meetstudents with more and more detailed guidance, when they were not ableto solve the problems we presented them with. I think it is interesting andactually a bit of a paradox that students previously were given much moredirect help, and still complained that they did not receive enough, whilethis year’s students seemed much more at ease with only being presentedto strategies with finding help elsewhere.

This new approach did not seem to help students much in terms ofsolving more questions during exercise classes, but the fact that they man-aged to find some of the solutions on their own, has probably resulted in a


more meaningful understanding of the concepts, rather than rote learning ofcodes provided (Entwisle, 2009). In addition, students are now more awareof where to find resources that will be useful for other situations, whereasthe direct help for specific analysis in previous versions was difficult tomake use of later on.

I am not sure how many students will work on the remaining questionsafter class, but I think it would be very useful for them to spend more timeon the exercises and to compare their approach and results with the answerfile that I prepared. In next year’s course, I will emphasize this and encour-age students to get back to me in the following class, if they have questionsor difficulties with the previous exercises.

The use of teacher resources for the course now better match the UATsallocated. On a more general note, I think it is notable, that although it iswell-known and widely acknowledged that better learning is achieved whenstudents are active, they often request more teacher-driven input and this isoften also the intuitive response from teachers. In the case described here,it required an active push out of the comfort zone for both students andteachers to improve student-centred learning.

It is also generally acknowledged, that the existing level of knowledgeamong students is one of the most important factors influencing learningand that teaching should be adjusted according to an assessment of thestudents (Ausubel et al., 1978). This highlights the challenge of teachingin a class of highly diverse students. The student-driven approach pro-vides some flexibility and allows for differentiated teaching, whereas a veryteacher-directed approach must aim to fit all.

In this course, the challenge of activating students was exacerbated bythe large diversity among them, but the strategy of helping less to promotestudent-centred learning will be equally relevant in a less diverse group ofstudents.

Our University Pedagogy pre-project on how to facilitate a more inde-pendent mode of learning also included a conclusion about giving studentsenough time to get stuck and figure out on their own how to move on. Thisis probably one of the central experiences for becoming self-driven, but of-ten not feasible or realistic to do as part of courses. However, after this ex-periment with exercise classes with limited teacher-directions, I think theycan serve as an opportunity for students to be exposed to a more proactiveapproach to finding solutions.


References

Ausubel, D. P., Novak, J. D., & Hanesian, H. (1978). Educational psychol-ogy: A cognitive view. New York: Holt, Rinehart & Wilson

Entwisle, N. (2009). Teaching for understanding at university. Basingstoke:Palgrave Macmillan

Prince, M. (2004). Does active learning work? A review of the research.Journal of engineering education, 93(3), 223-231.

22

The learning of veterinary students duringpractical ultrasound exercises

Anna Müller

Department of Veterinary Clinical SciencesUniversity of Copenhagen

Introduction

This report concerns a part of the course Veterinary Imaging1, which amandatory course in the Master of Veterinary Medicine curriculum. First,the course is briefly introduced. Then a background for the project is given,based on current literature, qualitative interviews and previous student eval-uations. This information forms the basis of a number of interventions thathave been introduced in the course. Finally, qualitative feedback from stu-dents who took the course, and a discussion with the course responsible areused to evaluate and discuss the effect of the implemented interventions.

Overall description of the course

In the course Veterinary Imaging, students are taught how veterinary medi-cal images are acquired and interpreted. Such medical images are typicallyused to assess whether disease is present in different anatomical regions.The course covers radiology, ultrasound, computed tomography, magneticresonance imaging, scintigraphy and radiation safety. It runs over 5 weeksduring the Master of Veterinary Medicine. The first week consists of half-day lectures. One of these lectures covers the basic physical principles ofultrasound. The following 4 weeks contain a few days of theoretical groupteaching and then have a more practical approach with mandatory groupexercises, case-exercises and e-learning. Finally, students take a practical

1 http://kurser.ku.dk/course/SVEK13043U, last visited 04.12.2018

276 Anna Müller

test in radiography and a test covering theory from the whole course. Thesetests must be passed in order to receive a course certificate.

Ultrasound exercises

Part of the mandatory practical exercises in this course covers ultrasoundscanning of the abdomen. Ultrasound scanning is a skill that requires a lotof individual practice. Approximately 12 students at a time have two ultra-sound machines at their disposal for two full days. They are given a 30-minute introduction to these machines by a teacher on the first day. Afterthe introduction, the students are expected to practice scanning on healthydogs. In several previous evaluations of the course, students describes deepfrustration about these exercises, for reasons described below. It is frustrat-ing for the teacher to leave the students with these frustrations. Our sectiondoes not have resources to let a teacher stand by the students one-on-onefor those two days. Although we try to help the students in between othertasks such as clinical work, the given time varies from day to day, whichcauses further frustrations when students compare their exercises with oneanother.

Definition of the problem

The introduction above raises the question: How can the course be adapted,so that students become better prepared for two days of independent groupwork in ultrasound?

Resume of selected literature

Ultrasound is an important diagnostic tool for many veterinarians in prac-tice, and is a skill that many veterinary students rate as important (Röschet al., 2014). For an unexperienced operator, it can be hard to contextualizethe resulting cross sectional images. Hands-on practice in the safe environ-ment, such as clinical skills labs, can motivate veterinary students, reducetheir anxiety in clinical situations later and reduce gaps between theory andpractice (Rösch et al., 2014; Langebæk et al., 2012). Our practical ultra-sound exercises have one thing in common with such skills labs: they in-volve independent training on healthy animals, as opposed to real patients.However, using live animals for practice can be considered controversial,

22 The learning of veterinary students... 277

and a different way of practicing ultrasound scanning is to use phantoms.Phantoms allows principles to be demonstrated, knowledge to be gainedand invasive techniques such as biopsies can be practiced safely (Carriget al., 2001; Yoo et al., 2004). The feeling of making progress has beenshown to positively affect the emotions, motivation and perceptions thatpeople experience in relation to their work - as long as the work is consi-dered meaningful to the person. Even small progresses can have an impact.On the other hand, setbacks have a negative impact, which is even strongerthan that of progression. The feeling of progress is usually followed by abetter performance (Amabile et al., 2011). If this principle is translated toteaching of veterinary students, it seems important to set aside time andattention for noticing and collecting evidence of progress among the stu-dents. For this reason, focus on expectations and progress were addressedin the interventions presented below. It is has been shown that verbal andnon-verbal information have equal importance for human cognition (Clarket al., 1991). Therefore, in teaching situations, it is beneficial to use a mixof speech, discussions and non-verbal information such as images, videosand variation in body language (Ulriksen, 2016).

Description of documentation

For the purpose of this report, I read the qualitative and quantitative resultsof the course evaluations as of autumn 2017 and spring 2018. These eval-uations were anonymized with regard to personal comments on teachers. Ialso interviewed associate professor Lene Buelund, who is course responsi-ble, as well as a student who took the course before the interventions wereintroduced.

Later, the introduced interventions were assessed based on personal ex-perience, a second interview with course responsible and written interviewswith two students who took the course after the interventions had been im-plemented. The focus of these interviews were the students’ opinions abouttheir learning on the described course. Finally, the 1-minute evaluationswritten on the ultrasound checklists (introduced below) were also includedin the assessment. In total, 10 checklists had been handed in.

278 Anna Müller

Results

The course evaluations showed that students in general were positive aboutthe Veterinary Imaging Course. However, as given in the introduction, therewere many frustrations related to teaching and learning ultrasound. Themain causes of frustration were:

i. A general wish to learn more ultrasound than the course offers.ii. A need for theoretical preparation, such as a lecture introducing ultra-

sound images, machine settings etc. before practical exercises.iii. A need for more guidance when moving from theory to practice - there

is a big difference in assessing images acquired by a skilled ultrasono-grapher versus producing and interpreting images on your own.

iv. Difficulties setting the ultrasound machine correctly.v. Lack of structure during the practical exercises. Many students describe

good teamwork, but issues using time efficiently. They have troublesplanning their own time, such as splitting up in teams and mix theoryand practice. Many students leave early.

vi. Questions arise during scanning, but there is no teacher available to askfor help.

vii. Observing teachers being busy with clinical work and writing reportsmakes some students feel less important. For example, one studentwrote “It seemed like there was very little time to explain things to us,because patients arrived. Then we were left on our own to just watchand try things on the dog we had brought”.

The student who took the course the previous year also described frus-tration with the ultrasound exercises. Although she had tried to preparewell, she was in doubt about how to prepare and what to read. This stu-dent expressed an ambitious wish to learn ultrasound, but theory was over-whelming to her, and the gap between theory and practice felt huge. Asa possible improvement she suggested that some kind of practical guidemight help reduce this gap. Dr. Buelund confirmed many of the issues men-tioned above. She said: “There is a need for a theoretical introduction to ul-trasound in the course and development of teaching material”. Dr. Buelundhad observed that many students came poorly prepared, and that studentshad difficulties working independently. “However”, she said, “our studentsdo learn a lot during the exercises, but unfortunately, they do not seem tobe aware of their own progress”.


To summarize, the student evaluations, the qualitative interviews andmy own experience showed a strong agreement: There was a lack of theo-retical introduction to ultrasound and difficult for students to perform inde-pendent practice. The students needed help to become better prepared, andmore guidance to get going with the practical exercises. Equally important,there was a need to adjust the level of expectations before our students ar-rive to the exercises. Lastly the students need to be able to recognize theirown progression along the course.

Description of interventions

A number of interventions were introduced in the course. The overall aimof these interventions was to cover the need for a theoretical introductionand to guide students when they move from theory to practice. Other aimswere to help students prioritize their time during the exercises, monitor theirown progress, and to encourage students to take responsibility for their ownlearning. Hopefully that can reduce the idea that they must have teacher toguide them.

Lecture covering ultrasound

The aim was to introduce ultrasound theoretically and prepare students forthe practical exercises. The lecture covers 6 hours of teaching radiology andultrasound of the abdomen. There are approximately 25 students at a timeand the lecture is repeated eight times a year. The lecture includes learn-ing goals, terminology and machine settings, followed by a section withcase based teaching including a large number of normal and abnormal ra-diographs and ultrasound images. It also allows presentations of ultrasoundvideos mimicking real-time ultrasound scans. Moreover, the practical exer-cises are introduced and students are encouraged to prepare by reading thepresented teaching material before and during the exercises. To stimulatelearning, the lecture is organized with an overlap of constructivist and psy-chodynamic approaches (Dolin, 2013; Bada et al., 2015). To foster studentactivation and participation, a large number of case-based discussions areincluded during the day (Clark et al., 1991; Dolin, 2013; Wang et al., 2013).These cases encourage dialogue among students as well as between teacherand students.

280 Anna Müller

Ultrasound teaching material

Lecture handouts: Most of the slides from the lecture are handed out inadvance, so that machine settings, terminology and many examples of nor-mal and abnormal ultrasound images are available to the students. How-ever, most slides with cases does not show written image descriptions, topromote student discussions during class.Compendium: A guide of 23 pages is handed out to the students beforethe lecture and practical exercises. This guide introduces the relevance ofclinical ultrasound, describes machine settings and transducer manipula-tions. Relevant organs are described and normal ultrasound appearance ofthese organs are illustrated. This level of detail is in agreement with ourinterpretation of the expected day 1 competences for a Danish veterinar-ian according to the European Association of Establishment for VeterinaryEducation (EAEVE) (ECCVT, 2015).Checklist: A checklist (Appendix A) with different tasks is handed outduring the introduction to the practical ultrasound exercises. The purposeof this list is to clearly state what we expect from the students during theexercises, and to guide them through relevant teaching material, such asthe compendium mentioned above. The list is also meant to help studentsprioritize their time, and to increase their feeling of being responsible fortheir own learning. Furthermore, the list is a way of monitoring progressat a personal level. Finally, in the checklist, students are asked to reflecton their own effort during exercises, and there is a 1-minute evaluation forstudents to fill in. Then the checklist is asked to be handed in to anyone inour staff.Quick guide: A pictogram instruction (Appendix B) on how to prepare thepatient, instructions on how to select a transducer, find relevant buttons onthe ultrasound machine and show how to set these buttons.

Analysis and discussion of the raised question

During the time span of this project, I held the lecture three times and sixgroups of students had the practical ultrasound exercises. The checklist wasonly available to one of these groups, so the value of the checklist remainsto be assessed. My personal experience was that the students participatedactively in the case discussions and seemed engaged during the lecture andexercises. They also seemed to appreciate the new teaching material.


“It is now clear what our students can expect from the ultrasound teach-ing, what we as teachers expect from the students and what they achieveduring the practical exercises”, Dr. Buelund said. To improve teaching fur-ther, Dr. Buelund mentions that we need to explore how additional ultra-sound teaching can be added. “But that is a complex matter, because itrequires coordination with other courses that run parallel with ours”, shesays.

The two interviews showed that both students had markedly improvedtheir knowledge and skills in the field of ultrasound. They both found theavailable teaching material useful, and both said that their own effort in thecourse had contributed a lot to their learning. Nevertheless, both studentswished they had had a teacher available at least some of the time. Thiswas also a wish among most of the checklist evaluations. One interviewstudent said that although she was well prepared and had read all of theavailable material prior to class, she found it hard to move from theory topractice. Both students expressed that further training is needed for themto use ultrasound as a diagnostic tool. This is in line with our expectationsas teachers – but obviously not the expectations of the students. Thus forfuture teaching, it seems relevant to highlight the level of expectations, forexample by discussing expectations in the beginning of the group teaching(Horst et al., 2013). It has become clear that our students seem to expect togain way better skills than the teachers expect them to have after only twodays of training. As a comparison, many students accept that they cannotperform advanced surgery on their first day as a vet. However, many seemto expect that they can perform a diagnostic abdominal scan on day one.

From a practical point of view, the implementation of the described in-terventions was fairly straight forward. One challenge occurred when onegroup of students were re-scheduled, so that they had the ultrasound exer-cises before the ultrasound lecture. That was frustrating for them and forme as a teacher. But our discussions in class later showed that they hadgained basic knowledge about ultrasound. Furthermore, the teaching ma-terial described in this report needs to be familiar to all of my colleaguesin the section, because we may all engage with the students during theirultrasound exercises.

I was glad to experience that some students actually prepared for thelecture and the exercises. It is not my general impression that students comewell prepared. My colleagues and I have even considered if it would be bet-ter to only teach ultrasound in theory, because unfortunately the frustrationsthat grow from the practical exercises often obscure the realization that stu-

282 Anna Müller

dents actually do learn much. If we simply shortened the time availablefor scanning, I think frustrations would decrease. But so would the scan-ning skills of the students and that would contradict the EAEVE Day onecompetencies (ECCVT, 2015).

The independent format means that students gain ultrasound skills moreor less on their own during the ultrasound exercises. We can see that ourstudents gain a basic knowledge about ultrasound, and that their scanningskills are markedly improved after two days of practice and active learning.Despite this, the students are disappointed and want more guidance. Thiscontradiction might partly express that they are not used to constructiveteaching and independent work (Bada et al., 2015).

This report covers a limited time span for implementation of a re-organized lecture and new teaching material. But there are of course manypossibilities for improvement in the future. A few alternative examples areFlipped Classroom, where lectures are given online in advance, and con-frontation time is spent on problem solving activities (McLaughlin et al.,2014), or a mix of clinical demonstrations and online resources (Roshier etal., 2011).

This project shows that there is a continuous need to set the level ofexpectation among students before practical ultrasound exercises. The newlecture format and the new teaching material are valuable and useful, andcan be improved continuously. But it has also become clear that some issuescannot be solved by simply improving the quality of teaching. The workload of the teachers as well as the students, is already high. More manpowerwould be needed to satisfy the students wish for one-on-one teaching. Inthis case, although the level of learning might be acceptable already, I thinkstudents are right: the two days of ultrasound exercises would be even betterif a teacher was present to a greater extent.

Perspectivation

There is a clear interest for the topic of ultrasound among veterinary stu-dents. This can be considered a luxury in an academic education with anextensive curriculum (Hashizume et al., 2016). There might be a great po-tential in this interest if it can be allowed to flourish and lead to improvedlearning of ultrasound. It would be ideal to create a diagnostic imagingskills lab where phantoms and interactive video material are used as part ofthe ultrasound teaching. Other than reducing the time where live animals


must participate, practice in skills labs can improve skills and reduce anx-iety among students prior to practice on live animals (Rösch et al., 2014;Langebæk et al., 2012; Carrig et al., 2001). For this to happen, however,more staff resources and funding would be required. Furthermore, the useof ultrasound already during practical anatomy lessons can be beneficial forlearning, and might prime students for experience with ultrasound later on(Swamy et al., 2012).

Acknowledgements

I would like to thank our students for participating in discussions and in-terviews, my colleagues in Veterinary Diagnostic Imaging and Tina MøllerSørensen for relevant input with regard to this project and report.

284 Anna Müller

References

Rösch, T., Schaper, E., Tipold, A., Fischer, M. R., Dilly, M., & Ehlers, J. P.(2014). Clinical skills of veterinary students–a cross-sectional study of theself-concept and exposure to skills training in Hannover, Germany. BMCveterinary research, 10(1), 969.

Langebæk, R., Eika, B., Jensen, A. L., Tanggaard, L., Toft, N., & Berendt,M. (2012). Anxiety in veterinary surgical students: a quantitative study.Journal of veterinary medical education, 39(4), 331-340.

Carrig, C. B., & Pyle, R. L. (2001). Anatomic models and phantoms for di-agnostic ultrasound instruction. Veterinary Radiology & Ultrasound, 42(4),320-328.

Yoo, M. C., Villegas, L., & Jones, D. B. (2004). Basic ultrasound curricu-lum for medical students: validation of content and phantom. Journal ofLaparoendoscopic & Advanced Surgical Techniques, 14(6), 374-379.

Amabile, T. M., & Kramer, S. J. (2011). The power of small wins. HarvardBusiness Review, 89(5), 70-80.

Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Edu-cational psychology review, 3(3), 149-210.

Ulriksen, L. (2016). God undervisning på de videregående uddannelser: enforskningsbaseret brugsbog. Nota.

Dolin, J. (2013). Undervisning for læring, in Universitetspædagogikum, L.Reinecker, et al., Editors. Samfundsliteratur: Frederiksberg C. p. 65-91.

Bada, S. O., & Olusegun, S. (2015). Constructivism learning theory: Aparadigm for teaching and learning. Journal of Research & Method in Ed-ucation, 5(6), 66-70.

Wang, H., Rush, B. R., Wilkerson, M., Herman, C., Miesner, M., Renter,D., & Gehring, R. (2013). From theory to practice: integrating instructional


technology into veterinary medical education. Journal of veterinary medi-cal education, 40(3), 264-271.

ECCVT. (2015). European Coordination Committee for Veterinary Train-ing (ECCVT) Day One Competences.

Horst, S., et al. (2013). Evaluering af undervisning, in Universitetspæda-gogik, L. Reinecker, et al., Editors. Samfundslitteratur: Frederiksberg C. p.409-422.

McLaughlin, J. E., Roth, M. T., Glatt, D. M., Gharkholonarehe, N., David-son, C. A., Griffin, L. M., ... & Mumper, R. J. (2014). The flipped class-room: a course redesign to foster learning and engagement in a health pro-fessions school. Academic medicine, 89(2), 236-243.

Roshier, A. L., Foster, N., & Jones, M. A. (2011). Veterinary students’ us-age and perception of video teaching resources. BMC medical education,11(1), 1.

Hashizume, C. T., Myhre, D. L., Hecker, K. G., Bailey, J. V., & Lockyer, J.M. (2016). Exploring the teaching motivations, satisfaction, and challengesof veterinary preceptors: a qualitative study. Journal of veterinary medicaleducation, 43(1), 95-103.

Swamy, M., & Searle, R. F. (2012). Anatomy teaching with portable ultra-sound to medical students. BMC medical education, 12(1), 99.

286 Anna Müller

A Ultrasound exercise checklist

10

Appendix 1: Ultrasound exercise checklist


B Quick guide to practical ultrasound exercises

11

Appendix 2: Quick guide to practical ultrasound exercises

288 Anna Müller

Improving University Science Teaching and Learning

Documents