An#integrated#approach#to#evaluation:## A#participatory# ...eprints.qut.edu.au/97728/3/Ellise Barkley Thesis.pdfAn#integrated#approach#to#evaluation:##...

An#integrated#approach#to#evaluation:##A#participatory#model#for#reflection,#evaluation,#analysis#and#

documentation#(the#‘READ’#model)#in#Community#Arts#

!

!

!

!

!

!

!

!

!

!

!

Ell ise!Jane*Ann!Barkley!

Creative!Industries,!Queensland!University!of!Technology!

!

Submitted!in!fulfilment!of!the!requirements!of!the!degree!of!!

Doctor!of!Creative!Industries!

!

July!2016!

!

!

–!!This!dissertation!is!accompanied!by!the!READ%in%Practice%portfolio!!–%

!

AN!INTEGRATED!APPROACH!TO!EVALUATION! ELLISE!BARKLEY!

! ii!

#

#

#

#

#

#

#

#

#

#

#

©!2016!–!Ellise!Barkley.!

Cover!image:!Designed!by!Phoebe!Barton!

!

#

! #


! iii!

Abstract#

Evaluation!is!a!routine!requirement!for!external!funding!in!community!arts,!and!practitioners!face!mounting!

pressure!to!demonstrate!the!value!of!their!work!in!competitive!environments.!Evaluating!the!art*making!

processes,!outcomes!and!impact!of!community!arts!requires!practitioners!to!traverse!myriad!philosophical!and!

technical!challenges.!Despite!the!significant!theoretical!developments!over!the!past!30!years,!effective!

evaluation!practice!remains!elusive,!unresolved!and!highly!contested.!At!one!extreme,!opponents!argue!

evaluation!in!its!current!usage,!oriented!towards!the!quantification!of!arts!impact,!is!detrimental!for!the!field.!!

This!research!focuses!on!the!development,!operational!use,!and!critical!review!of!the!READ!model,!a!dynamic!

and!rigorous!model!for!evaluation!of!community!arts.!The!innovation!of!the!model!lies!in!the!integration!of!

four!key!appraisal!and!learning!strategies*!Reflection,!Evaluation,!Analysis!and!Documentation!(READ)!*!to!

cater!for!the!multi*faceted!demands!of!complex!creative!partnership!initiatives.!The!READ!concept!was!

collaboratively!devised!at!the!site!of!my!professional!practice,!and!my!role!as!researcher!has!involved!

transforming!the!READ!concept!into!a!substantiated!model!of!practice!and!evaluation!product,!by!engaging!

with!partners!and!participants,!leading!the!design!and!delivery!process,!and!then!evaluating!the!efficacy!of!the!

developed!model!in!action.!The!body!of!evidence!represents!the!first!iteration!of!critically!analysing!the!READ!

model’s!development!and!testing!during!the!initial!phase!of!the!Nuclear!Futures!Partnership!Initiative!(2013!*!

2015),!an!international!arts!and!cultural!program!funded!by!the!Australia!Council!for!the!Arts!(2014!*!2016).!!

The!research!is!embedded!in,!and!furthers,!my!own!professional!evaluation!practice,!advancing!my!expertise!

in!the!design!and!delivery!of!reflexive!learning!and!appraisal!systems!suited!to!the!creative!arts!context.!The!

findings!establish!an!important!foundation!for!the!further!development!of!innovative!evaluation!practices!

across!the!Nuclear!Futures!program,!including!the!identification!of!the!newly!developed!model’s!strengths!and!

constraints!critical!for!the!program’s!summative!evaluation!in!2017.!At!the!site!of!practice,!the!successful!READ!

approach!created!impact!through!an!enhanced!understanding!of!the!artistic,!theoretical,!academic!and!wider!

social!implications!of!the!community!arts!practice,!and!promoted!engagement,!acceptance,!and!participation!

in!evaluation!by!mixed!stakeholder!groups.!!

In!addressing!identified!knowledge!gaps!on!effective!community!arts!evaluation,!the!research!engages!with!

the!complexities!of!evaluation!of!creative!partnership!initiatives,!delivering!an!original!and!practical!

contribution!that!will!have!significant!impact!within!the!field.!!For!the!field,!the!READ!model!offers:!

•! an!integrated!evaluation!method!that!can!be!conducted!‘in!house’;!

•! a!model!of!evaluation!best!practice!that!embraces!the!ethos!of!community!arts;!

•! a!flexible!framework!for!generating!robust!data,!that!is!linked!with!management!and!creative!arts!

practice!so!that!data!can!be!fed!back,!continuously,!into!the!project!for!real*time!impact;!

•! original!strategies!for!engagement!and!for!developing!stakeholder!ownership!of!project!evaluation,!

particularly!where!trepidation!or!resistance!to!traditional!evaluation!means!exists;!

•! possibilities!for!more!general!application!in!(research*led)!creative!management.!

! !


! iv!

! !

Blank!page!


! v!

Table#of#contents#

Abstract!...................................................................................................................................!iii!

Table!of!contents!......................................................................................................................!v!

List!of!figures!............................................................................................................................!ix!

List!of!tables!.............................................................................................................................!x!

List!of!abbreviations!.................................................................................................................!xi!

Acknowledgements!................................................................................................................!xiii!

1! Introduction+...............................................................................................................+1!

1.1! Research!context!....................................................................................................................!3!

1.2! Background!.............................................................................................................................!3!

1.3! Problem!statement!.................................................................................................................!6!

1.4! Aims!and!objectives!................................................................................................................!8!

1.5! READ!product!development!...................................................................................................!9!

1.6! The!scope!and!significance!of!the!research!..........................................................................!11!

1.7! The!DCI!journey!....................................................................................................................!12!

1.7.1! Research!design!overview!.............................................................................................!12!

1.7.2! Research!phases!............................................................................................................!13!

1.7.3! Examinable!DCI!outputs!................................................................................................!14!

1.8! Chapter!overview!.................................................................................................................!14!

1.8.1! Dissertation!roadmap!...................................................................................................!17!

2! Contextual+scan+........................................................................................................+19!

2.1! Community!arts!....................................................................................................................!20!

2.1.1! Community!arts!and!cultural!development!(CACD)!.....................................................!20!

2.1.2! The!rise!of!community!partnerships!............................................................................!21!

2.2! Evaluation!.............................................................................................................................!23!

2.2.1! Program!evaluation!......................................................................................................!24!

2.2.2! Participation*oriented!approaches!to!program!evaluation!..........................................!26!

2.3! Community!arts!evaluation!..................................................................................................!28!

2.3.1! Trends!in!community!arts!evaluation!...........................................................................!29!

2.3.2! Relevant!approaches!and!models!for!the!Nuclear!Futures!site!of!practice!.................!32!

2.4! Conclusion!............................................................................................................................!37!

!


! vi!

!

3! Research+design+........................................................................................................+39!

3.1! Qualitative!research!methodology!.......................................................................................!40!

3.1.1! Practice*informed!research!paradigm!..........................................................................!41!

3.2! Research!strategy:!forensic!reflective!practice!....................................................................!42!

3.2.1! Theoretical!underpinnings!of!reflective!practice!..........................................................!42!

3.2.2! Methodological!considerations!for!forensic!reflective!practice!...................................!44!

3.3! Data!collection!and!analysis!methods!..................................................................................!45!

3.3.1! Reflective!methods!.......................................................................................................!47!

3.3.2! Interviews!and!consultation!.........................................................................................!52!

3.3.3! Observation!..................................................................................................................!54!

3.3.4! Evaluation!theory!and!READ!product!development!.....................................................!54!

3.4! Research!phases!...................................................................................................................!57!

3.5! Ethical!considerations!..........................................................................................................!57!

3.6! Conclusion!............................................................................................................................!58!

4! Site+of+practice+.........................................................................................................+59!

4.1! Nuclear!Futures!Partnership!Initiative!(Nuclear!Futures)!.....................................................!60!

4.2! Balaklava!Community!Arts!Project!.......................................................................................!64!

4.2.1! Origins!and!inspiration!.................................................................................................!64!

4.2.2! Project!partners!and!stakeholders!...............................................................................!66!

4.2.3! Project!overview!...........................................................................................................!67!

4.2.4! Summary!of!the!Balaklava!community!arts!practice!....................................................!69!

4.3! Conclusion!............................................................................................................................!78!

! !


! vii!

!

5! The+READ+application,+Balaklava+(2014H15)+...............................................................+79!

A!guide!to!the!READ!application!.............................................................................................................!80!

Part+A:+Envisaging+the+READ+model+and+preliminary+activities+.........................................+81!

5.1! Background!...........................................................................................................................!81!

5.1.1! The!READ!model!...........................................................................................................!81!

5.1.2! A!systems!approach!......................................................................................................!85!

5.1.3! Research!phases!...........................................................................................................!87!

5.2! READ!approach!and!assumptions!.........................................................................................!89!

5.2.1! Context!and!parameters!...............................................................................................!89!

5.2.2! Internal!position!for!READ!facilitation!..........................................................................!90!

5.2.3! Interpretative!stance!....................................................................................................!90!

5.2.4! Participatory*oriented!approach!..................................................................................!91!

5.3! Scoping!.................................................................................................................................!92!

5.3.1! Stakeholders!identification!and!consultation!...............................................................!92!

5.3.2! Methods!and!analysis!...................................................................................................!93!

5.3.3! Results!for!the!scoping!phase!.......................................................................................!95!

Part+B:+READ+methodology+in+action+................................................................................+98!

5.4! Design!and!implementation!.................................................................................................!98!

5.4.1! Support!structures!........................................................................................................!99!

5.4.2! Reflection!...................................................................................................................!101!

5.4.3! Evaluation!...................................................................................................................!108!

5.4.4! Analysis!.......................................................................................................................!116!

5.4.5! Documentation!...........................................................................................................!120!

5.5! READ!outputs,!sharing!and!use!..........................................................................................!125!

5.6! Conclusion!..........................................................................................................................!131!

! !


! viii!

!

6! Findings+..................................................................................................................+133!

6.1.1! Analysis!of!the!READ!application!................................................................................!134!

6.1.2! Questions!arising!from!forensic!reflective!practice!....................................................!135!

6.2! Findings!and!discussion!......................................................................................................!137!

6.2.1! Modular!and!integrated!approach!.............................................................................!137!

6.2.2! Social!constructivist!stance!........................................................................................!142!

6.2.3! Internal!facilitation!.....................................................................................................!146!

6.2.4! Emergent!evaluation!practice!....................................................................................!149!

6.2.5! Meta*evaluation!.........................................................................................................!153!

6.2.6! Opportunities!for!enriching!evaluation!practice!........................................................!164!

6.3! The!final!READ!product!......................................................................................................!169!

6.3.1! Product!development!.................................................................................................!169!

6.3.2! Profile!for!the!READ!product!......................................................................................!171!

6.3.3! Infographic!.................................................................................................................!172!

6.3.4! Assumptions,!constraints!and!variables!.....................................................................!175!

6.4! Chapter!conclusion!.............................................................................................................!178!

7! Conclusion+..............................................................................................................+181!

7.1! The!DCI!research!................................................................................................................!182!

7.2! Summative!conclusions!......................................................................................................!186!

7.3! Implications!and!significance!of!the!research!....................................................................!193!

7.4! Dissemination!.....................................................................................................................!196!

7.5! Future!research!..................................................................................................................!197!

7.6! Dissertation!conclusion!......................................................................................................!199!

Appendix+1:!Interview!design!for!scoping!interviews…………………………………………………….…………….203!

Bibliography+..................................................................................................................+205!

!

Supplementary!material!

Document+2:+READ%in%Practice+portfolio+(RIPP)+

Companion!document!to!the!dissertation!(164!pages)!–!See!attached!document!

! !


! ix!

List#of#figures##

Figure!1.1:!The!Doctor!of!Creative!Industries!research!design!............................................................!12!

Figure!1.2:!The!six!research!phases!for!the!DCI!research!.....................................................................!13!

Figure!1.3:!Roadmap!for!dissertation!...................................................................................................!18!

Figure!3.1:!The!Doctor!of!Creative!Industries!research!design!............................................................!41!

Figure!3.2:!Model!for!forensic!reflective!practice!................................................................................!43!

Figure!3.3:!Hierarchy!of!theory!development!via!forensic!reflective!practice!.....................................!55!

Figure!4.1:!Map!of!South!Australia!and!relevant!place*names!............................................................!61!

Figure!4.2:!Map!of!Balaklava!and!Wakefield!Regional!Council!area,!South!Australia!..........................!64!

Figure!5.1:!The!READ!model!as!system!................................................................................................!86!

Figure!5.2!:!The!research!phases!for!the!DCI!........................................................................................!88!

Figure!5.3:!Key!elements!of!Community!Arts!project!to!be!reviewed!.................................................!90!

Figure!5.4:!Mapping!Reflection!within!READ!for!the!Balaklava!Community!Arts!Project!..................!107!

Figure!5.5:!!READ!timeline!of!activities!and!outputs!for!Balaklava!Community!Arts!Project!.............!129!

Figure!6.1:!Extracts!from!reflective!practice!......................................................................................!147!

Figure!6.2:!Evaluation!theory!tree!by!Christie!and!Alkin!(version!3)!..................................................!150!

Figure!6.3:!Structured!design!and!development!for!the!READ!product!.............................................!170!

Figure!6.4:!The!READ!model!infographic!............................................................................................!173!

!

!

Box!6.1:!Reflective!questions!arising!from!the!forensic!reflective!practice!sessions!for!the!DCI!.......!136!

!

! #


! x!

List#of#tables#

Table!3.1:!Qualitative!methods!employed!for!the!DCI!research!design!(2013*15)!..............................!46!

Table!4.1:!Key!activity!strands!for!the!Nuclear!Futures!program!(2014*16)!........................................!63!

Table!6.1:!Summary!of!READ!model!processes!and!outputs!.............................................................!152!

! #


! xi!

List#of#abbreviations#

AES! Australasian!Evaluation!Society!

ANVA! Australian!Nuclear!Veterans!Association!

CACD! Community!arts!and!cultural!development!

CCD! Community!cultural!development!

CCPI! Creative!Communities!Partnership!Initiative!

KEC! Key!evaluation!checklist!

KEQ! Key!evaluation!questions!

Nuclear!Futures! Nuclear!Futures!Partnership!Initiative!

READ! The!READ!model:!Reflection,!Evaluation,!Analysis,!

Documentation!

RIPP! READ%in%Practice!portfolio!(companion!document) !

SA! South!Australia!

SLSA! State!Library!of!South!Australia!

SWOT! Strengths,!weaknesses,!opportunities,!threats!(situation!

analysis!tool)!

QUT! Queensland!University!of!Technology!

!

!

!

! #

Ellise Barkley 19th July, 2016


xii!

The!work!contained!in!this!dissertation!and!accompanying!portfolio!has!not!been!previously!

submitted!to!meet!requirements!for!an!award!at!this!or!any!other!higher!education!

institution.!To!the!best!of!my!knowledge!and!belief,!the!dissertation!contains!no!material!

previously!published!or!written!by!another!person!except!where!due!reference!is!made.!

QUT Verified Signature


! xiii!

Acknowledgements#

I!extend!my!appreciation!to!my!supervisory!team:!Professor!Helen!Klaebe!and!Associate!Professor!Sandra!

Gattenhof!for!the!quality!input,!direction!and!expertise!contributed,!as!well!as!their!sunny!encouragement.!I!

thank!Dr!Jess!Dart!for!the!wisdom!and!depth!of!knowledge!she!brought!to!the!research!as!industry!mentor,!

and!her!generosity!in!sharing!her!practitioner!insights.!Thanks!also!to!Adjunct!Professor!Cheryl!Stock!and!Dr!

Lee!McGowan!for!their!leadership!through!the!DCI!process!and!igniting!my!interest!in!reflective!practice.!

Substantial!input!for!this!research!has!been!offered!at!my!site!of!practice,!and!I!am!indebted!to!all!those!who!

have!contributed!to!the!investigation!(directly!and!inadvertently).!First,!my!colleague!Paul!Brown!(Creative!

Producer,!Alphaville)!has!made!significant!intellectual!contribution!and!provided!constructive!commentary!to!

advance!the!research.!There!is!always!something!to!learn!from!Paul!Brown!–!thank!you.!I!am!appreciative!for!

the!ideas,!effort!and!cooperation!of!the!Nuclear!Futures!team!and!partners,!who!worked!on!the!collective!

design!and!application!of!the!READ!model.!In!particular!community!members!Avon!and!Kate!Hudson,!and!

creative!team!members!Jessie!Boylan,!Carly!Friedrichs,!Linda!Dement,!Teresa!Crea!and!John!Romeril!–!what!

delightful!folk!to!work!with.!I!thank!State!Library!of!South!Australia!staff!for!helping!set!the!compass!for!the!

community!archiving!in!the!research,!and!am!grateful!to!the!hundreds!of!participants!who!have!engaged!with!

the!READ!process!through!the!art*making!and!evaluation!practice.!I!also!thank!the!Alphaville!arts!organisation!

for!their!cooperation!during!the!research!and!for!making!Nuclear!Futures!program!content!available.!

Behind!the!scenes,!the!research!has!been!made!possible!by!the!love,!patience!and!humour!of!my!family!and!

friends.!In!particular:!the!unwavering!generosity!of!Peter!Graham!(words!cannot!express),!the!optimism!and!

encouragement!of!daughters!Lily!and!Sophie,!the!thoughtful!input!of!my!mum,!Jan,!and!the!caring!support!of!

Karen!and!Ian.!I!would!like!to!thank!Jaymie!Maley!for!her!graphic!design!services!and!input!during!the!

development!of!the!READ!infographic,!Mel!Rumble!for!copyediting!and!proofreading!Chapters!1,!6!and!7!(in!

accordance!with!the!university’s!editing!guidelines),!and!Matt!Bailey!for!technical!support!with!the!electronic!

compilation!of!the!READ%in%Practice!portfolio!(companion!document!to!this!dissertation);!and!for!the!

energising!friendship!of!all!three!during!these!processes.!

! !


! xiv!

!

Blank!page!


! 1!

1! Introduction#

This!Doctor!of!Creative!Industries!(DCI)!research!was!conceived!through!a!practitioner’s!lens,!devised!

to!address!the!evaluation!needs!at!the!site!of!my!professional!practice!in!community!arts.!The!field!

of!community!arts!encompasses!the!wide!range!of!arts*based!collaborative!practices!run!by,!or!with,!

communities!in!partnership!with!professional!artists!and!arts!workers!(Australia!Council!for!the!Arts,!

2011).!Over!the!past!30!years,!despite!increasing!requirements!and!developments!in!evaluation!in!

this!context,!no!agreed!professional!framework!for!evaluating!community!arts!exists,!and!the!criteria!

for!effective!and!appropriate!evaluation!practice!in!community!arts!remains!highly!contested.1!!

The!focus!of!this!research!is!the!development,!operational!use,!and!critical!review!of!the!READ!

model,!a!dynamic!and!rigorous!model!for!evaluation!of!community!arts.!The!innovation!of!the!model!

lies!in!the!integration!of!four!key!appraisal!and!learning!strategies!*!Reflection,!Evaluation,!Analysis!

and!Documentation!(READ)!*!to!cater!for!the!multi*faceted!demands!of!complex!creative!community!

partnerships.!Prior!to!this!research,!‘READ’!existed!in!concept!form!only,!arising!from!professional!

practice!and!coined!in!2012!by!Sydney*based!arts!organisation!Alphaville!to!meet!the!needs!of!the!

Nuclear!Futures!Partnership!Initiative!(Nuclear!Futures).!I!was!a!key!contributor!in!this!process,!

which!resulted!in!a!one*page!outline!and!statement!of!intent!for!the!to*be*developed!READ!model!

(see!Section!1.5).!The!DCI!offered!a!means!to!advance!the!READ!concept,!create!a!model!of!practice,!

and!test!the!READ!methodology!developed!in!an!industry!setting!to!study!its!efficacy!and!potential.!!

The!research!is!embedded!in,!and!furthers,!my!own!professional!evaluation!practice,!advancing!my!

expertise!in!the!design!and!delivery!of!reflexive!learning!and!appraisal!systems!suited!to!the!creative!

arts!context.!It!also!establishes!seminal!protocols!for!evaluation!practice!at!my!site!of!practice.!My!

role!as!researcher!involved:!engaging!with!project!partners!and!participants!at!my!site!of!practice;!

facilitating!stakeholder!input!and!participation;!leading!the!methodological!design!and!delivery!of!

the!READ!model;!trialling!READ!methods;!consolidating,!articulating!and!analysing!the!model;!and!

creating!communication!materials!such!as!the!READ!infographic!and!partner!resources.2!Through!the!

DCI!I!have!also!designed!a!practitioner*friendly!READ!product!for!future!practice!(see!Section!6.3).!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

1!A!standard!definition!for!evaluation!is:!‘the!systematic!collection!and!analysis!of!information!to!make!judgments,!usually!

about!the!effectiveness,!efficiency!and/or!appropriateness!of!an!activity’!(Australasian!Evaluation!Society,!2010,!p.3).!2!In!the!DCI,!I!refer!to!my!role!as!‘READ!facilitator’!to!define!the!active,!leadership!role!I!played!in!administering!the!

collective!READ!application!at!my!site!of!practice.!This!was!not!a!neutral!role,!and!I!was!the!driving!agent!for!coordinating,!

delegating,!and!enabling!the!collective!input,!skills!development,!and!stakeholder!brokerage!necessary!for!the!model’s!

development.!Further,!beyond!the!collective!process,!I!played!a!major!role!in!designing!the!READ!methodology!presented,!

contributing!significant!intellectual!input!into!the!model’s!development,!and!have!independently!generated!the!original!

findings!and!knowledge!on!the!READ!model,!based!on!my!data!collection!and!analysis,!as!presented!in!Chapter!6!and!7.!


! 2!

The!body!of!evidence!presented!in!this!dissertation!represents!the!first!iteration!of!trialling!and!

critically!analysing!the!READ!model’s!development!during!the!initial!phase!of!the!Nuclear!Futures!

initiative!(2013!*!2015),!a!three!year!international!arts!and!cultural!program!funded!by!the!Australia!

Council!for!the!Arts!(2014!*!2016).!The!Nuclear!Futures!program!brings!together!artists!and!atomic!

survivors!to!develop!creative!works!exposing!the!legacies!of!the!atomic!age.!For!the!DCI,!I!studied!

the!first!industry!development!and!application!of!the!READ!concept,!conducted!for!the!Balaklava!

Community!Arts!Project!that!was!delivered!in!South!Australia!as!part!of!Nuclear!Futures.!In!order!to!

study!and!evaluate!the!READ!application!in!the!Nuclear!Futures!setting,!I!employed!a!rigorous!

qualitative!research!methodology!for!the!DCI,!utilising!forensic!reflective!practice!and!methods!such!

as!written!and!video!reflection,!interviews,!digital!storytelling!and!observation!(refer!to!Chapter!3!for!

the!DCI!research!design).!The!DCI!research!design!created!the!critical!lens!to!systematically!review!

my!evaluation!practice,!and!the!READ!application,!to!generate!the!DCI!findings.!

In!addressing!identified!knowledge!gaps!on!effective!community!arts!evaluation,!the!research!

engages!with!the!complexities!of!evaluation!of!creative!partnership!initiatives!and!contributes!to!the!

discourse!within!the!field!on!cultural!value,!offering!an!innovative!stakeholder*oriented!approach!to!

program!evaluation!that!aligns!with!the!ethos!of!community!arts.!The!impact!of!the!research!has!

been!significant!and!transformative,!and!the!READ!model!of!practice!developed!has!extended!and!

enriched!my!own!evaluation!practice,!as!well!as!improved!the!evaluation!practice!at!the!Nuclear!

Futures!site!of!practice.!The!findings!establish!an!important!foundation!for!the!further!development!

of!innovative!evaluation!practices!across!the!three*year!Nuclear!Futures!program,!and!have!

contributed!to!an!enhanced!understanding!of!the!artistic,!theoretical,!academic!and!wider!social!

implications!of!the!Nuclear!Futures!community!arts!practice.!!

In!this!introductory!chapter,!Chapter!1,!I!establish!the!motivation!and!problem!statement!for!the!DCI!

and!state!the!aims!and!objectives!of!the!enquiry.!I!explain!why!the!research!problem!is!significant!for!

my!professional!practice!as!program!manager!and!evaluator,!and!how!the!problem!relates!more!

broadly!as!a!persistent!issue!facing!community!arts!practitioners!and!evaluators.!Chapter!1!

concludes!with!a!synopsis!of!the!dissertation!structure!and!research!progression!(see!Section!1.8.1!

for!the!dissertation!roadmap),!providing!an!overview!of!each!chapter!in!the!dissertation.!

Accompanying!the!dissertation!is!an!important!companion!document,!the!READ%in%Practice!portfolio.!

The!portfolio!includes!examples!from!the!industry*based!evaluation!practice,!and!comprises!

documentation!and!artefacts!demonstrative!of!the!scope!of!the!READ!model!development!and!

application!undertaken!for!the!research.!The!dissertation!and!READ%in%Practice!portfolio!are!

submitted!as!a!set,!and!are!two!interrelated!yet!distinct!research!outputs!resulting!from!the!DCI.!


! 3!

1.1! Research#context##

The!research!presented!in!this!dissertation!has!been!completed!as!part!of!the!Doctor!of!Creative!

Industries!(DCI)!in!the!Creative!Industries!Faculty,!Queensland!University!of!Technology!(QUT).!The!

DCI!is!a!professional!research!doctorate!for!industry!practitioners!that!combines!advanced!critical!

reflection!of!practice!with!rigorous!scholarship.!The!research!is!dually!positioned!in!both!academic!

and!industry!contexts!in!the!field!of!community!arts.!Community!arts!encapsulates!the!diverse!range!

of!arts*based!collaborative!practices!run!by,!or!with,!communities!in!partnership!with!professional!

artists!and!art!workers!(MacDowall,!Mulligan,!Panucci,!&!Badham,!2013).!!

The!research!problem!arises!out!of!professional!community!arts!practice,!conceived!through!an!

industry!lens!from!an!evaluator!and!program!manager!perspective.!The!topic!of!research!represents!

an!area!of!professional!interest!(and!challenge)!associated!with!my!vocation!as!manager!and!

evaluator!over!the!past!15!years!in!community!arts!and!cultural!development,!education,!and!

sustainability!across!local!government,!business!and!community!organisation!settings.!The!research!

problem!is!positioned!within!the!unfolding!contemporary!debate!in!the!field!on!cultural!value,!

framed!by!the!trend!toward!cultural!measurement!and!the!rising!conceptualisations!of!evaluation!as!

a!means!to!articulate,!assess!and!quantify!the!impact!of!community!arts!practice.!

1.2! Background#

The!impetus!for!the!research!began!during!the!pre*program!development!of!the!Nuclear!Futures!

Partnership!Initiative!(Nuclear!Futures)!in!2012,!led!by!Sydney*based!arts!production!company!

Alphaville,!while!in!my!role!as!prospective!Nuclear!Futures!program!manager.3!Nuclear!Futures!is!a!

three*year!international!arts!and!cultural!program!grounded!in!linked!community!arts!and!cultural!

development!projects,!delivered!between!2014*16!(Alphaville,!2015).!The!initiative!is!driven!by!

international!partnerships,!including!the!collaboration!of!artists!working!with!atomic!survivor!

communities!to!bear!witness!to!the!legacies!of!the!atomic!age!through!creative!arts.!Across!the!

program,!communities!and!artists!use!theatre,!film,!paintings,!sculpture,!photography,!digital!arts!

and!other!art!forms!to!make!creative!works!that!reflect!both!the!horror!of!living!with!nuclear!

radiation!and!the!resilience!of!affected!communities.!The!latter!part!of!the!program!involves!the!

internationalisation!of!the!community*based!work,!including!the!touring!and!showcasing!of!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

3!Alphaville!is!an!arts!organisation!specialising!in!community!arts,!cultural!development!and!production,!with!a!focus!on!

social!justice!and!sustainability!issues.!I!was!officially!appointed!manager!in!late!2013,!once!funding!was!secured.!


! 4!

integrated!artworks!in!multiple!countries.!The!image!below!is!one!example!of!the!creative!works.4!!

!

%%

10%Minutes%to%Midnight!immersive!projection!installation,!part!of!the!Nuclear!Futures!!

showcase!presented!for!the!Balaklava!Community!Arts!Project.!Photo!by!Tania!Safi.!

!

In!order!to!understand!the!background!to!the!emergent!READ!evaluation!practice,!it!is!necessary!to!

provide!a!brief!historical!overview!of!the!development!of!the!Nuclear!Futures!program.!

In!2013,!Nuclear!Futures!received!funding!approval!via!the!then!Creative!Communities!Partnership!

Initiative!(CCPI)!scheme!administered!by!the!Australia!Council!for!the!Arts,!and!has!since!secured!

supplementary!funding!via!other!government,!private!and!philanthropic!sources.!Underpinned!by!a!

community!partnerships!logic!(see!Chapter!2),!the!Nuclear!Futures!program!is!characterised!by!

diverse!and!inter*sectoral!partnerships,!a!multi*arts!and!experimental!creative!practice,!and!has!a!

focus!on!community!development!and!mutual!partnership!benefits.!Like!many!externally!funded!

arts!programs,!evaluation!became!a!priority!for!the!program!as!both!a!prerequisite!for!some!of!the!

funding!agencies!approached,!and!as!a!valued!practice!for!review!and!assessment!by!many!of!the!

stakeholders.!At!a!minimum,!the!Australia!Council!stipulated!that!10!per!cent!of!Nuclear!Futures’!

grant!value!be!dedicated!toward!evaluation,!which!represented!a!significant!investment!given!the!

level!of!funding!secured!through!the!CCPI.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

4!Full!program!details!and!creative!works!are!listed!on!Alphaville’s!Nuclear!Futures!website:!http://nuclearfutures.org.!


! 5!

With!over!20!partners!involved,!Alphaville!was!eager!to!ensure!that!resources!dedicated!to!

evaluation!were!directed!towards!learning!and!evaluative!practices!that!could!offer!wide!and!

relevant!benefits!to!all!partners!and!a!range!of!stakeholders.!Early!consultation!with!partners!and!

stakeholders!had!revealed!that!for!evaluation!to!be!effective!in!this!context,!it!would!need!to:!meet!

an!array!of!program!and!partner*specific!evaluation!needs,!in!ethical!and!culturally!appropriate!

ways;!and!be!based!on!rigorous,!reflexive!and!inclusive!methodologies!designed!to!generate!

partner*preferred!outputs!and!learning!exchange.!As!is!detailed!in!Chapter!5,!there!were!both!

general!and!site*specific!evaluation!challenges!arising!from!the!multiple!and!culturally!diverse!set!of!

partners!participating,!each!having!a!stake!in!how!the!program!evaluation!was!conducted!and!how!

the!community!partnerships!and!artworks!were!to!be!represented.!In!anticipating!the!program’s!

divergent!evaluation!needs,!prior!to!securing!the!Australia!Council!funding,!Alphaville!explored!

potential!options!for!solving!the!problem!of!how!to!evaluate!the!Nuclear!Futures!program.!It!was!

hoped!that!if!an!original!and!effective!approach!for!the!complex!CCPI!was!designed!in*house,!it!

could!be!tailored!to!meet!the!needs!at!the!site!of!practice,!as!well!as!establish!protocols!for!the!

future!evaluation!practice!of!Alphaville!and!partners.!

Through!Alphaville’s!process,!I!was!a!key!member!of!a!team!of!creative!practitioners!who!devised!

then!went!on!to!implement!the!READ!concept,!a!participatory!approach!integrating!Reflection,!

Evaluation,!Analysis!and!Documentation!(READ).5!The!READ!concept!had!been!inspired!by!the!

anticipated!needs!of!the!Nuclear!Futures!CCPI!and!represents!a!practice*led!response!to!the!

challenges!arising!from!community!arts!evaluation.!Conceived!was!a!wide!summative!and!integrated!

framework!to!help!creative!practitioners!and!community!partners!reflect!on,!represent!and!appraise!

their!art*based!activities,!partnerships!and!impact.!As!is!elaborated!in!the!dissertation,!the!READ!

approach!is!grounded!in!the!philosophies!and!art*making!traditions!of!community!arts!and!cultural!

development,!and!spans!across!creative!practice,!program!management!and!evaluation.!Prior!to!the!

DCI!research,!the!concept!for!the!READ!framework!based!on!the!four!overarching!READ!strategies!

had!been!identified,!however!had!not!been!developed!beyond!a!basic!level!(see!Section!1.5).!No!

READ!methodology!or!model!of!practice!existed!or!had!been!tested.!To!take!the!READ!concept!to!

the!next!level,!to!advance!the!hypothesis!into!a!working!model,!I!turned!to!the!Doctor!of!Creative!

Industries!as!the!means!for!completing!this!work!as!part!of!a!process!of!distinct!academic!research.!

The!DCI!professional!doctorate!offered!a!unique!opportunity!for!me!as!an!industry!practitioner!to!

apply!rigorous!scholarship!to!the!task!of!developing!the!READ!model!at!my!site!of!practice.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

5!This!team!was!led!by!Paul!Brown!as!Creative!Producer!and!myself!as!program!manager!for!Nuclear!Futures,!for!the!host!

organisation!Alphaville.!


! 6!

While!the!evaluation!practice!for!Nuclear!Futures!–!and!in!particular,!for!the!Balaklava!Community!

Arts!Project,!which!was!the!sole!focus!of!the!DCI!research!–!was!structured!as!interrelated!with!my!

professional!role!as!the!program!manager,!the!DCI!research!enquiry!was!designed!and!approached!

as!separate!from!this!managerial!role.!Framed!through!my!practitioner!perspective,!the!DCI!research!

fits!within!the!tradition!of!artists,!managers!and!evaluators!seeking!to!address!the!persistent!

epistemological!and!technical!problems!raised!by!community!arts!evaluation!in!practice.!In!doing!so,!

the!research!aimed!to!embrace,!as!well!as!unleash,!the!potential!of!evaluation!within!its!community!

arts!setting!by!designing!and!experimenting!with!solutions!grounded!in!the!creative!arts!practice.!

1.3! Problem#statement#

Practitioners!endeavouring!to!appraise!the!art*making!processes,!outcomes!and/or!impact!of!

community!arts!must!traverse!myriad!philosophical!and!technical!challenges,!particularly!for!large!

and/or!multifaceted!programs!(MacDowall!et!al.,%2013).!As!a!routine!requirement!of!government!

and!philanthropic!funding,!evaluation!has!become!a!recurrent!aspect!of!community!arts!practice!and!

management!(Keating,!2002).!While!substantial!advances!have!been!made!over!the!past!30!years,!

community!arts!evaluation!remains!problematic,!ad!hoc!and!inconsistent!(Goldbard,!2015;!Belfiore!

&!Bennett,!2010).!In!the!literature,!many!areas!remain!unresolved,!such!as!best!practice!standards!

for!evaluating!sizeable!or!complex!programs,!or!an!agreed!professional!framework!for!practice.!!

Community!arts!practitioners!also!face!mounting!pressure!to!demonstrate!the!value!of!their!work!in!

terms!of!the!instrumental!benefits!and!impact!of!the!arts!practice!(Badham,!2015;!Hadley!&!

Gattenhof,!2012).!As!discussed!in!Chapter!2,!the!trend!to!measure!the!social!and!economic!impacts!

of!the!arts!practice,!linked!to!the!instrumentalisation!of!the!arts!and!debate!on!cultural!value,!has!

strongly!influenced!how!contemporary!community!arts!practice!is!framed,!evaluated!and!

represented!(MacDowall,!2015;!Matarasso,!1996).!Arising!are!epistemological!debates!in!the!

literature,!with!some!opponents!arguing!that!evaluation!in!its!current!usage!is!detrimental!and!

contravenes!the!traditional!ethos!of!community!arts!practice!(Goldbard,!2015,!2008;!Belfiore,!2014).!!

In!the!industry!domain,!the!notion!of!what!constitutes!best!practice!and!effective!evaluation!is!also!

elusive!(MacDowall!et!al.,%2013).!While!some!good!practitioner!guides!and!platforms!exist!for!

sharing!the!program!evaluation!models!that!are!currently!available!(see!Chapter!2),!overall,!there!is!

limited!public!access!to!industry*produced!exemplars!(and!critical!analysis!of!these).6!As!a!

practitioner!interested!in!learning!from!evaluations!in!the!public!domain,!the!task!of!ascertaining!the!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

6!Based!on!data!arising!from!interviews!with!industry!stakeholders!and!practitioners!undertaken!for!this!research,!and!from!

my!own!experience!of!seeking!permissions!via!organisations!to!access!exemplars!during!the!contextual!scan!for!the!DCI.!


! 7!

quality!of!examples!accessed!is!self*determined,!given!there!is!no!industry!standard!or!benchmark!

for!best!practice!in!the!field.!Questions!also!arise!about!how!representative!industry!exemplars!are!

of!evaluation!and!evaluative!practice!in!the!field,!as!community!arts!evaluation!has!been!

traditionally!orientated!towards!acquittal!and!accountability!agendas,!which!may!not!always!reflect!

the!cultural!or!aesthetic!values!of!all!participating!partners!(Hadley!&!Gattenhof,!2012).!!

Based!on!observation!from!practice,!and!anecdotal!discourse,!there!appears!to!be!a!tendency!for!

community!arts!practitioners!to!feel!pressure!to!apply!a!positivist!and!utilitarian!evaluative!lens,!or!

to!orientate!towards!conservative!or!success*oriented!framing!of!evaluation!findings.7!Practitioners!

adopting!a!cautious!or!narrow!approach!may!do!so!for!myriad!reasons,!such!as!in!response!to:!

perceived!funder!expectations;!the!desire!to!enhance!future!funding!eligibility;!time!and!resource!

constraints;!or!lack!of!evaluation!expertise!(MacDowall!et!al.,%2013;!Fitzpatrick,!Sanders,!&!Worthen,%

2012;!Badham,!2010).!Also,!evaluation!practice!since!the!late!twentieth!century!has!been!influenced!

by!the!global!movement!toward!cultural!and!progress!measurement,!which!has!led!to!increasing!

attempts!to!quantify!cultural!value!and!arts!impact!via!indicators!and!metrics!(MacDowall,!2015).!!

Beyond!the!standard!evaluation!approaches!undertaken!for!acquittal!reporting!in!the!field,!harder!to!

find!are!evaluation!exemplars!from!industry!that!provide!genuine!insight!into!the!internal!and!

nuanced!organisational!learning,!appraisal!and!knowledge*generating!processes!at!play!in!the!

community!arts!practice!and!evaluation,!or!practitioner!reflections!on!experimental!practices!and!

failures!during!art*making!and!evaluation.!For!creative!practitioners,!evaluators!and!managers!

responsible!for!evaluating!community!arts,!the!philosophical!and!technical!challenges!of!evaluation!

in!this!context!must!be!traversed!independently!(Australasian!Evaluation!Society,!2010).!That!is,!in!

the!absence!of!industry*wide!models!or!benchmarks!for!best!practice,!practitioners!must!use!

evaluation!models!according!to!what!they!determine!to!be!most!effective!and!ethical!on!a!project*

to*project!basis.!In!undertaking!the!methodological!design!and!implementation!of!community!arts!

evaluation,!the!challenges!may!include:!addressing!diverse!or!unclear!stakeholder!evaluation!needs;!

partnership!brokerage;!method!selection!to!ensure!rigorous!and!defensible!findings;!and!delivering!

end*utility!and!relevant!outputs!for!partners!(MacDowall,!2015;!Fitzpatrick!et!al.,%2012;%Davidson,!

2005).!With!the!rise!of!participatory*oriented!evaluation!for!community!arts!initiatives,!such!issues!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

7!Based!on!discourse!I!was!privy!to!at!three!industry!events!organised!as!part!of!the!three*year!Australian!Research!Council!

(ARC)!Linkage!Project!Towards%an%integrated%evaluation%framework%for%communityAbased%arts!by!the!Centre!for!Cultural!Partnerships,!University!of!Melbourne!(see!MacDowall!et!al.,!2013).!This!included:!a!two*day!roundtable!with!40!

stakeholders,!a!masterclass!for!postgraduate!students!on!the!topic!of!community!arts!evaluation,!and!network!and!forum!

discussions!with!members!of!the!international!community!of!community!arts!workers!and!evaluators!attending!the!

Spectres%of%Evaluation%conference!held!in!2014!at!Footscray!Arts!Centre,!Melbourne.!


! 8!

become!compounded!depending!on!the!scale!and!complexity!of!the!partnerships!involved!(Guba!&!

Lincoln,!2004).!It!is!not!surprising!that!trepidation!of!evaluation!is!evident!in!the!field!(Belfiore,!2014;!

Goldbard,!2008).!

The!epistemological!and!technical!quandaries!arising!from!the!task!of!evaluating!community!arts,!

and!the!broader!debate!on!cultural!value,!forms!the!backdrop!to!the!evaluation!challenges!faced!in!

my!professional!practice.!From!my!experience,!questions!of!what!constitutes!effective!evaluation!

have!been!recurrent!in!my!roles!of!program!manager!and!evaluator.!On!a!project*by*project!basis,!

over!the!many!years!that!I!have!worked!in!these!domains,!I!have!attempted!to!continuously!improve!

my!evaluation!practice!and!build!on!the!mixed!successes!that!have!ensued.!The!scope!and!scale!of!

the!Nuclear!Futures!arts!practice!however!presented!uncharted!ground!from!my!previous!evaluation!

experience.!In!order!to!gain!the!necessary!in*depth!understanding!of!community!arts!evaluation!

relevant!for!tackling!the!Nuclear!Futures!evaluation!problem,!in!2013!I!decided!to!frame!this!

industry*based!challenge!as!a!research!problem!via!the!DCI.!As!a!research!problem,!the!question!of!

how!to!evaluate!the!Nuclear!Futures!program!had!significance!for!my!site!of!practice!and!my!own!

professional!development,!and!strongly!resonated!with!identified!research!and!knowledge!gaps!

currently!present!in!the!field.!

1.4! Aims#and#objectives#

This!DCI!seeks!to!make!an!original!contribution!to!the!discourse!on!effective!evaluation!in!community!

arts,!by!developing!and!examining!a!newly!devised!participation*oriented!model!for!creative!

community!partnerships!and!community!arts!that!adopts!an!integrated!and!wide*lens!approach!to!

evaluation.!!

Specifically,!the!research!aimed!to!develop!the!pre*existing!READ!concept,!coined!during!professional!

practice!prior!to!this!research,!into!an!operational!model!through!an!iterative!industry*based!process!of!

development,!application,!refinement!and!critical!analysis!in!order!to!create!a!substantiated!READ!

product!for!immediate!and!future!evaluation!practice.!!

!

!

!


! 9!

The!DCI!research!addressed!the!overarching!questions:8!

a)! How!does!the!READ!model!contribute!to!program!learning!and!appraisal!for!creative!

community!partnerships!within!community!arts,!based!on!the!Nuclear!Futures!Partnership!

Initiative!experience?!!

b)! What!innovation!does!READ!offer!evaluation!practice!and!program!evaluation?!

The!research!objectives!of!the!DCI!were!to:!

•! Ascertain!the!dominant!conceptualisations!and!challenges!of!evaluation!identified!from!

representatives!in!community!arts.!

•! Identify!and!assess!the!program!evaluation!needs!of!Nuclear!Futures!partners!and!stakeholders.!

•! Examine!current!evaluation!practice!and!models!of!evaluation!for!community!arts!of!relevance!

to!the!Nuclear!Futures!program!evaluation.!

•! Develop!and!implement!the!proposed!READ!model!for!industry!application.!

•! Modify,!improve!and!evaluate!the!READ!model!at!the!site!of!practice.!

•! Refine!and!critique!the!READ!model!product!as!a!methodology!for!community!arts!evaluation.!

•! Contribute!to!the!discourse!on!effective!evaluation!practice,!identifying!areas!of!innovation!and!

wider!practitioner!relevance.!

1.5! READ#product#development#

In!meeting!the!above!aims!and!objectives,!there!are!two!key!ways!the!research!has!advanced!the!

READ!concept.!First,!through!the!process!of!designing,!and!experimenting!with,!an!applied!READ!

methodology!for!the!Balaklava!project!a!model!of!practice!specific!to!that!context!was!developed.!

This!included!the!how!and!why!of!the!approach!as!it!was!designed,!applied!and!refined!(see!Chapter!

5).!Second,!in!parallel!and!interrelated!to!this,!I!examined!the!emergent!evaluation!practice!with!a!

view!to!identify!the!elements!of!the!evaluation!practice!and!methodology!that!might!be!

transferrable!and!relevant!for!a!general!READ!model!of!practice!(applicable!and!reliable!for!other!

projects!across!the!Nuclear!Futures!suite!and!beyond!the!site!of!practice).!!As!explained!in!Chapter!6,!

I!examined!the!theoretical!and!technical!aspects!of!the!developed!model!in!order!to!gain!an!in*depth!

understanding!of!the!READ!approach,!its!operational!framework,!and!the!implications!for!evaluation!

practice.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

8!To!meet!the!requirements!of!the!DCI!course,!this!research!was!conducted!as!two!one*year!research!projects,!Project!1!

and!Project!2,!in!2014!and!2015!respectively.!Both!Project!1!and!2!focused!on!the!industry*based!Balaklava!Community!

Arts!Project!and!READ!model!application.!For!the!final!DCI!submission,!this!dissertation!and!accompanying!READ%in%Practice!portfolio,!I!present!the!research!aims,!phases!and!findings!of!Project!1!and!2!as!a!complete!(integrated)!body!of!work.!


! 10!

Culminating!from!these!processes,!I!developed!an!evaluation!product!for!READ!that!encapsulated!

the!refined!elements!and!principles!of!the!model,!to!present!a!simplified!and!generic!READ!model!of!

practice!communicated!via!a!practitioner*friendly!infographic!and!profile!statement.9!It!was!

envisaged!that!the!READ!product!(presented!in!Chapter!6)!would!be!useful!for!the!future!evaluation!

practice!of!the!Nuclear!Futures!program,!as!well!as!a!tool!for!my!own!professional!evaluation!

practice!and!that!of!other!practitioners!or!organisations.!The!process!of!developing!the!model!of!

practice!and!READ!product!has!led!to!new!understanding!into!previously!unknown!aspects!of!the!

READ!concept,!including!original!knowledge!related!to!the!tested!methodological!design.!!

In!order!to!demonstrate!the!level!to!which!the!READ!concept!has!been!developed!via!this!research,!

it!is!helpful!to!reference!my!starting!point.!Below!is!an!extract!from!Alphaville’s!funding!submission!

to!the!Australia!Council!for!the!Arts!(based!on!ideas!from!2012)!that!illustrates!the!extent!to!which!

the!READ!concept!had!been!formulated!and!articulated!prior!to!this!research.!I!refer!to!this!early!

iteration!of!the!approach!(prior!to!the!research)!as!the!READ!concept!or!conceptual!READ!model:!

We!propose!to!develop!a+new+participatory+model+that+integrates+Reflection,+Evaluation,+

Analysis+and+Documentation!(the!‘READ’!model)!to!evaluate!the!program’s!processes,!

partnerships!and!creative!outputs.!!

•! We!will!use!a!partnership!model!based!on!participatory!evaluation!methodology!and!

reflective!analysis,!with!evaluative!goals!set!by!partners!to!inform!academic*supported!

framework!design!and!implementation.!

•! The!approach!blends!traditional!data!collection!and!analysis!(qualitative!and!quantitative)!

with!emergent!techniques!such!as!‘Most!Significant!Change’!process!(story*based!

evaluation).!It!involves!all!partners!in!asking!the!questions,!data!collection!and!analysis.!(It!

is!socio*dynamic.)!

•! READ!includes!formative!evaluation!and!analysis!for!ongoing!appraisal!and!summative!

evaluation!at!completion,!with!focus!on!CACD!processes,!the!creative!works!and!the!

quality!of!the!partnerships.!

•! Reflective!analysis!originates!in!individual!and!group!practice!within!individual!projects,!

and!proceeds!via!conferences,!seminars!and!publications!that!link!creative!artists,!

academics!and!communities.!!

Strengths:!

•! The!overall!approach!is!documentation*oriented!(written,!visual,!audio,!ephemeral/!rich!

media,!as!well!as!archival),!with!commitment!to!disseminate!READ!findings!to!

communities,!partners!and!industry!to!make!lessons!explicit!&!accessible!to!wider!field.!

Our!on*line!community!is!a!key!delivery!mechanism.!

•! READ!is!evidence*based,!continuous!over!multiple!years,!and!guided!by!socially!inclusive!

principles.!It!is!transferable!and!relevant!for!other!Community*based!Arts!and!Creative!

Industries!practitioners.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

9!A!product!is!defined!as!‘a!good,!idea,!method,!information,!object!or!service!created!as!a!result!of!a!process!and!serves!a!

need!or!satisfies!a!want.!It!has!a!combination!of!tangible!and!intangible!attributes’!(WebFinances,!2016).!


! 11!

•! READ!develops!a!culture!of!learning!and!reflection!that!will!support!professional!

development!and!is!a!key!strategy!for!sustaining!long*term!collaborative!partnerships!and!

capacity!of!communities!and!practitioners.!

•! Evaluation!is!supported!by!an!extensive!‘community!of!researchers’!committed!to!action!

research,!and!therefore!is!likely!to!generate!robust!evaluation!findings.!The!lead!institution!

is!QUT’s!Creative!Industries!Faculty,!which!will!host!two!doctoral!level!research!projects!

dedicated!to!Nuclear!Futures.!

(Extract,!Alphaville!fundraising!documentation,!2012!describing!the!proposed!READ!model)!

Above,!is!the!complete!version!of!what!had!been!written!on!READ!prior!to!the!research.!While!the!

basic!model!concept!and!acronym!were!coined,!no!methodology!or!implementation!strategy!had!

been!conceived.!The!comprehensive!and!sophisticated!READ!findings!presented!in!this!dissertation!

demonstrate!significant!advancement!and!innovation!of!the!READ!model!since!its!early!inception.!!

1.6! The#scope#and#significance#of#the#research#

The!research!has!impacted!my!professional!evaluation!practice!and!established!protocols!for!

evaluation!at!my!site!of!practice.!The!findings!lay!the!foundations!for!the!further!development!of!

innovative!evaluation!practices!across!the!Nuclear!Futures!program,!as!well!as!the!future!trial!and!

application!of!the!new!model!to!other!projects!and!contexts.!For!the!field,!the!research!presents!an!

original!and!practical!contribution!to!community!arts!evaluation!practice,!delivering:!

•! an!integrated!evaluation!method!that!can!be!done!‘in!house’;!

•! a!model!of!evaluation!best!practice!that!embraces!the!ethos!of!community!arts;!

•! a!flexible!framework!for!generating!robust!data,!that!is!linked!with!management!and!

creative!arts!practice!so!that!data!can!be!fed!back,!continuously,!into!the!project!for!real*

time!impact;!

•! original!strategies!for!engagement!and!for!developing!stakeholder!ownership!of!project!

evaluation,!particularly!where!trepidation!or!resistance!to!traditional!evaluation!exists;!

•! possibilities!for!more!general!application!in!(research*led)!creative!management.!

While!the!research!makes!the!above!contributions,!it!is!acknowledged!that!the!body!of!evidence!

presented!constitutes!the!first!of!several!iterations!of!critically!analysing!the!READ!model’s!

development!and!testing,!and!thus!the!DCI!research!does!not!offer!a!complete!critique!of!the!READ!

model.!The!DCI!study!has!been!limited!to!the!first!of!many!READ!applications!for!the!Nuclear!Futures!

program,!which!is!ongoing!beyond!the!timeframe!of!the!research.!Further,!in!identifying!the!trialled!

model’s!strengths!and!constraints,!the!DCI!research!raises!several!questions!that!will!need!to!be!

addressed!for!future!development,!to!maintain!validity!and!credibility!during!more!complex!

applications!such!as!the!Nuclear!Futures!summative!program!evaluation!in!2017.!To!minimise!and!

manage!the!identified!vulnerabilities!of!the!model,!I!make!several!recommendations!in!Chapter!6!

intended!to!improve!future!evaluation!practice!(see!Section!6.2.5).!


! 12!

1.7! The#DCI#journey#

1.7.1! Research#design#overview#

In!order!to!critically!analyse!my!evaluation!practice,!the!READ!application!and!the!emergent!READ!

model,!I!employed!a!robust!practice*informed!research!methodology!for!the!DCI!that!created!a!

reflective!lens!on!my!professional!practice.!As!outlined!in!Chapter!3,!the!DCI!research!design!

harnessed!forensic!reflective!practice,!a!knowledge*driven!research!strategy!involving!the!systematic!

examination!of!thinking!and!action!for!critical!enquiry!and!for!the!continuous!improvement!of!

practice!(Schon,!1983).!The!aim!of!the!reflective!enquiry!was!to!generate!original!knowledge!about!

the!READ!approach,!and!my!own!evaluation!practice,!as!well!as!identify!areas!of!innovation!for!

evaluation!practice!more!generally.!Figure!1.1!depicts!the!relationship!between!the!overarching!DCI!

research!methodology!and!the!READ!model!application/!methodology!(as!subject).!!

!

!

Figure!1.1:!The!Doctor!of!Creative!Industries!research!design!

!

!

!

!


! 13!

1.7.2! Research#phases#

It!is!important!to!emphasis!that!the!overarching!DCI!methodology!employed!for!this!research,!

illustrated!in!Figure!1.1,!is!distinct!from!the!READ!methodology!which!was!the!subject!of!the!

research.!The!DCI!research!design!(see!Chapter!3)!created!the!primary!lens!for!the!analysis!of!the!

READ!application!for!the!purpose!of!understanding,!assessing!and!developing!the!model!in!action,!

and!for!the!continuous!improvement!of!evaluation!practice.!Forensic!reflective!practice!prior,!during!

and!following!the!READ!application!enabled!the!critical!investigation,!review!and!iterative!

modification!of!the!developing!READ!model,!and!it!was!via!this!phased!research!progression!that!the!

DCI!findings!and!conclusions!presented!in!Chapter!6!and!7!were!generated.!This!progression!for!the!

DCI!research!consisted!of!the!following!six!research!phases:!

•! A!contextual!scan!covering!the!DCI!research!topic!and!community!arts!context.!

•! Scoping!undertaken!with!Nuclear!Futures!and!industry!stakeholders.!

•! Planning!and!design!of!the!tailored!READ!methodology!(system)!for!Balaklava.!

•! READ!implementation!(data!collection!and!analysis!for!the!READ!application).!

•! Production,!dissemination!and!use!of!READ!outputs!for!the!READ!application.!

•! DCI!research!analysis!of!the!data!collected!through!the!READ!application.!

As!is!represented!in!Figure!1.2,!the!six!research!phases!completed!for!the!DCI!included!four!phases!

associated!with!the!READ!application!for!the!Balaklava!Community!Arts!Project.!While!the!progress!

of!DCI!research!phases!appears!linear!in!Figure!1.2,!they!did!not!conform!to!a!chronological!

sequence!as!the!diagram!suggests,!and!in!reality!were!iterative!research!processes.!

!

Figure!1.2:!The!six!research!phases!for!the!DCI!research!!

!

!

!

Research phases across DCI

Forensic Reflective Practice (ongoing)

Research phases related to READ application (see white boxes)


! 14!

1.7.3! Examinable#DCI#outputs##

The!DCI!research!journey!culminated!in!two!assessable!research!outputs:!

•! A!written!dissertation!(this!document)!–!detailing!the!problem!statement,!research!

methodology,!rich!descriptions!of!evaluation!practice,!key!findings!and!the!implications!of!

the!research!for!the!field.!(See!Section!1.8!for!chapter!overview.)!

•! The!READ%in%Practice!portfolio!(RIPP)!(companion!document!to!the!dissertation)!–!includes!

selected!examples!from!my!industry*based!evaluation!practice!and!READ!outputs.!

The!DCI!research!outputs!have!been!collated!to!illuminate!the!knowledge!claims!made!in!the!

research!findings!(Chapter!6!and!7).!The!curation!of!the!outputs!is!based!on!the!rationale!that!the!

evaluation!practice!undertaken!in!the!professional!context!for!the!DCI!research!constitutes!relevant!

research!deliverables.!Reference!to!and!explanation!of!the!items!included!in!the!accompanying!

practice*based!portfolio!are!detailed!throughout!the!dissertation.!For!cross*referencing!purposes,!I!

have!used!the!icon!in!the!right!margin,!to!signpost!when!original!materials!from!the!READ%in%

Practice!portfolio!(RIPP)!are!relevant!to!the!discussion.!The!article!and!page!numbers!for!RIPP!

items!are!referenced!in!the!dissertation!text.!

The!two!documents!are!submitted!as!companion!research!outputs.!The!dissertation!articulates!the!

evaluation!theory!and!developed!model!of!practice!for!the!READ!approach!and!provides!a!

descriptive!account!of!the!collective!application!at!the!site!of!practice.!The!portfolio!includes!

demonstrations!of!the!industry*based!evaluation!practice.!While!I!cross*reference!from!the!

dissertation!to!the!RIPP,!the!portfolio!is!intended!as!an!accompanying!research!and!documentation!

output.!!

1.8! Chapter#overview#

In!this!Chapter!1,!I!have!introduced!the!Doctor!of!Creative!Industries!research!as!a!scholarly!enquiry!

arising!from!my!professional!practice!as!a!program!manager!and!evaluator,!and!explained!its!dual!

position!in!the!academic!and!industry!contexts!of!community!arts.!In!establishing!the!relevance!of!

the!research!problem,!the!chapter!canvasses!several!of!the!unresolved!and!contested!aspects!of!

evaluation!practice!in!the!field,!and!relates!them!to!the!challenges!faced!at!my!site!of!practice.!The!

research!seeks!to!contribute!to!the!identified!knowledge!gaps!on!effective!evaluation!of!community!

arts,!in!particular!for!complex!creative!partnership!initiatives,!using!a!professional!practitioner!lens!

to!frame!the!research.!!

!

!


! 15!

Central!to!the!research!enquiry!is!the!development!and!critical!review!of!a!newly!devised!

participatory!evaluative!model!that!integrates!Reflection,!Evaluation,!Analysis!and!Documentation!

(READ).!As!explained!in!the!Introduction,!the!model!has!been!designed!via!a!collective!process!for!

the!international!Nuclear!Futures!arts!and!cultural!program,!which!I!led!and!substantially!informed!

as!researcher.!The!research!centres!on!the!reflective!analysis!of!the!READ!model’s!development,!and!

performance!as!a!learning!and!appraisal!framework,!based!on!the!Nuclear!Futures!experience!of!

applying!the!model!in!practice.!In!the!introduction,!I!also!outlined!the!research!phases!of!the!DCI!and!

the!two!DCI!research!outputs!–!a!dissertation!(this!document)!and!a!portfolio!of!materials!generated!

through!the!evaluation!practice!of!the!READ!application!(the!READ%in%Practice!portfolio!–!RIPP).!%

In!Chapter!2,!I!provide!a!comprehensive!contextual!scan.!The!contextual!scan!charts!the!historical!

and!theoretical!trends!relevant!to!the!research!problem!based!on!the!wide*ranging!review!of!

scholarly!and!industry!secondary!sources!from!key!literature!areas!in!community!arts!and!

evaluation.!In!detailing!the!broader!context!of!the!READ!model!application,!I!outline!the!political!and!

cultural!traditions!defining!community!arts!practice!and!overview!the!rapidly!growing!field!of!

evaluation,!honing!in!on!program!evaluation!of!community!arts.!Of!relevance!are!the!contemporary!

participation*oriented!evaluation!approaches!(Sections!2.2.2!and!2.3.2),!where!partnerships!and!

participation!become!central!to!evaluation!practice!and!stakeholder!engagement,!empowerment!

and!capacity!building!get!embedded!in!the!evaluative!processes.!!Equally!significant!for!the!research!

topic!is!the!debate!on!cultural!value,!and!trend!for!evaluation!to!achieve!a!determination!and!

valuation!of!the!impact!of!community!arts!practice,!which!has!had!various!ramifications!for!the!field.!

In!Chapter!3,!I!present!the!overarching!DCI!research!design!and!outline!the!methodology!employed!

for!studying!the!READ!application.!In!order!to!adequately!address!the!research!problem,!a!robust!

qualitative!methodology!was!implemented!that!harnessed!forensic!reflective!practice,!and!this!

utilised!qualitative!methods!such!as!reflection,!digital!storytelling,!interviews!and!observation.!

Chapter!3!explains!the!rationale!of!the!practice*informed!research!approach!and!argues!the!benefits!

afforded!by!the!methods!selected!(Section!3.3).!I!detail!the!systematic!program!of!forensic!reflective!

practice!undertaken,!which!involved!reflection!prior,!during!and!following!the!READ!application,!as!

the!central!research!strategy!for!the!investigation.!Combined,!these!first!three!chapters!set!up!the!

research!enquiry!and!position!it!within!the!field,!ahead!of!the!following!chapters!that!cover!the!

industry*based!evaluation!practice!for!the!Balaklava!Community!Arts!Project.!

Chapter!4!provides!a!rich!descriptive!account!of!the!site!of!practice,!and!the!art*making!defining!the!

Balaklava!Community!Arts!Project.!The!chapter!opens!with!background!information!about!the!

Nuclear!Futures!initiative,!the!multifaceted!international!community!arts!program!from!which!the!


! 16!

Balaklava!project!is!delivered.!The!Balaklava!project!encompassed!a!series!of!multi*arts!residencies!

driven!by!multiple!partnerships!and!documentary!art*making!activities!in!the!Wakefield!region!

(north!of!Adelaide)!in!South!Australia.!The!focus!of!the!project!was!to!engage!regional!communities!

in!the!arts!and!work!with!atomic!survivor!community!partners!to!make!story*based!artworks!

exploring!the!legacies!of!the!nuclear!age.!To!showcase!the!diversity!and!complexity!of!the!

community!arts!processes,!and!the!creative!works!produced!for!the!Balaklava!project,!I!use!creative!

documentation!in!this!chapter!to!represent!the!community!arts!practice.!!

In!Chapter!5!I!elaborate!on!my!role!as!the!READ!facilitator,!in!which!I!led!the!collective!design!and!

implementation!process!to!develop!the!READ!concept!into!a!functioning!model!for!the!Balaklava!

project.!This!was!a!dynamic!and!demanding!role,!and!in!Chapter!5!I!provide!in*depth!coverage!of!the!

evaluation!practice!undertaken!and!the!emergent!READ!methodology.!The!chapter!begins!with!

background!information!on!the!READ!model’s!inception,!and!uses!a!basic!systems!approach!to!

explain!the!prospective!model!(Section!5.1).!Six!core!system!elements!of!the!READ!concept!are!

identified,!and!each!is!discussed!in!turn!in!the!chapter.!The!six!elements!charted!are:!the!support!

structures!for!READ!evaluation!practice;!Reflection;!Evaluation;!Analysis;!Documentation;!and!the!

integration,!utilisation!and!dissemination!of!the!READ!outputs.!In!profiling!the!READ!application,!the!

implications!of!the!interpretative!stance,!participatory!ethos!and!internal!positioning!of!the!

evaluation!are!elucidated.!Arising!from!the!scoping!phase!with!stakeholders,!also!presented!in!the!

first!half!of!Chapter!5,!are!the!site*specific!needs!and!challenges!for!evaluating!the!project,!several!of!

which!resonate!with!the!trends!and!issues!identified!in!the!contextual!scan.!Issues!included!a!lack!of!

stakeholder!clarity!about!evaluation!needs,!the!presence!of!divergent!world!views,!methodological!

design!issues!and!for!some!stakeholders,!an!existing!level!of!trepidation!of!evaluation.!!

In!the!second!part!of!Chapter!5!I!detail!each!of!the!four!enquiry!strategies!of!the!developed!READ!

model!(Reflection,!Evaluation,!Analysis!and!Documentation)!to!illuminate!how!each!was!designed!

and!implemented.!This!in*depth!documentation!of!the!READ!application!defines!both!the!technical!

and!epistemological!aspects!of!the!evaluation!practice!for!the!READ!approach!that!I!conducted!for!

the!research.!Finally,!in!Section!5.5,!I!discuss!the!processes!I!employed!for!integrating,!utilising!and!

disseminating!the!READ!data!and!outputs!generated!for!the!Balaklava!project.!Ultimately,!the!

systems!investigation!of!the!READ!design!and!implementation!presented!in!Chapter!5!demonstrates!

the!operations,!functionality!and!outcomes!of!the!applied!READ!approach!during!the!Balaklava!

experience,!evidenced!by!qualitative!data!generated!and!practical!examples!from!the!READ%in%

Practice!portfolio.!

!


! 17!

Next,!in!Chapter!6,!I!present!the!findings!arising!from!READ!application!and!explore!a!number!of!

emergent!themes!about!the!evaluation!practice.!The!DCI!research!methodology!elicited!rich!data!

about!the!READ!application!and!this!chapter!engages!with!the!practical!lessons!that!were!made!

explicit!via!the!reflective!analysis.!The!identified!themes!and!lessons!provide!insight!into!the!efficacy!

and!issues!of!the!developed!READ!model!as!an!applied!learning!and!appraisal!framework,!to!inform!

the!summative!conclusions!presented!in!Chapter!7.!Revealed!is!the!relevance!of!the!internal!

positioning!of!the!READ!facilitation,!the!model’s!versatility!as!a!modular!or!integrated!system,!and!

the!demonstrated!benefits!of!creative!documentation!and!collective!reflection.!The!findings!also!

highlight!the!implications!of!including!legacy!goals!as!a!purpose!of!evaluation!practice,!which!

resulted!in!the!long*term!public!storage!and!distribution!of!several!READ!outputs.!Chapter!6!

culminates!in!the!presentation!of!the!consolidated!READ!product!which!I!designed!via!the!DCI,!that!

articulates!the!collectively!developed!and!refined!model.!For!the!purpose!of!general!presentation,!I!

devised!an!infographic!to!visually!encapsulate!the!developed!READ!model,!as!a!communications!tool!

for!future!applications,!such!as!the!summative!evaluation!of!the!Nuclear!Futures!program!in!2017.!

In!the!final!chapter!of!the!dissertation,!Chapter!7,!I!present!my!summative!conclusions!for!the!DCI!–!

a!final!reflection!on!the!research!problem!and!findings!generated.!Based!on!the!research!findings,!I!

state!the!claims!of!relevance,!effectiveness!and!originality!for!the!READ!model!in!meeting!the!

stakeholder!evaluation!needs!at!the!site!of!practice,!and!elucidate!the!areas!of!innovation!offered!

for!evaluation!practice.!I!argue!that!READ’s!innovation!lies!in!its!deliberate!modular!and!integrated!

approach,!and!that!READ’s!diversified!learning!and!appraisal!strategies!fulfil!multiple!purposes!

relevant!for!community!arts,!including!accountability,!improvement,!broader!learning!and!instituting!

legacy.!In!defining!the!significance!of!the!research,!I!discuss!the!implications!of!the!findings!for!

evaluation!practice!–!on!my!own!practice,!at!the!site!of!practice!and!for!the!field.!In!doing!so,!I!argue!

that!the!research!makes!an!original!contribution!to!the!discourse!and!practice!of!community!arts!

evaluation!and!that!the!research!has!the!potential!to!significantly!impact!the!field.!To!close!the!

chapter,!I!outline!my!intentions!for!research!dissemination!and!directions!for!future!research.!

1.8.1! Dissertation#roadmap#

In!order!to!map!the!chapter!progression!of!the!dissertation,!as!summarised!in!Section!1.8,!I!have!

graphically!represented!the!dissertation’s!structure!and!relationship!to!the!READ%in%Practice%

portfolio!in!Figure!1.3.!As!illustrated,!the!first!three!chapters!(see!below!in!blue)!focus!on!the!context!

and!research!design!for!the!DCI.!Next,!in!Chapters!4!and!5,!I!document!the!industry*based!process!of!

developing!and!testing!the!READ!model!for!the!Nuclear!Futures!site!of!practice!(see!below!in!red).!

These!chapters!have!strong!links!with!the!accompanying!READ%in%Practice!portfolio,!and!I!cross!


! 18!

reference!to!the!portfolio’s!practical!examples!throughout!Chapters!4!–!6!to!reveal!the!depth!and!

breadth!of!the!evaluation!practice!conducted.!Next,!I!provide!a!critical!analysis!of!the!READ!model!in!

action,!in!Chapter!6,!that!scrutinises!the!READ!application!based!on!the!findings!of!my!forensic!

reflective!practice!undertaken!before,!during!and!after!the!READ!trial!and!consultation!with!

stakeholders.!The!knowledge!presented!in!Chapter!6!is!then!refined!in!Chapter!7,!to!directly!answer!

the!key!research!questions!for!the!DCI.!

!

!

!

!

#

!

!

!

Figure!1.3:!Roadmap!for!dissertation!

! !

!

!


! 19!

2! Contextual#scan#

The!contextual!scan!presented!in!Chapter!2!examines!the!historical!and!theoretical!background!

relevant!to!community!arts!evaluation!and!the!participation*oriented!approach!of!the!emergent!

READ!model.!Extensive!review!of!scholarly!and!industry!secondary!sources!(from!several!English*

speaking!continents)!has!been!undertaken,!in!the!key!literature!areas!of!community!arts!and!

evaluation,!to!ensure!the!DCI!research!builds!on!existing!knowledge,!scholarship,!and!industry!

practice.!In!order!to!navigate!the!vast!quantity!of!literature!within!these!fields,!I!begin!the!contextual!

scan!broadly!and!then!hone!in!on!the!evaluation!theory!and!methodologies!of!most!relevance!for!

community!arts,!including!an!examination!of!contemporary!debates!and!pertinent!trends.!!

First,!the!political!and!cultural!traditions!defining!community!arts!practice!are!outlined!to!chart!the!

emergence!of!the!community!partnerships!sector!within!community!arts.!It!is!within!this!domain!

that!the!Nuclear!Futures!Partnership!Initiative!(Nuclear!Futures),!my!site!of!practice,!is!positioned.!

Second,!a!theoretical!overview!of!evaluation!is!presented,!with!a!focus!on!program!evaluation.!

While!program!evaluation!is!a!distinct!subset!within!the!evaluation!field,!the!epistemologies!and!

methodologies!applied!for!the!evaluation!of!programs!and!projects!is!extremely!divergent.!

Underlying!the!dominant!approaches!and!models!for!program!evaluation!is!a!wide!spectrum!of!

competing!paradigms,!from!positivist!to!values*oriented,!which!have!significant!bearing!on!how!

evaluation!is!undertaken!and!for!what!purpose!and!outcome.!

Next,!the!dominant!trends!and!challenges!for!community!arts!evaluation!are!discussed,!with!

reference!to!the!pertinent!debates!that!set!the!backdrop!to!this!DCI!research.!Of!relevance!is!the!

contemporary!debate!on!cultural!value,!and!the!increasing!orientation!of!conventional!evaluation!

towards!the!use!of!quantitative!tools!for!impact!measurement!(such!as!cultural!metrics!and!

indicators).!Evident!is!the!extent!to!which!community!arts!evaluation!has!mirrored!broader!

evaluation!trends,!which!has!led!to!the!instrumentalisation!of!the!arts!over!the!past!30!years.!To!

frame!such!debates!in!relation!to!the!research,!I!canvass!the!political,!technical!and!philosophical!

dimensions!of!contemporary!evaluation!practice!and!examine!the!possible!ramifications!of!current!

evaluation!trends,!that!some!scholars!argue!is!highly!detrimental!to!the!field!(Goldbard,!2015).!

Finally,!a!synopsis!of!contemporary!participation*oriented!approaches!and!methods!for!community!

arts!evaluation!is!provided.!For!the!Nuclear!Futures!context,!where!partnerships!and!participation!

are!central!to!the!arts!practice,!exemplars!and!models!that!have!been!able!to!achieve!stakeholder!

engagement,!empowerment!and!capacity!building!as!part!of!the!evaluative!process!were!of!high!

relevance!for!the!collective!design!and!implementation!of!the!READ!model!at!my!site!of!practice.!


! 20!

2.1! Community#arts##

In!general!terms,!the!field!of!community!arts!encapsulates!the!diverse!range!of!arts*based!

collaborative!practices!run!by,!or!with,!communities!in!partnership!with!professional!artists!and!arts!

workers!(MacDowall!et!al.,%2013;!Australia!Council!for!the!Arts,!2011).10!!The!field!emerged!out!of!

the!radical,!socially!and!politically!driven!community!arts!movement!in!the!United!Kingdom!in!the!

1960s,!and!has!since!developed!to!encompass!a!broad!range!of!practices!from!grass*roots!political!

arts!activism!(sometimes!referred!to!as!the!movement!for!cultural!democracy)11!to!more!

institutionalised!activities!such!as!community!arts!and!cultural!development!(CACD)!(Van!Erven,!

2001;!Hawkins,!1993).!!

Community!arts!is!widely!recognised!as!a!reflexive!approach!for!community!cultural!development!

and!a!resource!for!previously!unrepresented!or!marginalised!groups!to!express!cultural!self*

determination!(Hawkins,!1993).+Leading!American!community!arts!scholar!Arlene!Goldbard!(1993,!

p.23)!identifies!two!uniting!principles!underlying!community!arts!practice:!

The!first!essential!point!of!definition!is!that!community!arts!practice!is!based!on!the!belief!that!

cultural!meaning,!expression!and!creativity!reside!within!a!community,!that!the!community!

artist's!task!is!to!assist!people!in!freeing!their!imaginations!and!giving!form!to!their!

creativity…The!second!essential!point!of!definition!is!that!collaboration!between!artists!and!

others!is!central!and!necessary!to!the!practice!of!community!arts.!

While!consensus!exists!in!the!literature!that!the!prioritisation!of!participatory!processes!of!art!

creation!are!central!to!community!arts!practice,!Badham!points!out!however,!that!‘(d)espite!a!more!

than!thirty*year!history!of!community!arts!in!Australia,!practitioners!engaging!in!transformative!

practice,!and!government!support!for!projects,!there!is!no!definitive!understanding!or!

comprehensive!theory’!within!the!field!(Badham,!2010,!p.85*6).!!!

2.1.1! Community#arts#and#cultural#development#(CACD)#

In!the!Australian!context,!the!key!shifts!within!community!arts!over!the!past!four!decades!have!been!

strongly!tied!to!the!graduations!in!the!rhetoric,!funding!priorities!and!policy!of!the!Australia!Council!

for!the!Arts,!Australia’s!principal!arts!funding!and!advisory!body!(Hadley!&!Gattenhof,%2012;!

Mulligan!&!Smith,!2009;!Van!Erven,!2001;!Hawkins,!1993).!In!particular,!community!arts!practice!has!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

10!Terminology!in!the!literature!varies,!and!alternative!terms!for!‘community!arts’!are!widely!applied,!such!as!‘community*

based!arts’!and!‘socially*engaged!arts’.!!In!some!cases,!variations!reflect!international!differences!in!rhetoric,!or!

practitioner!preferences.!For!the!purpose!of!this!scan,!such!terms!have!been!considered!to!be!comparable!at!the!general!

level!of!analysis.!While!it!is!beyond!the!scope!of!this!review,!it!is!important!to!note!however!that!subtle!differences!and!

nuances!do!exist!between!the!alternative!terms!when!analysed!in!detail.!11!Sourced!in!Lippard!(1984,!p.341)!with!reference!to!Arlene!Goldbard!and!Don!Adams,!‘From!the!Ground!Up:!Cultural!

Democracy!as!a!National!Movement’,!Upfront,!no.!8!(Winter!1983*4):!6.!


! 21!

been!significantly!shaped!by!government*led!CACD!objectives,!formerly!referred!to!as!Community!

Cultural!Development!(CCD).!From!the!mid!1970s,!CCD!became!a!formal!part!of!the!Australia!Council!

for!the!Arts!agenda,!established!to!support!community!arts!projects!working!to!achieve!cultural!

development!and/or!innovative!artistic!outcomes!(Australia!Council!for!the!Arts,!2011).!!

Practitioner!approaches!to!CCD!(and!CACD)!vary!significantly!and!have!been!influenced!over!time!by!

external!factors!such!as!technological!developments,!government!funding!priorities!and!changing!

community!expectations!(MacDowall!et!al.,!2013;!Goldbard,!2006).!The!approaches!and!methods!

documented!in!the!literature!for!community!engagement!and!art*making!practice!in!this!sector!are!

extremely!diverse.!Mulligan!and!Smith!(2009)!define!three!generations!of!CCD!practitioners!shaping!

community!arts!in!Australia:!

There!are!probably!three!generations!of!CCD!practitioners!who!are!still!active:!those!who!

began!their!work!in!the!1970s!or!who!are!heavily!influenced!by!social!and!political!movement!

theory!that!emerged!in!the!1970s;!those!who!were!able!to!take!advantage!of!new!

professional!opportunities!for!CCD!practice!that!emerged!in!the!latter!part!of!the!1980s!and!

early!1990s;!and!those!who!have!taken!advantage!of!new!media!and!new!technologies!that!

have!emerged!within!the!last!decade.!(Mulligan!&!Smith,!2009,!p.12)!

One!recent!paradigm!shift,!led!by!the!Australia!Council!for!the!Arts,!gave!rise!to!‘community!

partnerships’!emerging!as!another!key!sector!within!the!field.12!With!a!focus!on!creative!enterprise,!

sustainability!and!revitalisation!of!the!Arts,!the!CCD!priorities!of!the!1980s!(to!support!socially!

marginal!communities)!were!replaced!by!the!objective!to!move!community!arts!from!solely!

traditional!government!subsidised!funding!models!to!increasingly!shared!partnership!investment!

between!government,!business!and!communities!(Hadley!&!Gattenhof,!2012).!

2.1.2! The#rise#of#community#partnerships#

Following!a!major!scoping!report!in!2006!(see!Dunn,!2006),!the!Australia!Council!for!the!Arts!in!2010!

released!a!three*year!strategic!plan!for!the!community!partnerships!sector!to!better!support!artistic!

practice!and!cultural!engagement!with!a!diversity!of!communities!(Australia!Council!for!the!Arts,!

2011).13!The!rationale!for!this!was!that!‘partnerships!between!the!arts!and!non*arts!sectors!should!

advance!positive!outcomes!for!building!culturally!vibrant!communities!and!provide!working!

opportunities!for!artists’!(Dunn,!2006,!p.19).!Commencing!in!2008,!one!of!the!key!instruments!

employed!by!the!Australia!Council!for!the!Arts!for!advancing!community!arts!and!cultural!

development!has!been!the!Creative!Communities!Partnership!Initiative!(CCPI)!grant!scheme.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

12!The!Australia!Council!for!the!Arts!uses!both!‘creative!partnerships’!and!‘community!partnerships’!as!interchangeable!

terms!describing!this!agenda!(Hadley!&!Gattenhof,%2012,!p.8).!13!Community!refers!to!‘any!group!of!people!that!identify!with!each!other!through!commonalities!such!as!geographical!

location,!shared!cultural!heritage,!age,!professional,!social!or!recreational!interests’!(Dunn,!2006,!p.39).!


! 22!

The!CCPI!operated!as!a!competitive!funding!program!to!provide!significant!and!sustained!funding!

over!a!maximum!of!a!three*year!period!to!partnership!projects!that!aimed!to!deliver!increased!

participation!in!arts!and!cultural!activities,!encourage!innovation,!and!enhance!community!wellbeing!

in!a!number!of!government*led!priority!areas.!The!premise!underpinning!the!CCPI!is!that!deliberate!

cooperative!partnerships!between!arts!and!non*arts!organisations!have!the!potential!to!produce!a!

range!of!benefits!across!the!communities!they!work!with.!!Badham!(2010.!p.93)!elaborates:!

The!new!focus!on!community!partnerships!reflects!a!move!away!from!short*term!‘one*off’!

projects.!There!are!now!new!collaboration!models,!inter*sectoral!partnerships!including!non*

arts!organisations,!new!entrepreneurial!models!with!diversified!and!self!generated!revenue!

sources,!and!longer!term!projects!embedded!in!community!in!order!to!build!local!capacity.!!!

Though!still!aligned!with!many!of!the!principles!of!the!earlier!community!arts!movement,!the!shift!

towards!community!partnerships!reflected!the!influence!of!broader!trends!in!government!policy!and!

industry!practice!during!the!2000s.!In!the!community!partnerships!space,!arts!projects!that!deliver!

and!substantiate!an!‘exchange!of!benefits’!between!the!organisations,!communities!and!individuals!

working!together!to!deliver!positive!social!and!cultural!impacts!are!prioritised!and!funded!(Australia!

Business!Arts!Foundation!&!Australia!Council!for!the!Arts,!2010,!p.1;!MacDowall!et!al.,%2013).!!

While!the!rise!of!community!partnerships!has!been!a!prominent!trend!in!community!arts!funding!

over!the!past!decade,!future!trends!are!less!certain.!In!2014,!the!Australia!Council!for!the!Arts!

announced!its!most!significant!changes!in!funding!in!the!organisation’s!40*year!history,!introducing!a!

streamlined!grants!scheme!and!revised!peer!assessment!process.!The!changes!aimed!to!deliver!a!

simpler,!more!transparent,!and!accessible!grant!application!and!approval!process!for!artists!and!arts!

organisations*!!and!minimised!their!previous!140!categories!to!five!(Australia!Council!for!the!Arts,!

2015).!The!new!funding!model!was!the!result!of!a!two*year!reform!and!consultation!process,!and!

represents!the!most!recent!major!shift!for!how!the!Australia!Council!invests!in!Australian!arts.!While!

community!arts!and!cultural!development!(including!community!partnerships)!remains!a!key!

strategic!goal!for!the!organisation,!previously!dedicated!funding!streams!such!as!the!CCPI!have!been!

disbanded.!At!the!time!of!writing,!the!CACD!and!community!partnerships!sectors!also!face!new!

pressures!(and!uncertainty)!from!the!potential!funding!impacts!of!the!2015*16!Australian!

Government!budget!measures!to!reduce!Australia!Council’s!overall!funding.!

! !


! 23!

2.2! Evaluation#

Determining!the!benefits!of!creative!partnership!exchanges!and!the!impact!of!community!arts!

practice!(cultural,!social,!aesthetic,!economic,!environmental)!demands!that!the!processes,!outputs!

and/or!outcomes!be!articulated!and!valued.!With!no!national!standard!or!peer!agreed!framework!

catering!to!the!breadth!and!complexity!of!community!arts,!practitioners!must!traverse!multiple!

philosophical!and!technical!challenges!when!evaluating!their!processes!and!creative!works!

(MacDowall!et!al.,!2013;!Goldbard,!2006).!To!understand!the!trends!and!challenges!affecting!

community!arts!evaluation,!it!is!necessary!to!first!examine!the!broader!developments!in!application!

and!theory!of!evaluation!practice!more!generally.!

The!Australasian!Evaluation!Society!defines!evaluation!as!encompassing!‘the!systematic!collection!

and!analysis!of!information!to!make!judgments,!usually!about!the!effectiveness,!efficiency!and/or!

appropriateness!of!an!activity’!(Australasian!Evaluation!Society,!2010,!p.3).!While!definitions!vary!

within!the!literature,!most!encapsulate!a!cyclic!process!with!distinct!stages!of!planning!and!decision*

making;!methodology!design;!data!collection;!organisation!of!findings;!evaluative!reasoning!and!

assessment!presentation;!and!reporting!(Davidson,!2005;!Clarke,!1999;!Matarasso,!1996;!Wilde!&!

Sockey,!1995).!Some!scholars!in!evaluation,!such!as!Scriven!(2012)!and!Davidson!(2005),!argue!that!

central!to!the!evaluator’s!work!is!the!systematic!determination!of!merit!or!worth.!

Evidence!of!formal!evaluation!practice!dates!back!over!two!thousand!years!(Fitzpatrick!et!al.,%2012).!

Despite!this!long!history,!the!majority!of!theoretical!developments!and!innovations!in!application!

have!occurred!in!the!last!sixty!years!(Scriven,!2012;!Sharp,!2003).!Influential!in!the!early!scholarship!

in!evaluation!were!scholars!such!as!Scriven!(1967),!Stake!(1967)!and!Weiss!(1972).!Since!then,!

scholarship!in!evaluation!has!expanded!significantly,!amounting!to!a!vast!and!sophisticated!body!of!

literature.!Considered!and!debated!within!the!literature!are!divergent!epistemologies!of!practice!

relating!to!evaluation!paradigms,!design,!methods,!and!strategies!for!stakeholder!engagement!and!

partnership!brokerage!(Fitzpatrick!et!al.,%2012;!Hadley!&!Gattenhof,!2012;!Friedman,!2008;!Owen!&!

Rogers,!1999).!The!breadth!of!evaluation!discourse!is!evident!in!compendiums!such!as!Program%

evaluation:%Alternative%approaches%and%practical%guidelines!(Fitzpatrick!et!al.,%2012),!Handbook%of%

Evaluation!(Shaw,!I.,!Shaw,!I.G.R,!Greene,!&!Mark,!2006),!and!the!Encyclopedia%of%Evaluation!

(Mathison,!2005).!

Historical!accounts!attribute!the!widespread!application!of!evaluation!across!disciplines!and!

countries!to!an!increased!demand!by!policymakers,!managers,!consumers!and!the!general!public!for!

improved!accountability,!efficacy,!and!understanding!of!social!problems!(Fitzpatrick!et!al.,%2012;!


! 24!

Mark,!Henry,!&!Julnes,!2000).!Since!the!1990s,!growth!in!the!field!has!also!been!linked!with!trends!

such!as!the!internationalisation!and!professionalisation!of!evaluation!(Fitzpatrick!et!al.,%2012;!House,!

1993).!Documented!in!the!literature!are!increased!numbers!of!professional!evaluation!organisations!

(growing!from!a!handful!in!the!1980s!to!being!established!in!approximately!100!countries!in!2011)!

and!published!scholarly!and!industry!work.!Schwandt!(2009)!cites!research!by!Dahler*Larsen!

estimating!10,000!individuals!at!least!nominally!self*identify!as!evaluators!and!belong!to!one!or!

more!of!the!many!national!and!international!evaluation!associations!around!the!world.!Online,!!a!

diversity!of!evaluation*specific!networks,!resource!guides!and!blogs!are!available!(Scriven,!2012).14!

2.2.1! Program#evaluation#

Within!the!field!of!evaluation,!the!domain!of!relevance!for!this!research!is!program!evaluation.!A!

program!is!defined!as!a!‘group!of!planned!activities!that!contribute!to!a!common!strategic!objective’!

(Australian!Government,!2000,!p.4).!Program!evaluation!is!used!to!ascertain!program!impact,!worth,!

efficacy!and!efficiency!for!the!purpose!of!continuous!improvement,!decision*making,!accountability,!

partnership!development,!and!program!acquittal!(Fitzpatrick!et!al.,!2012;!Davidson,!2005;!Wilde!&!

Sockey,!1995).!!

Program!evaluation!has!its!origins!in!Education!in!the!1950s,!linked!with!the!rise!of!the!standards*

based!education!movement!and!social!experimentation!and!research!(Fitzpatrick!et!al.,%2012).!By!the!

end!of!the!1950s!evaluation!began!to!flourish,!linked!with!the!increase!in!social!research,!and!

systematic!approaches!were!being!developed!using!scientific!experimental!methods!to!rationalise!

the!evaluative!process.!From!the!1950s!to!the!1970s!in!countries!such!as!the!United!States,!Sweden!

and!Germany,!social!inquiry!laid!the!foundations!for!the!‘first!wave’!of!modern!program!evaluation.!

Evaluation!practice!in!this!context!was!linked!to!program!improvement!and!social!experimentation!

to!test!new!interventions!(Fitzpatrick!et!al.,%2012,!p.!50;!Rist,!1999).!In!other!countries,!such!as!the!

United!Kingdom,!a!second!wave!of!evaluation!is!documented!as!being!driven!by!the!rise!in!demand!

for!public!sector!accountability!and!measuring!program!efficacy!(Fitzpatrick!et!al.,%2012).!In!the!

Australian!context,!program!evaluation!(such!as!in!health!and!community!development)!became!a!

major!government!commitment!during!the!1970s!and!has!since!remained!a!priority!(Sharp,!2003).!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

14!Some!examples!of!online!resources/hubs!include!American!Evaluation!Association!

365!|!A!Tip*a*Day!by!and!for!evaluators!http://aea365.org/!;!IDEAS!International!Development!Evaluation!Association!

http://ideas*global.org!;!BetterEvaluation!website!http://betterevaluation.org/!;!Community!Sustainability!Toolbox!

http://evaluationtoolbox.net.au!;!Does!Your!Project!Make!a!Difference?!

http://www.environment.nsw.gov.au/resources/community/projecteval04110.pdf!.!

!


! 25!

Christie!and!Alkin!(2008)!argue!that!such!drives!for!increased!social!inquiry!and!fiscal!control/!

accountability,!constitute!the!central!roots!of!modern!evaluation.!!

The!epistemological!undercurrents!of!program!evaluation!encompass!positivist,!methods*focused!

approaches!(Weis,!1972;!Campbell,!Stanley,!&!Gage,!1963),!to!post*positivist!paradigms!prioritising!

realist!evaluation!(Davidson,!2005;!Scriven,!2003a),!to!models!espousing!the!philosophy!of!

subjectivity!where!it!is!argued!that!there!is!no!one!objective!truth!(Greene,!2005;!Guba!&!Lincoln,!

1985).!15!!These!theoretical!domains!are!described!and!classified!in!different!ways!by!scholars!in!the!

literature.!For!example,!in!theorising!the!different!branches!of!evaluation!Christie!and!Alkin!(2008)!

map!theorists!across!their!tree!of!evaluation!in!three!predominant!orientations!*!valuing,!methods!

and!utility!(the!branches).!Guba!and!Lincoln!(2004)!classify!the!theoretical!developments!by!defining!

distinct!‘generations’!of!program!evaluation.!The!most!recent!is!fourth!generation!evaluation!(FGE)!

based!on!an!epistemology!of!constructivism!based!on!a!radical!critique!of!the!positivist!foundation!

of!traditional!program!evaluation!(Lay!&!Papadopoulos,!2007;!Guba!&!Lincoln,!2004).!

The!wide*ranging!theoretical!orientations,!based!on!the!differing!philosophical!and!ideological!

stances,!have!a!profound!impact!on!the!methodological!preferences!and!guiding!approach!adopted!

by!program!evaluators.!They!drive!what!is!prioritised!and!practiced!at!the!site!of!practice.!Fitzpatrick!

et!al.,!(2012)!for!example,!identifies!four!dominant!evaluation!approaches:!!

•! expertise!and!consumer*oriented!(performance,!cost*analysis)!

•! program*oriented!(logic!models,!theory*based!evaluation,!goal*free)!

•! decision*oriented!(impact,!results*based,!utility!focussed!evaluation)!

•! participant*oriented!(stakeholder,!empowerment,!participatory!evaluation).!

Approaches!to!program!evaluation!are!nuanced,!and!the!emergent!models!of!practice!become!

specialised,!contextualised!and!politicised!based!on!the!theoretical!concepts!guiding!the!approach!

(Edenloff,!2011;!Davidson,!2005).!!

! !

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

15!Some!examples!of!the!interpretative!positions!in!evaluation!include!postpositivism,!constructivism,!transformative!and!

pragmatist!paradigms,!with!shifts!mirroring!trends!and!debates!in!social!research!including!both!quantitative!and!

qualitative!stances!(Fitzpatrick!et!al.,!2012,!p.124;!for!analysis!of!social!research!trends!see!Denzin!&!Lincoln,!2011;!Cresswell,!&!Clark,!2007;!Schwandt,!2000;!Alvesson!&!Skoldberg,!1999;!Becker,!1996;!Babbie,!1992).!

!


! 26!

Alkin!(2004)!refers!to!the!plethora!of!evaluation!approaches!and!models!shaping!evaluation!practice!

since!the!1950s!as!prescriptive!theories.!Fitzpatrick!et!al.!(2012,!p.112)!argue!‘(t)he!various!

approaches,!or!theories,!proposed!by!evaluators!make!up!the!content!of!the!field!of!evaluation’!and!

are!testament!to!the!dynamism!involved!in!designing!and!conducting!evaluation.!Consider!for!

example:!Utilisation*Focused!evaluation!(Patton,!2008);!Naturalistic!evaluation!(Guba!&!Lincoln,!

1981);!Empowerment!evaluation!(Fetterman!&!Wandersman,!2005);!Participatory!evaluation!

(Cousins!&!Whitmore,!1998);!and!Feminist!evaluation,!with!its!focus!on!the!gender!inequalities!that!

lead!to!social!injustice!(Hood,!2002;!Sielbeck*Bowen,!Brisolara,!Seigart,!Tischler,!&!Whitmore,!2002).!

Practitioner!guides!and!frameworks!include!the!Key!Evaluation!Checklist!(Scriven,!1991);!

BetterEvaluation!Rainbow!Framework!(RMIT!&!partners*!website);!Real/!Actionable!Evaluation!

(Davidson,!2010);!and!systems!approach!to!program!evaluation!(Darabi,!2002).!!

Program!evaluation!has!also!been!demarcated!and!theorised!in!the!literature!on!the!basis!of!the!

timing!of!implementation!in!relation!to!the!program!being!evaluated.!Scriven!(1967)!was!the!first!to!

identify!that!depending!on!when!an!evaluation!is!undertaken!in!a!program!or!project!cycle!different!

evaluative!functions!may!be!achieved.!Irrespective!of!the!type!of!approach!adopted!or!methods!

employed,!evaluation!may!be!defined!as!being!(or!a!mix!of):!preformative!(prior!to!commencement);!

formative!(during!the!early!stages!of!implementation);!developmental!(ongoing!or!for!uncertain!

and/or!innovation*oriented!contexts);!summative!(once!activity!is!completed);!or!ascriptive!(for!the!

sake!of!the!knowledge!gained)!(Scriven!2012;!Davidson,!2005).!!

2.2.2! ParticipationSoriented#approaches#to#program#evaluation#

Within!the!domain!of!program!evaluation,!it!is!the!participation*oriented!(or!participatory)!

approaches!that!best!align!with!the!CACD!principles!governing!my!site!of!practice,!which!include!

participation,!representation,!democracy,!and!inclusion!(common!attributes!of!a!community!arts!

ethos).!!Participation*oriented!evaluation!is!defined!by!the!level!of!engagement!and!contribution!by!

participants!and!partners!in!the!process,!and!encompasses!a!wide!spectrum!of!applications.!In!the!

literature,!there!are!many!interrelated!models!described!as!participatory!approaches!such!as!

empowerment,!stakeholder*oriented,!action,!and!transformative!evaluation.!What!unites!these!

approaches!is!the!participatory!spirit!of!how!evaluation!is!designed!and!undertaken.!!

Butterfoss,!Francisco,!and!Capwell!(2001,!p.115)!argue!that!evaluation!is!participatory!when!

‘evaluators!collaborate!in!some!way!with!individuals,!groups,!or!communities!who!have!a!stake!in!

the!program!or!project!that!is!being!evaluated’.!To!varying!degrees,!partners!get!involved!or!

collaborate!in!the!formulation!of!evaluation!questions,!goals,!approaches,!as!well!as!with!the!


! 27!

delivery!and!implementation.!Evaluators!actively!draw!on!the!knowledge,!skills,!perspectives!or!

participation!of!partners!and!stakeholders!during!the!evaluation!process.16!The!parameters!for!

participatory!practice!are!vast!and!flexible.!

Over!the!past!forty!years,!the!orientation!towards!stakeholder!and!partner!participation!in!

evaluation!has!developed!and!expanded!significantly.!The!origins!of!the!genre!date!back!to!the!

1960s,!when!reaction!to!the!then!dominant!social!science!research!methods!applied!in!the!field!of!

Education,!catalysed!the!turn!to!participation!(Fitzpatrick!et!al.,!2012).!Robert!Stake!(1967)!is!cited!as!

the!first!theorist!to!lay!claim!to!a!participation*oriented!methodology,!when!he!introduced!a!suite!of!

responsive,!qualitative!and!pluralist!approaches!in!his!work!on!Education!evaluation!(Fitzpatrick!et!

al.,!2012).!Another!Influential!era!was!in!the!1980s,!when!Guba!and!Lincoln’s!(1985)!naturalistic!

approach!asserted!a!move!from!quantitative!to!qualitative!methods!such!as!interviews,!observations!

and!case!studies.!Underpinning!the!formative!development!of!participation*oriented!approaches,!

was!the!emergent!position!that!evaluation!needed!to!extend!beyond!measurement,!description!and!

judgment*!to!develop!methodologies!that!cater!for!the!multiple!realities!and!stakeholder!views!

needed!to!produce!authentic!and!representative!findings!(Guijt,!Brouwers,!Kusters,!Prins,!&!

Zeynalova,!2011).!!

Prominent!movements!within!the!participatory!approaches!since!the!1980s!include!Practical!

Participatory!Evaluation!(Cousins!&!Earl,!1995),!Developmental!Evaluation!(Patton,!1994,!2010),!

Empowerment!Evaluation!(Fetterman!&!Wandersman,!2005),!and!Deliberative!Democratic!

Evaluation!(House!&!Howe,!2000).!The!differences!between!participatory!approaches!can!be!defined!

by!applied!elements!such!as!the!control!of!the!evaluation!process!offered!to!stakeholders!via!

participant!ownership,!stakeholder!selection!processes,!depth!of!participation,!and!the!role!played!

by!the!professional!evaluator!in!designing!and!implementing!evaluation!(Fitzpatrick!et!al.,!2012,!

Moreau!&!Cousins,!2012;!Dunst,!Trivette,!&!LaPointe,!1992).!While!the!breadth!of!participatory!

rhetoric!and!application!in!the!field!is!vast,!the!quality,!impact!and!authenticity!of!participation*

oriented!evaluation!practice!varies.!Echoed!across!the!literature!is!the!cautionary!reminder:!

‘Collaboration!is!not!a!guarantee!of!fairness!any!more!than!quantitative!methods!were!once!thought!

to!ensure!objectivity’!(Sielbeck*Bowen!et!al.,!2002,!p.6).!

! !

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

16!Related!theories!of!participatory!practice!and!participatory!research!more!broadly!are!also!relevant!(see!Wilcox,!1996;!

Wadsworth,!1998;!Lennie,!2006;!Jackson!et!al.,!2003)!and!community!of!practice!(Lave!&!Wenger,!1991;!Wenger,!

McDermott,!&!Snyder.,!2002;!Hildreth!&!Kimble,!2004;!Cox,!2005).!


! 28!

2.3! Community#arts#evaluation#

While!evaluation!practice!in!community!arts!has!been!influenced!by!broader!trends!for!program!

evaluation!in!Australia!during!the!1970s!(such!as!in!health!and!community!development),!it!was!not!

until!the!1990s!that!evaluation!practice!became!prominent!within!the!community!arts!field!

(MacDowall,!Badham,!Blomkamp,!&!Dunphy,!2015).!Prior!to!this!time,!during!the!1970s!and!1980s,!

community!arts!programs!did!not!‘depend!on!detailed!post!hoc!analysis’!and!were!delivered!

‘without!any!particular!managerial,!administrative!or!bureaucratic!pressure!to!provide!facts,!figures!

and!information!after!the!activity’!(Hadley!&!Gattenhof,%2012,!p.5).!The!rise!of!evaluation!practice!in!

community!arts!reflects!a!paradigm!shift!within!the!field,!from!the!mid!1990s,!where!managerial!and!

entrepreneurial!approaches!were!increasingly!integrated!into!programs!and!practice!(MacDowall!et!

al.,!2013).!!

The!proliferation!of!evaluation!in!the!arts!is!evidenced!by!an!increase!in:!funder!stipulation!for!

evaluation;!arts*related!evaluation!conferences,!professional!cohorts,!and!associations;!evaluation!

consultancy;!and!technical!‘How!to’!guides,!tool!boxes,!training!and!resources!aimed!at!developing!

professional!knowledge,!improving!capacity!and!increasing!evaluation!uptake!by!CACD!practitioners!

(Mulligan!&!Smith,!2009;!Jackson,!Harranz,!&!Kabwasa*Green,!2003;!Owen!&!Rogers,!1999;!Williams,!

1997;!Hawe,!Degeling,!&!Hall,!1990).!While!evaluation!in!this!field!is!burgeoning,!the!role!and!

relevance!of!formal!evaluation!in!community!arts!remains!contentious!and!is!the!subject!of!much!

scholarly!and!practitioner!debate.!!Central!are!philosophical!disputes!about!the!notion!of!cultural!

value,!that!is,!how!the!value!and!impact!of!community!arts!is!conceptualised,!defined!and!measured!

(if!at!all).!Related,!is!a!technical!debate!about!the!tools!for!evaluation!and!measurement!*!those!

methodological!approaches!that!can!produce!credible,!appropriate!and!reliable!means!to!analyse,!

evaluate!and!report!on!the!outcomes!and!impact!community!arts!(MacDowall!et!al.,!2015).!!

The!next!section!explores!the!trends!in!community!arts!evaluation!that!have!been!influential!for!this!

research,!particularly!the!epistemological!and!technical!dimensions!of!evaluation!practice!in!this!

context.!In!the!community!partnership!domain!of!community!arts,!where!creative!initiatives!involve!

multiple!and!culturally!diverse!partners,!establishing!common!ground!for!notions!of!cultural!value,!

arts!impact!and!evaluation!practice!presents!challenges.!For!example,!MacDowall!(2015,!p.4)!states!

that!‘increasing!partnerships!between!arts!and!non*arts!agencies!can!mean!differing!frameworks!for!

evaluation,!with!divergent!assumptions!about!success!and!measurement’.!The!following!discussion!

provides!the!contextual!backdrop!for!later!discussions!in!the!dissertation,!on!the!conceptualisations!

and!evaluation!needs!identified!at!the!Nuclear!Futures!site!of!practice!that!have!informed!the!design!

and!development!of!the!READ!model!(see!Chapters!5!and!6).!


! 29!

2.3.1! Trends#in#community#arts#evaluation#

Arguably!one!of!the!most!dominant!and!influential!trends!identified!in!the!community!arts!literature!

of!relevance!to!this!research!is!the!instrumentalisation!of!the!arts!over!the!past!40!years.!MacDowall!

et!al.!(2013,!p.4)!describes!the!trend!as!being!‘signalled!by!a!range!of!social!uses!for!art,!and!the!

interest!from!non*art!agencies!and!government!departments!across!all!public!policy!areas,!from!

health!and!urban!renewal,!to!crime!prevention!and!anti*discrimination’.!Resulting,!is!an!increased!

focus!and!resource!allocation!to!the!articulation!and!measurement!of!impact!of!the!social,!cultural,!

economic!and!environmental!benefits!generated!by!community!arts!practice!and!activities!

(Goldbard,!2008,!Mills!&!Brown,!2004;!Keating,!2002;!Matarasso,!1997,!1996).!As!well!as!framing!

contemporary!policy,!arts!practice,!and!partnership!development!in!the!field,!the!era!of!

instrumentalisation!has!influenced!evaluation!practice!(Belfiore,!2014;!Hadley!&!Gattenhof,%2012).!!

Linked!to!this!is!the!more!recent!global!movement!towards!cultural!measurement,!since!the!late!

twentieth!century,!evidenced!by!the!growing!interest!by!governments!worldwide!to!establish!

frameworks!that!quantify!cultural!value!and!progress!using!tools!such!as!indicators!and!metrics!

(MacDowall!et!al.,!2015).!These!trends!fit!within!the!broader!movement!of!global!progress!

measurement,!of!which!culture!is!now!considered!to!be!key!domain!(Woolcock!&!Davern,!2015).!

MacDowall!(2015,!p.2)!defines!cultural!measurement!as!covering!‘a!range!of!systems!for!measuring!

aspects!of!culture,!from!quantified!cultural!indicators!that!lend!themselves!to!comparison!and!

analysis!over!time,!to!qualitative!measures!of!cultural!value!and!techniques!for!the!evaluation!of!arts!

and!cultural!programmes’.!Such!practices!have!been!adopted!by!countries!worldwide,!including!the!

United!Kingdom!and!the!United!States!of!America,!and!are!gaining!increasing!traction!in!Australia!

(Badham,!2015;!Blomkamp,!2015;!Goldbard,!2015).!One!Australian!example!of!the!application!of!

cultural!metrics!is!the!Community!Indicators!Victoria!(CIV)!framework!arising!from!the!Victorian!

Community!Indicators!Project!commissioned!by!VicHealth!in!2005*6,!which!is!used!as!a!state*wide!

system!for!measuring!community!well*being!(MacDowall,!2015;!Woolcock!&!Davern,!2015).17!!

The!interest!in!cultural!measurement!aligns!with!historical!trends!in!evaluation!and!public!policy,!

driven!by!the!demand!for!evidence*based!policy!and!management!by!governments!as!a!means!for!

improving!the!allocation!and!accountability!of!public!funding!and!expenditure!(MacDowall!et!al.,!

2015;!Christie!&!Alkin,!2008).!MacDowall!(2015,!p.1)!asserts!that!‘culture!and!cultural!development!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

17!This!framework!utilises!statics!about!arts!participation!as!a!cultural!indicator!in!the!context!of!community!wellbeing!and!

local!government,!where!culture!is!one!of!five!key!domains.!While!the!cultural!indicators!included!in!the!framework!

constitute!only!a!small!part!of!the!framework,!the!inclusion!of!such!metrics!in!this!type!of!scheme!is!significant!in!the!

Australian!context!(MacDowall,!2015;!Woolcock!&!Davern,!2015).!Accessible!at:!http://www.communityindicators.net.au!!


! 30!

have!become!internationally!recognised!as!important!dimensions!of!contemporary!governance!and!

public!policy’.!In!the!arts,!cultural!measurement!is!being!established!for!a!number!of!purposes,!

including!to!quantify!the!impact!of!arts!and!cultural!policies,!activities,!and!organisations;!to!monitor!

progress!or!cultural!vibrancy!of!communities!and!nations!across!time;!and!to!calculate!the!long*term!

and/or!cross*sectoral!benefits!and/or!costs!being!generated!through!investment!in!the!arts!(related!

to!community!development,!economic,!social!well*being,!health,!and!sustainability!benefits)!

(MacDowall,!2015).!In!order!to!undertake!these!types!of!measurement,!criteria!for!cultural!value!

and!impact!need!to!be!determined,!and!this!remains!a!complex!and!controversial!area!of!evaluation!

in!the!arts.!

Traditionally,!the!notion!of!cultural!value!has!been!contested!through!a!binary!lens!where!

discussions!centred!on!the!value!of!the!arts!as!being!intrinsic!(often!linked!with!inherent!and!

aesthetic!worth)!or!instrumental!(often!associated!with!economic!or!social!outcomes)!(Goldbard,!

2008;!Geursen!&!Rentschler,!2003).!This!has!shifted!in!recent!years!and!the!contemporary!discourse!

on!cultural!value!encompasses!much!wider!terms!of!reference.!Blomkamp!(2015)!for!example,!

proposes!a!spectrum!of!four!dimensions!of!cultural!value:!individual,!collective,!commodity!and!

process,!which!can!be!measured!via!cultural!indicators!for!‘economic,!human!rights,!ecologic,!

symbolic,!artistic!and!social!policy’!impact/worth!(Blomkamp,!2015!p.12).!Dunphy’s!(2015)!‘holistic’!

model!for!arts!engagement!is!another!example!of!a!broadened!approach,!drawing!on!frameworks!of!

community!wellbeing!and!sustainability!across!‘personal!wellbeing,!cultural,!social,!economic,!civic!

and!ecological’!domains!(Dunphy,!2015,!p.243).!!

The!trend!towards!the!categorisation!and!quantification!of!cultural!value!can!be!attributed!to!both!

practical!and!symbolic!reasons,!including!that!metrics!hold!the!promise!of!addressing!reoccurring!

issues!within!arts!evaluation!by!turning!what!was!once!intangible,!into!quantitative,!evidence*based!

knowledge!(Blomkamp,!2015;!MacDowall,!2015).!While!international!benchmarks!and!cultural!

indicators!potentially!allow!for!comparative!analysis!and!progress!measurement,!questions!are!

raised!in!the!literature!about!the!efficacy,!politics,!suitability,!and!the!long*term!ramifications!of!

cultural!measurement!in!the!community!arts!context!(MacDowall!et!al.,!2015;!Belfiore,!2014;!

Goldbard,!2008).!Some!scholars!argue,!for!example,!that!the!pursuit!of!cultural!metrics!and!

valuation!is!ineffectual,!and!that!serious!technical!and!methodological!questions!remain,!as!well!as!

persistent!questions!on!the!reliability!of!cultural!data!(Goldbard,!2015;!MacDowall,!2015).!Others!

assert!a!reliance!on!indicators!can!create!limitations!to!focus!on!what!can!be!measured,!as!opposed!

to!what!should!be!measured!(Blomkamp,!2015).!MacDowall!(2015.!p.1)!further!cautions!that!


! 31!

systems!of!measurement!can!‘create!rather!than!capture!cultural!value’!and!often!neglects!the!

context!of!the!measurement.!

In!the!broader!discourse!framing!community!arts!evaluation,!there!are!also!ethical,!philosophical!

and!political!questions!raised!on!the!legitimacy!of!the!notion!of!cultural!value!and!measurement.!

Parsons!(2015,!p.53)!argues!that!‘defining!cultural!value!is!not!a!question!of!finding!the!right!

“system”!but!resolving!inherent!flaws!in!the!terms!of!the!debate’.!What!has!ensued!is!an!

epistemological!debate!in!the!field!on!cultural!value!and!the!impact!agenda.!!

Countering!the!instrumentalisation!and!measurement!trend,!are!practitioners!and!scholars!

expressing!trepidation!towards!empirical!evaluation!in!the!community!arts!context!(Belfiore,!2014;!

Jackson,!2009;!Jennings!&!Baldwin,!2010;!Newman!et!al.,!2003).!This!is!evident!in!the!peer*reviewed!

literature!as!well!as!in!the!cross*sectoral!discourse!on!academic!led!blogs!and!professional!forums!

(Donovan,!2012;!Selwood,!2002).!One!of!the!lead!advocates!of!this!counter!position!is!Goldbard!

(2008,!2015)!who!asserts!deep!concerns!about!the!far*reaching!impacts!of!evaluation!on!the!field,!

and!the!perpetuation!of!the!instrumentalisation!of!the!arts.!Goldbard!argues!that!the!‘Metrics!

Syndrome’!(a!term!coined!in!reference!to!her!American!context!of!arts!funding,!evaluation,!and!

cultural!measurement)!is!creating!an!irreconcilable!ideological!crisis!for!community!arts!(Goldbard,!

2015,!2008;!Matarasso,!1996):!

The!trouble!is,!the!very!quest!for!metrics!is!contaminated!with!ideas!and!assumptions!

borrowed!from!worlds!that!have!nothing!in!particular!to!do!with!community!and!creativity.!

The!notion!that!everything!of!value!can!be!weighed!and!measured,!which!is!one!of!the!most!

grotesque!artefacts!of!post*Enlightenment!thinking,!is!antithetical!to!the!deep!values!of!

community!cultural!development.!Indeed,!in!this!domain,!the!search!for!metrics!actually!

harms!what!it!seeks!to!help.!(Goldbard,!2008,!p.1)!

This!argument!is!based!on!the!premise!that!the!social!and!economic!indicators!being!used!for!

cultural!measurement!and!evaluation!in!the!arts!have!been!adopted!from!other!fields,!such!as!

economics,!and!have!inherent!rhetoric,!paradigm!values!and!power!structures!that!are!not!always!

suited!to!the!community!arts!context!(Goldbard!2015,!2008).!!

One!ramification!is!that!there!has!become!an!increased!orientation!toward!the!socio*economic!

impacts!of!the!arts!over!the!past!two!decades,!and!this!is!closely!linked!to!how!public!funding!and!

policy!decisions!are!being!rationalised!and!legitimated!(MacDowall!et!al.,!2015;!Belfiore,!2014).!

Belfiore!(2014,!p.96)!explains!that!‘discussions!of!“value”!have!become!inextricably!linked!to!the!

challenge!of!“making!the!case”!for!the!arts!to!justify!public!cultural!funding!given!increasing!funding!

pressures!facing!public!service!provision,!especially!in!the!West’.!Within!the!discourse!on!cultural!

value,!it!is!also!evident!that!the!concepts!of!value!and!impact!are!increasingly!used!interchangeably!


! 32!

(with!a!focus!on!the!measurable)!which!some!scholars!argue!has!obscured!the!debate!on!cultural!

value!and!reduced!its!scope!to!technocratic,!economic!and!instrumental!interpretations!(Goldbard,!

2015;!Belfiore,!2014).!Conceptualising!cultural!value!in!relation!to!public!value!has!implications!for!

arts!practice,!as!art*making!processes!and!creative!works!become!viewed!as!commodities!valued!for!

the!social!return!on!investment!they!deliver!(Blomkamp,!2015).!In!addition,!some!scholars!argue!

that!evaluation!and!benchmarking!of!practice!are!techniques!employed!by!government!as!a!way!of!

exercising!control!over!civil!society,!which!can!prevent!effective!community!engagement!and!

undermine!meaningful!collaboration!(Milbourne,!2009;!Larner!&!Butler,!2005;!Ilcan!&!Basok,!2004).!!

While!the!opponents!to!the!instrumentalisation!of!the!arts!and!the!intensification!of!impact!

measurement!are!not!arguing!for!a!data*free!approach!to!evaluation!in!community!arts,!they!do!

challenge!the!emergent!conventions!of!cultural!measurement!and!urge!practitioners!to!look!beyond!

a!narrow!instrumental!prescription!of!impact*oriented!evaluation!to!justify!arts!practice.!Badham!

(2015,!p.193)!argues!that!‘top*down!hegemonic!benchmarks!tend!to!disregard!the!diverse!cultural!

values!that!make!up!society’!and!advocates!a!more!‘bottom*up!approach’!to!cultural!indicators!

‘through!the!use!of!arts*based!and!participatory!methods!of!community*based!research’.!Goldbard!

(2015)!argues!for!a!shift!away!from!metrics!is!needed,!to!art!and!story*based!methods!as!a!means!to!

elucidate!the!value!of!arts!and!culture,!encouraging!practitioners!to!resist!the!imperative!to!measure!

culture.!Likewise,!Belfiore!(2014,!p.95)!pleas!‘for!a!collaborative!effort!to!resist!the!economic!doxa,!

and!to!reclaim!and!reinvent!the!impact!agenda!as!a!route!towards!the!establishment!of!new!public!

humanities’*!and!for!the!field!to!move!beyond!the!limiting!aspects!of!the!impact!debate!and!

economic!rationalist!framing!to!initiate!a!more!open!and!rich!discourse!on!cultural!value.!

2.3.2! Relevant#approaches#and#models#for#the#Nuclear#Futures#site#of#practice#

Despite!the!significant!developments!in!the!field!relating!to!the!evaluation!of!community!arts,!the!

limitations!of!the!conventional!approaches,!and!their!more!recent!quantitative!and!metrics*driven!

orientation,!are!widely!documented!(Goldbard,!2015;!Belfiore!&!Bennett,!2010;!2008).!In!the!field,!

what!remains!is!a!persistent!demand!for!tested,!effective!and!accepted!techniques!able!to!‘capture!

the!depth,!as!well!as!the!breadth,!of!the!encounter!between!communities!and!the!creative!arts’!

(Newman,!Curtis,!&!Stephens,!2003,!p.13).!This!reflects!more!general!calls!for!improved!

methodologies!in!evaluating!the!arts,!such!as!by!the!Australian!Business!Arts!Foundation!(AbaF)!

assertion!that!the!development!effective!evaluation!methodologies!is!a!key!challenge!facing!the!arts!

sector!(AbaF!&!Australia!Council!for!the!Arts,!2010).!While!it!is!beyond!the!scope!of!this!contextual!

scan!to!comprehensively!review!the!wide!spectrum!of!evaluation!methodologies!and!methods!


! 33!

currently!applied!in!community!arts,!a!brief!review!of!methods!identified!in!the!literature!as!relevant!

to!the!research!(and!the!task!of!designing!the!READ!model!for!the!site!of!practice)!is!presented.!

As!part!of!the!contextual!scan!I!examined!a!number!of!existing!models!and!frameworks!developed!

for!evaluation!practice!for!community!arts!in!Australia!and!overseas,!as!well!as!evaluation!reporting!

and!exemplars!provided!by!the!Australia!Council!for!the!Arts!and!available!in!the!public!domain.!In!

the!Australian!context,!one!notable!evaluation!framework!of!relevance!was!Arts!Victoria’s!

Evaluating%Community%Arts%and%Community%Well%Being%(Keating,!2002).!This!practical!guide!was!

developed!for!community!art!practitioners!and!organisations,!providing!a!comprehensive,!step*by*

step!guide!to!evaluating!the!process,!outcomes!and!impacts!of!community!arts!programs!(Edenloff,!

2011).!!

Also!of!interest!for!this!research,!were!models!and!examples!in!the!field!that!demonstrated!effective!

strategies!for!designing!and!implementing!evaluation!using!collaborative!processes.!As!was!

introduced!in!Section!2.1,!the!Nuclear!Futures!site!of!practice!constitutes!a!‘creative!communities!

partnership!initiative’,!a!partner*oriented!niche!of!community!arts!promoted!by!the!Australia!

Council!for!the!Arts.!The!collaborative!nature!of!the!arts!practice,!involving!artists,!communities!of!

interest,!cultural!and!academic!institutions,!government!agencies,!and!not*for*profit!organisations,!

means!that!there!are!diverse!and!numerous!stakeholders!with!an!interest!in!the!program’s!

evaluation.!For!the!evaluators!of!such!complex!programs,!the!contextual!scan!revealed,!that!several!

fundamental!questions!arise!such!as:!How!is!cultural!value!and!evaluation!conceived!and!defined!

across!the!partnerships?!Which!values,!agendas!and!questions!should!drive!evaluation!practice?!

How!is!program!and!partnership!performance,!progress,!quality,!and!success!understood!and!

represented?!What!types!of!data!and!outputs!are!needed!to!meet!partner!expectations!for!

measuring!and!evidencing!impact!or!making!summative!conclusions?!Who!within!the!partnership!

should!be!involved!to!ensure!such!processes!are!representative?!(Swan!&!Atkinson,!2012;!Badham,!

2010;!Donaldson,!Christie,!&!Mark,!2008).!As!revealed!in!Section!2.3.1,!it!is!clear!that!questions!

arising!for!evaluations!involving!multiple!and!culturally!diverse!partners!are!not!merely!technical,!

they!are!philosophical!and!political!as!well.!

Within!the!community!arts!field,!there!is!a!growing!interest!in!how!the!participation*oriented!

evaluation!approaches,!discussed!in!Section!2.2.2,!might!be!applied!to!deal!with!these!issues!

routinely!arising!in!the!community!arts!context!(MacDowall!et!al.,!2013).!Clements!(2007,!p.325)!

argues!that!the!ideal!democratic!participatory!approach!for!community!arts!evaluation!would!

encourage!‘participant!involvement,!empowerment!and!self*management’!as!compared!to!what!he!

describes!as!the!‘normative!bureaucratic!and!autocratic!approaches!steeped!in!top*down!agendas!


! 34!

which!may!exacerbate!social!problems’.!Advocates!for!a!‘bottom*up!approach’!argue!that!cultural!

value!and!credibility!of!findings!about!the!possible!impact!of!community!arts!need!to!include,!if!not!

be!wholly!determined!by,!local!knowledge,!expertise!and!ownership!(Badham,!2015;!Mercer!2005).!

Badham!(2015,!p.210)!argues!‘it!is!only!the!people!who!experience!the!community!directly!who!can!

determine!what!progress!means!locally’.!

In!the!international!context,!the!literature!review!revealed!numerous!exemplars!relevant!to!the!site!

of!practice!in!terms!of!how!participation*oriented!approaches!to!evaluation!were!successfully!

adopted!for!partnership*oriented!community!arts!projects.!One!example,!integrating!academic!and!

research!systems!for!learning!and!analysis,!was!the!participatory!evaluation!undertaken!by!the!

Chicago!Arts!Partnership!in!Education!(CAPE).!Based!in!the!USA,!CAPE!aimed!to!improve!education!

and!school!outcomes!(such!as!increased!student!academic!success,!critical!thinking!and!creativity)!

through!research*based,!arts!driven!education.!Central!to!the!CAPE!program!was!an!emphasis!on!

strong!partnership!development,!and!a!commitment!to!inquiry,!documentation,!professional!

development!and!collaborative!action!research!through!evaluation!(Burnaford,!2006).!The!approach!

adopted!by!CAPE’s!longitudinal!partners,!challenges!traditional!non*participatory!models!and!

demonstrated!a!turn!toward!a!culture!of!evidence.!Partners!and!participants!were!enabled!to!join!a!

community!of!researchers!to!collaborate!on!the!evaluation!design!and!implementation,!and!also!

contribute!to!the!generation!of!new!knowledge!in!the!field!of!Arts!Education!(Burnaford,!2006,!p.3).!

While!this!is!an!older!example!from!the!literature,!it!resonated!as!a!case!study!relevant!for!

facilitating!the!READ!application!because!of!the!strong!academic!orientations!of!several!of!the!key!

partners.!(This!will!be!expanded!on!in!Chapters!4!and!Chapter!5!about!the!READ!application.)!

Leading!the!way!in!innovating!applied!participatory!evaluation!practice!in!community!arts!in!

Australia!are!organisations!such!Big!hART!(community!arts!and!production!company),!Somebody’s!

Daughter!Theatre!company,!Arts!Victoria,!Queensland!University!of!Technology!and!the!University!of!

Melbourne.18!Of!particular!relevance!to!the!DCI!research!are!findings!emerging!from!the!three!year!

Australian!Research!Council!(ARC)!Linkage!Project!Towards%an%integrated%evaluation%framework%for%

communityAbased%arts!completed!in!2014!(see!MacDowall!et!al.,!2013).!The!ARC!research!tested!a!

number!of!evaluation!approaches!in!the!community!partnerships!context!based!on!four!industry*

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

18!Examples!from!these!organisations!are!also!referred!to!in!Chapter!5!and!include!(in!order):!Big!hART!(2010),!Palmer!

(2010);!Osmotherly!(2002),!Osfield!Consultants!(2007);!Keating!(2002);!Klaebe,!Ellison,!and!Kirk!(2015),!Gattenhof!and!

Seffrin!(2014);!MacDowall!et!al.!(2013).!I!also!reviewed!several!good!practice!evaluation!reports!and!industry!case!studies!

not!in!the!public!domain!which!were!informative,!with!assistance!from!the!Australia!Council!for!the!Arts!in!facilitating!a!

permissions!process!for!access!to!the!exemplars.!

!


! 35!

embedded!case*studies!exploring!the!themes!of!negative!value,!new!aesthetics,!networked!thinking!

and!dialogic!approach!(MacDowall!et!al.,!2013).!Engagement!with!this!ARC!Linkage!Project!research!

has!been!formative!for!advancing!and!consolidating!an!understanding!of!the!context!of!practice,!and!

provided!unique!access!to!practitioners!working!on!participatory!community!arts!evaluation.!

Individual!exemplars!were!also!relevant,!such!as!Evaluation%of%the%Community%Arts%Development%

Scheme!(Kelaher,!Dunt,!Curry,!Berman,!&!Joubert,!2009).!

Methods%of%relevance%for%the%READ%application%

The!implication!for!evaluators!interested!in!fostering!participation!is!the!need!for!sound!methods!

that!prioritise!equity!amongst!partner!needs!and!utilise!the!strengths!of!the!site!of!practice!and!

context!–!while!achieving!rigorous!and!credible!results!(Hadley!&!Gattenhof,!2012).!Despite!the!

previously!noted!trend!toward!social!impact!methodology,!including!quantification!via!cultural!

indicators!(discussed!in!Section!2.3.1),!participatory!evaluation!methodologies!in!practice!still!rely!

heavily!on!qualitative!methods,!and/or!use!of!mixed!research!methods!to!elicit!qualitative,!

quantitative!and!performative!data!(including!ephemeral!rich!media!such!as!video,!audio!and!

images)!(Belfiore!&!Bennett,!2010;!Clements,!2007;!Haseman,!2006).!!

From!the!contextual!scan!there!were!a!number!of!relevant!methods!studied!for!the!READ!

application!in!relation!to!various!domains!of!evaluation!practice.!These!included!a!mix!of!

conventional!methods!for!data!collection,!analysis,!and!stakeholder!engagement!that!were!suitable!

for!participation*oriented!evaluation.!Included!was!research!on!methods!such!as!collective!

workshops,!focus!groups,!evaluation!specific!tools!(such!as!evaluation!needs!assessment),!

partnership!assessment!techniques,!relevant!ethical!frameworks!(Markiewicz,!2012;!Edenloff,!2011).!

In!addition!to!these!conventional!methods,!I!examined!a!range!of!art*based!methods!for!evaluation!

of!relevance!for!the!Nuclear!Futures!site!of!practice.!Art*based!methods!use!creative!and!artistic!

mediums!(e.g.!video/film,!photography,!narrative/storytelling,!performance,!written!text)!to!

generate!understanding!of!a!subject!and!address!social!research!problems!(Leavy,!2009;!McNiff,!

2008).!Such!methods!are!diametrically!opposed!to!the!quantitative!approaches!described!in!Section!

2.3.1.!Goldbard!(2015,!p.226)!advocates!that!artists!and!cultural!policy*makers!have!the!right!to!

draw!on!‘story,!image,!metaphor,!and!experience’!as!tools!to!convey!cultural!value!and!meaning,!

and!argues!the!relevance!of!these!methods!for!future!evaluation!practice!in!the!community!art!

context.!!

!

!


! 36!

The!following!quote!elaborates!on!how!Goldbard!envisages!the!applications!of!art*based!methods:!

Right!now,!visionary!leaders!are!using!digital!stories!–!short,!first*person!sound*and*image!

video!narratives!–!to!capture!participant’s!experience!and!response.!They!are!employing!the!

methods!of!art!to!convey!art’s!value!and!identify!cultural!development!needs,!by!convening!

story!circles!and!turning!the!collective!wisdom!thus!extracted!into!plays!and!videos!that!can!

be!shared,!sparking!useful!dialogue;!or!by!engaging!community!members!in!envisaging!their!

collective!future!through!a!participatory!photography!project!in!which!images!and!captions!

show!what!needs!preservation!and!what!needs!development.”!(Goldbard,!2015,!p.226)!

While!art*based!methods!are!still!considered!to!be!‘on!the!fringe!of!social!science!research’!

(Edenloff,!2011,!p.!62),!and!are!most!commonly!used!in!conjunction!with!conventional!qualitative!

methods!and!not!as!stand*alone!methods,!supporters!argue!that!they!are!particularly!valuable!

techniques!for!arts*related!evaluation!(Rudkin!&!Davis,!2007).!Leavy!(2009)!argues!that!the!methods!

elicit!different!types!of!information!to!conventional!qualitative!and!quantitative!methods,!nuanced!

by!rich!descriptive!expression,!and!grounded!in!stories,!images!and!feelings.!Arts*based!evaluation!

methods!are!also!identified!as!effective!methods!when!working!with!children!and!youth,!alleviating!

the!reliance!on!written!and!oral!communication!(Edenloff,!2011).!Successful!application!is!also!

demonstrated!with!Indigenous!communities!due!to!the!reflexivity!that!art*based!methods!offer!

(Price,!McCoy,!&!Mafi,!2012).!All!of!these!attributes!of!the!creative!methods!suite!were!deemed!

highly!relevant!for!the!READ!application!(covered!later!in!this!dissertation)!to!supplement!the!more!

formal!and!conventional!evaluation!methods!being!included!in!the!READ!model!development.!

As!the!lead!practitioner!responsible!for!driving!the!design!and!implementation!of!the!READ!model,!

to!take!it!from!the!basic!idea!that!existed!prior!to!the!research!to!developing!a!complete!

methodological!framework!for!implementation,!the!contextual!scan!conducted!for!the!Doctor!of!

Creative!Industries!research!has!proved!extremely!valuable.!As!part!of!my!leadership!role!at!my!site!

of!practice,!I!shared!these!findings!with!my!colleagues!and!community!partners!to!ensure!our!

collective!efforts!towards!the!READ!application!(detailed!in!Chapter!5)!were!informed!by,!and!built!

upon,!existing!theories!and!knowledge!of!evaluation!practice.! !


! 37!

2.4! Conclusion#

The!contextual!scan!involved!the!far*ranging!review!of!scholarly!and!industry!literature!on!

evaluation,!and!exemplars!from!the!community!arts!context.!Investigating!the!key!literature!areas!

related!to!my!professional!practice!and!writing!up!the!findings!in!this!chapter!helped!to!make!sense!

of!the!vast!and!contested!practice!of!community!arts!program!evaluation.!Historical!and!theoretical!

trends!in!evaluation!were!charted,!and!the!cited!academic!analyses!were!revealing!*!particularly!in!

regard!to!the!impact!of!the!instrumentalisation!of!the!arts!on!evaluation!practice!and!the!still!

unfolding!debate!on!cultural!value.!Many!of!the!theorists!I!studied!(such!as!Goldbard,!MacDowall,!

Badham,!and!Belfiore)!resonated!with!my!own!philosophies!developed!through!my!professional!

practice,!and!I!experienced!immediate!benefits!for!my!evaluation!practice!as!I!underwent!an!

iterative!reflective!process!of!learning,!incorporating!new!understandings!into!practice,!and!

reframing!my!concept!of!evaluation!for!community!arts!as!I!delved!deeper!into!the!theory.!

Importantly,!the!scan!provided!the!basis!for!positioning!the!DCI!research!within!the!domain!of!

participation*oriented!program!evaluation!within!community!arts,!and!identified!gaps!in!the!

literature!for!participatory!frameworks!that!eloquently!deal!with!the!complexities!of!creative!

community!initiatives!involving!diverse!partnerships.!The!next!chapter,!Chapter!3,!outlines!the!

overarching!DCI!research!design!and!provides!the!rationale!for!the!qualitative!methodology!and!

methods!I!employed!to!study!and!critically!review!the!READ!model!developed!at!my!site!of!practice.!

!

!

!

!

!

!

!

!

!

Digital+story!–!Reflecting+on+the+problem+of+community+arts+evaluation+

!

As! part! of! the! program! of! forensic! reflective! practice! undertaken! for! the!

research!(detailed!in!the!next!chapter)! I!produced!a!digital!story!exploring!the!

research!problem,!as!a!sensemaking!exercise! in!response!to!themes!emerging!

from! the! contextual! scan.! The!digital! storytelling! process!was! employed! as! a!

research!method,!and!is!not!a!summative!product!of!the!scan.!It!does!however!

demonstrate!the!understanding!I!gained!during!the!early!phase!of!the!research!

in!relation!to!the!knowledge!gaps!for!community!arts!evaluation.!

The!video!(3mins!40sec)!and!script!for!the!digital!story!can!be!viewed!at:!

http://nuclearfutures.org/digital*story*community*arts*evaluation!!!

(Please!use!password:!!READ!)!


! 38!

! !

Blank!page!


! 39!

3! Research#design##

Chapter!3!outlines!the!research!design!implemented!for!the!Doctor!of!Creative!Industries!(DCI)!to!

study!the!development!and!first!industry!application!of!the!READ!model,!a!newly!devised!evaluation!

model!for!community!arts.!The!READ!approach!arises!from!professional!practice!and!integrates!

Reflection,!Evaluation,!Analysis!and!Documentation.!The!purpose!of!this!chapter!is!to!detail!the!key!

elements!and!rationale!for!the!DCI!research!design!*!the!critical!lens!I!devised!for!evaluating!the!

model,!in!order!to!substantiate!a!final!READ!product!for!evaluation!practice.!Summarised!is!the!

qualitative!methodology,!practice*informed!paradigm,!research!strategy,!and!the!data!collection,!

analysis!and!theory!building!methods!that!I!employed!during!the!DCI!to!meet!the!research!aims.!

One!of!the!significant!influences!on!the!research!design!was!the!context!of!the!enquiry,!and!the!

central!positioning!of!myself!as!researcher!and!lead!Evaluator.!The!research!was!dually!positioned!

within!industry!and!academic!contexts,!in!the!fields!of!community!arts!and!evaluation.!The!site!of!

practice!for!research!was!the!Nuclear!Futures!Partnership!Initiative!(Nuclear!Futures),!a!three*year!

creative!community!partnership!initiative!funded!by!the!Australia!Council!for!the!Arts!(overviewed!in!

Chapter!4).!!The!research!design!I!devised!for!the!DCI!aimed!to!provide!a!framework!for!the!

systematic!analysis!of!the!conceptualised!READ!model!(in!terms!of!performance,!strengths,!

weaknesses,!areas!of!innovation)!based!on!its!development!and!testing!for!the!Balaklava!Community!

Arts!Project!delivered!in!South!Australia!during!2014*5!(see!Chapter!5).!As!well!as!informing!the!

future!evaluation!practice!for!the!Nuclear!Futures!program,!the!research!aimed!to!identify!the!

potential!areas!of!innovation!of!broader!relevance!for!the!fields!of!evaluation!and!community!arts.!

To!address!the!DCI!research!problem,!I!adopted!a!practice*informed!approach!and!designed!a!robust!

qualitative!methodology!that!created!a!critical!and!reflective!lens!for!studying!the!READ!model’s!

development!and!first!industry!application.!Demanded!was!a!methodology!capable!of!capturing!and!

substantiating!the!learnings!arising!from!evaluation!practice!and!enabling!the!investigative!analysis,!

theorisation!and!refinement!of!the!model!based!on!the!Balaklava!experience.!The!primary!research!

strategy!for!the!enquiry!was!forensic!reflective!practice!–!a!knowledge*driven!approach!involving!the!

systematic!examination!of!thinking!and!action!for!critical!enquiry!and!for!the!continuous!

improvement!of!practice!(Schon,!1983).!This!strategy!utilised!reflective!and!qualitative!methods!for!

data!collection!including!journaling,!video!diaries,!interviews!and!observation.!In*depth!analysis!of!

the!data!generated!was!conducted!via!secondary!reflection!for!the!purpose!of!building!evaluation!

theory,!and!utilised!interpretative!techniques!such!as!coded,!textual!and!comparative!analysis,!as!

well!as!creative!and!meta*evaluation!techniques!(to!evaluate!the!evaluation).!! !


! 40!

3.1! Qualitative#research#methodology#

A!qualitative!methodology!was!selected!as!being!most!adept!at!situating!the!practitioner!as!a!

researcher!and!for!delivering!the!interpretative!aims!of!the!research.!The!qualitative!paradigm!

underpinning!the!research!inherently!valued!my!own!context!and!interpretation!as!a!researcher,!and!

allowed!reflexivity!to!emergent!themes!arising!from!the!inductive!research!process!and!reasoning!

(Denzin!&!Lincoln,!2011).!By!utilising!interpretative!research!practices!or!‘representations’!(such!as!

field!notes,!visual!texts,!interviews)!the!qualitative!methodology!allowed!me!‘to!make!sense!of!and!

interpret!phenomena!in!terms!of!the!meanings!people!bring!to!them’!(Denzin!&!Lincoln,!2011,!p.3).!

Specifically,!the!qualitative!research!methodology!endorsed!the!necessary!capture!and!

interpretation!of!meaning!relating!to!convergent!conceptualisations!(Ezzy,!2006;!Becker,!1996)!

which!in!this!case!related!to!catering!for!divergent!definitions!of!evaluation!and!the!READ!approach!

at!the!site!of!practice,!for!the!purpose!of!original!theory!development!of!the!READ!model.!

In!response!to!the!research!aims!(see!Section!1.4),!the!methodology!needed!to!address!the!dilemma!

of!conducting!research!required!to!both!answer!a!research!question!and!fulfil!a!practical!need!

(Blichfeldt!&!Andersen,!2006),!which!involved!‘the!active!and!deliberate!self*involvement!of!the!

researcher!in!the!context!of!his/her!investigation’!(McKay!&!Marshall,!2001,!p.!49).!That!is,!the!

research!sought!to!advance!evaluation!theory!and!practice!for!community!arts!and!this!involved!

facilitating!and!analysing!the!design,!implementation!and!analysis!of!the!READ!model!to!meet!the!

evaluation!needs!at!my!site!of!professional!practice.!As!explained!in!Section!3.2,!the!qualitative!

methodology!needed!to!elicit!data!prior!to,!during!and!following!the!READ!application,!and!this!was!

achieved!through!an!ongoing!schedule!of!reflective!practice.!Figure!3.1!depicts!the!relationship!

between!the!overarching!qualitative!methodology!for!the!DCI!(discussed!in!this!chapter)!and!the!

READ!application!under!review.19!

!

!

!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

19!It!is!important!to!note!that!the!approach!and!methods!documented!in!this!chapter!for!the!overarching!DCI!methodology!

are!separate!to!the!READ!methodology!presented!in!Chapter!5.!


! 41!

!

Figure!3.1:!The!Doctor!of!Creative!Industries!research!design!

!

!

The!following!sections!explain!the!key!elements!of!the!DCI!methodology!summarised!in!Figure!3.1.20!

3.1.1! PracticeSinformed#research#paradigm#

Central!to!Queensland!University!of!Technology’s!(QUT)!DCI!program!is!the!dominant!paradigm!of!

practice*led!research.!Practice*led!refers!to!the!emergent!performative!research!paradigm!defining!

artist*as*researcher,!where!artists!embrace!the!dual!roles!of!the!researcher!and!the!researched!

through!creative!practice!(Sullivan,!2010,!2009;!Haseman!&!Mafe,!2009).!In!practice*led!research,!

the!artist*as*researcher!engages!in!the!creation!of!data!through!performative!or!creative!methods!

and!outputs!(Smith!&!Dean,!2009;!Gray,!1996).!Creative!practice%is!the!research!activity!(Haseman,!

2006).!In!most!cases,!DCI!enquiry!becomes!a!form!of!research!that!will!incorporate!creative!or!

design!practice!in!the!research!methodology!and!outputs.!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

20!This!figure!is!included!in!Chapter!1!as!Figure!1.1.!I!have!included!the!diagram!again!in!Chapter!3!as!it!is!an!important!

reference!figure!demonstrating!the!difference!between!the!overall!DCI!methodology!and!the!trialed!READ!methodology!

(‘the!application’).!

!


! 42!

For!this!research,!my!role!in!the!professional!context!was!as!evaluator!for!a!creative!arts!program,!

and!not!as!a!creative!artist/!practitioner.!While!this!research!design!for!the!DCI!is!aligned!with!the!

practice*led!paradigm,!the!research!undertaken!was!non*performative!(in!terms!of!generating!

creative!arts)!and!better!described!as!practice*informed.21!The!practice*informed!approach!has!

nuances!of!the!practice*led!paradigm,!however!differs!in!that!the!research!is!undertaken!in!

relationship!to!creative!practice,!not!through!it.!!A!major!part!of!the!practice*informed!research!

methodology!employed!was!built!around!the!research!and!outputs!generated!through!the!

evaluation!practice.!While!not!primarily!geared!toward!creative!work,!it!is!important!to!note!that!

several!of!the!evaluation*related!outputs!utilised!arts*based!methods!to!produce!original!creative!

documentation.!

3.2! Research#strategy:#forensic#reflective#practice#

Over!the!past!30!years,!reflective!practice!has!gained!standing!as!a!research!strategy!for!critical!

enquiry!across!many!disciplines!including!Nursing,!Education!and!the!Creative!Industries!(Ryan!&!

Cousins,!2009;!Finlay,!2008;!Johns,!2006;!Alvesson!&!Skoldberg,!1999;!Gibbs,!1988).!Reflective!

practice!is!of!particular!relevance!for!uncertain!and/or!complex!social!and!technical!contexts!

(Bourner,!2003;!Schon,!1983)!and!highly!adaptable!for!practice*led!and!practice*informed!research!

(Haseman!&!Mafe,!2009).!For!the!DCI,!the!Creative!Industries!Faculty!at!QUT!developed!a!robust!and!

highly!critical,!knowledge*generating!application!of!reflective!practice!–!!forensic!reflective!practice!–!

which!is!the!recommended!research!strategy!for!DCI!candidates!(Stock,!2013).!The!next!section!

presents!the!theoretical!underpinnings!of!forensic!reflective!practice,!ahead!the!discussion!on!

methods!selected!for!this!DCI,!presented!in!Section!3.3.!

3.2.1! Theoretical#underpinnings#of#reflective#practice#

Central!to!reflective!practice!is!the!concept!of!reflection,!referring!to!the!active,!deliberate!and!in*

depth!examination!of!thoughts!and!action,!as!a!means!to!develop!or!improve!practice!(Bolton,!2010;!

Leitch!&!Day,!2000). Whether!directed!towards!thought!processes,!activities,!experiences,!events!or!

problematic!situations,!reflection!is!used!to!bring!a!subject!into!focus!and!generate!new!insights!and!

perspectives!about!the!subject!(Bolton,!2010;!Boud,!2001).!While!definitions!vary,!most!chart!a!

process!of!problem!identification,!reframing!(exploring!alternative!views)!and!analysis,!followed!by!

structured!action!(Loughran,!2002).!Figure!3.2!summarises!the!forensic!reflective!practice!process.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

21!The!term!practice*informed!is!not!cited!from!the!literature!and!was!sourced!from!personal!communication!with!Assoc.!

Prof.!Sandra!Gattenhof,!with!input!from!Prof.!Helen!Klaebe!(on!28.5.13).!It!is!based!on!early!concepts!of!practice*based!

research!strategies!emerging!in!the!United!Kingdom!in!the!mid!1970s.!


! 43!

!

+

!

!

!

!

!

!

!

!

!

!

!

!

Figure!3.2:!Model!for!forensic!reflective!practice!!

(Illustration!adapted!from!Scalan!&!Chernomas!(1997)!*!Three!Stage!Model)!

!

!

!

Early!scholarship!by!Dewey!(1933)!depicts!reflection!as!a!precise!and!rigorous!practice!using!

systematic!methods!of!observation,!description,!analysis,!theory!building!and!experimentation!to!

construct!meaning!(Rodgers,!2002).!Dewey!also!considered!the!practice!to!have!moral!ends,!

promoting!reflection!as!a!means!for!personal!and!intellectual!growth!of!self!and!others!(Rodgers,!

2002;!Dewey,!1933).!It!was!not!until!the!more!recent!era!of!theorisation!however!(from!the!1980s!

onwards),!introduced!by!Schon!(1983)!following!his!earlier!work!with!Argyris!in!the!1970s,!that!

reflective!practice!gained!traction!as!a!strategy!for!critical!enquiry!(Bourner,!2003).!!

Central!to!Schon’s!reflective!practice!theory!is!the!premise!that!‘our!knowing!is!in!our!action’!(Schon,!

1983,!p.49).!Schon!argues!that!reflection*on*action!(reflection!after!the!event)!and!reflection*in*

action!(reflection!during!the!event)!are!legitimate!and!valuable!sources!of!knowledge.!Bolton!

describes!the!reflective!practitioner’s!process!as!‘paying!critical!attention!to!the!practical!values!and!

theories!which!inform!everyday!actions,!by!examining!practice!reflectively!and!reflexively’!(Bolton,!

2010,!p.!xix).!Through!reflective!practice!the!implicit!knowledge!embedded!in!professional!practice!is!

able!to!be!unearthed,!critically!examined,!theorised,!tested!and!made!applicable!as!new!knowledge!

(Bourner,!2003;!Schon,!1983).!Bourner!asserts!that!it!is!the!process!of!critical!examination!through!

probing!questions!that!distinguishes!‘reflective’!from!‘unreflective’!thinking!(Bourner,!2003,!p.269).!


! 44!

The!rise!of!reflective!practice!has!been!prominent!in!health,!teacher,!and!professional!education!

across!the!United!Kingdom,!North!America!and!Australia!(Ryan!&!Cousins,!2009;!Greenwood,!1993).!

Since!Schon’s!seminal!work,!numerous!models!to!operationalise!reflective!practice!have!been!

developed!across!various!fields,!including!in!the!Creative!Industries!(see!Smith!&!Dean,!2009;!

Doloughan,!2002).!The!now!widely!cited!domain!of!reflective!practice!includes!a!spectrum!of!models,!

each!differing!in!rigour!and!critical!reflection.!As!is!described!in!the!following!sections,!forensic!

reflective!practice!constitutes!a!significant!nexus!between!critical!reflection,!professional!practice,!

and!research!enquiry!(Boud,!2001;!Newman,!1999).!

3.2.2! Methodological#considerations#for#forensic#reflective#practice#

For!this!research,!the!forensic!reflective!practice!strategy!has!been!designed!to!be!rigorous!while!

guiding!a!flexible!and!socially!responsive!approach!to!the!critical!review!of!practice,!through!

deepening!cycles!of!forensic!reflection.!As!well!as!framing!the!research,!reflective!practice!was!used!

as!an!evaluator’s!tool!for!improving!evaluation!practice,!which!is!argued!in!the!literature!to!help!

make!explicit!some!of!the!espoused!values!and!assumptions!that!commonly!get!embedded!within!

program!evaluation!design!and!implementation!(MacDowall!et!al.,%2013;!Bolton,!2010).!!

In!planning!the!program!of!forensic!reflective!practice!several!methodological!issues!were!

addressed.!First,!in!considering!the!interaction!between!researcher!and!research!material,!it!is!

acknowledged!that!through!the!reflective!practice!process!‘knowledge!is!negotiated’!as!a!result!of!

personal!construction!(Gray,!1996,!p.13).!While!traditional,!positivist!opponents!may!argue!that!

subjectivity!and/or!the!centrality!of!the!practitioner!in!the!research!context!‘stands!to!infect!the!

objective!‘truth’!and!universal!applicability!of!research!findings’!(Haseman!&!Mafe,!2009,!p.212),!the!

credential!of!the!reflective!practice!strategy!is!that!it!adequately!accommodates!and!makes!meaning!

from!the!value*based!terrain!of!practice*informed!research!to!unearth!valuable!tacit!knowledge.!

Second,!for!the!research!strategy!to!be!forensic,!knowledge*directed!and!produce!communicable!

findings,!the!methodology!for!the!collection!and!analysis!of!empirical!material!during!the!reflective!

process!needed!to!be!systematic!and!rigorous!(Stock,!2013;!Archer,!1995).!Schon!argues:!

The!dilemma!of!rigor!or!relevance!may!be!dissolved!if!we!can!develop!an!epistemology!of!

practice!which!places!technical!problem*solving!within!a!broader!context!of!reflective!inquiry,!

shows!how!reflection*in*action!may!be!rigorous!in!its!own!right,!and!links!to!the!art!of!practice!

in!uncertainty!and!uniqueness!to!the!scientist's!art!of!research.!(Schon,!1983,!p.69)!

Specifically,!scholarship!on!reflective!practice!suggests!that!the!type!and!quality!of!claims!to!

knowledge!able!to!be!made!as!a!reflective!practitioner!will!be!strongly!influenced!by!the!methods!

selected!and!the!effectiveness!of!implementation!(Stock,!2013;!Loughran,!2002).!!


! 45!

In!response,!recognising!the!implicit!and!explicit!influences!of!myself!as!researcher!during!the!

reflective!enquiry!demanded!accurate!representation!of!the!tensions!created!from!‘the!necessary!

interrelationship!of!(pre*existing)!materiality!and!subjectivity!as!they!interact!in!a!dynamic!and!

motivated!context’!(Doloughan,!2002,!p.61).!Techniques!to!promote!self*questioning,!dialogic!

reflection!and!metacognition!that!would!allow!me!to!stand!back!from!the!event,!consider!alternative!

perspectives,!explore!the!emotional!influences!present,!and!review!my!reflective!processes!were!

incorporated!based!on!practices!evidenced!in!the!literature!(such!as!Moon,!2001).!Comprehensive!

documentation!aided!to!make!these!tensions!explicit,!and!during!the!READ!application!I!attempted!

to!keep!a!balance!between!the!READ!practice!and!the!forensic!reflective!practice.!Scholars!such!as!

Haseman!and!Mafe!(2009)!and!Finlay!(2008)!argue!that!managing!the!flux!between!research!and!

practice,!a!balance!between!rigour!and!moderation!needs!to!be!achieved!so!that!the!reflective!

process!becomes!systematic!without!overbearing!the!practice!(which!is!in!my!case!relates!to!my!

evaluation!practice),!to!ensure!an!adequate!level!of!reflexive!detachment!is!maintained.!

3.3! Data#collection#and#analysis#methods#

The!suite!of!data!collection!and!analysis!methods!included!in!the!research!design!was!informed!by!a!

literature!review!of!qualitative!research!methods!(informed!by!scholars!such!as!Denzin!&!Lincoln,!

2011;!Cresswell!&!Clarke,!2007;!Ezzy,!2006;!Becker,!1996).!(See!Table!3.1!for!a!summary.)!!While!the!

methods!are!categorised!separately!in!Table!3.1,!the!phases!of!data!collection,!analysis!and!theory!

development!were!iterative!and!cyclic!processes!as!part!of!the!forensic!reflective!practice.22!!

The!primary!methods!for!data!collection!were!a!mix!of!tailored!reflective!methods!(journaling,!video!

diary,!structured!exercises)!and!standard!qualitative!methods!(interviews!and!observation).!

Secondary!reflection!for!the!purpose!of!in*depth!analysis!and!theory!building!(to!develop!a!READ!

model!of!practice)!utilised!mixed!integrative!techniques!and!creative!methods,!refining!data!and!

consolidating!emergent!themes.!Reflective!methods!included!conventional!data!capture!activities!as!

well!as!practice*related!methods!involving!the!generation!and!curation!of!creative!documentation!–!

where!critical!analysis!is!undertaken!through!the!compilation!process.!Throughout,!emergent!

findings!were!fed!into!the!testing!and!theory!development!of!the!READ!model!during!and!following!

the!Balaklava*based!application,!in!an!iterative!process!of!data!collection,!organisation,!analysis,!and!

testing!guided!by!deepening!cycles!of!reflective!review.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

22!In!the!evaluation!field,!reference!to!theory!development!differs!from!the!conventional!theory!building!of!other!fields!

that!adhere!to!an!academic!or!theoretical!standard,!and!tend!to!be!exclusively!prescriptive!(not!predictive)!offering:!‘a!set!

of!rules,!prescriptions!and!prohibitions!that!specify!what!a!good!or!effective!evaluation!study!is!and!how!an!evaluation!

study!should!be!conducted’!(Christie!and!Alkin,!2008,!p.132;!Alkin,!2004).!In!the!evaluation!field,!models!of!practice!

contribute!to!evaluation!theory.!


! 46!

Table!3.1:!Qualitative!methods!employed!for!the!DCI!research!design!(2013*15)!!

QUALITATIVE!METHODS! RATIONALE!

!

!

!

!

!

Data+collection+

(primary)+

++

Prior,!during!and!

post!the!READ!

application!!

!

Forensic!reflective!practice:!

•! journaling!(written)!

•! video!diary!

•! semi*structured!exercises!

•! collective!sessions!

•! creative!documentation!!

•! enabled!original!data!generation!via!individual/!collective!

reflection!using!multiple!methods!(Bolton,!2010)!

•! unlocked!tacit!knowledge!(Schon,!1983)!

•! responsive!to!the!emergent!research!themes!and!data!from!other!

(Archer,!1995)!

•! creative!documentation!(photo!books/essays!and!digital!

storytelling)!acted!as!dual!data!collection!and!analysis!methods!

Interviews!and!consultation:!

•! semi*structured!and!

unstructured!!

•! individual!and!group!face*to*

face!and!telephone/!Skype!

•! informal!stakeholder!

consultation!and!liaison!!

•! made!accessible!the!interpretations,!perceptions!and!experiences!

of!individuals!considered!to!have!a!detailed!knowledge!of!the!

research!topic!(Mason,!2002)!

•! allowed!for!mix!of!sample!and!question!design!techniques!

•! flexible!for!planned!and/or!spontaneous!interviews/!feedback!!

•! produced!rich!source!for!data!collection!and!was!used!for!

clarification!for!theory!and!product!development!(Charmaz,!2000)!

Observation!as!researcher!and!

READ!facilitator:!

•! field!notes!

•! anecdotal!notes!!

•! utilised!researcher/!practitioner!observations!!

•! provided!thick!and!rich!descriptions!of!context!and!key!project!

milestones!(Denzin!and!Lincoln,!2011)!

•! important!bridge!between!program!management!and!evaluation!

!

Data+collection+

(secondary)+

Contextual!Scan:!

•! academic!and!industry!

sources,!referred!exemplars!

•! online!and!social!media!

•! utilised!a!variety!of!data!sources!such!as!academic!literature,!

industry!reports/!exemplars,!website!and!online!forums!!

•! supported!the!theory!development!process,!and!informed!coding!

and!analysis!

!

!

!

!

!

!

!

Data+analysis+

and+product+

development+

Reflective!creative!documentation:!

•! digital!storytelling!•! photomedia!collation!

•! communications!

•! method!for!analysis!and!content!generation!(Burgess,!2006)!

•! reflection!used!as!communication!tools!(Couldry,!2008)!

•! the!process!of!analysis!and!producing!creative!documentation!

involved!a!hierarchy!of!reflection!from!descriptive!to!dialogic!to!

critical!enquiry!Hatton!and!Smith!(1995)!

Analysis!techniques!included:!

•! thematic!coding,!sorting,!and!

categorisation!!

•! coded,!interpretative,!

comparative,!textual,!

collective,!reflective!analysis!!

•! meta*evaluation!

•! well!suited!for!culturally!sensitive!research!topic!(Charmaz,!2000)!

•! micro!level!analysis!uncovered!how!stakeholder!constructs!of!

meaning!and!deeper!level!analysis!teased!out!the!underlying!

discourse/!ideologies!feeding!into!READ!constructions!

•! manual!approach!used!for!managing,!sorting,!and!interpreting!

information!to!deepen!reflective!practice!(i.e.!not!software)!

•! meta*evaluation!used!to!evaluate!the!evaluation!(Scriven,!2012)!

Theory!and!product!development!!

•! information!management!!

•! theoretical!sampling!

•! inductive!reasoning!

•! critical!reflection!

•! design!and!communication!

•! identified!interesting!patterns/!themes!(Braun!&!Clarke!2006).!

•! inductive!approaches!were!tailored!to!suit!theory!development!

(Lehmann,!2001;!Charmaz,!2000)!

•! the!process!involved!deep!and!critical!analysis!(Dobson,!1999)!•! constructivist!approaches!suited!to!the!qualitative!research!at!

hand!(Charmaz,!2000)!

•! structured!design!and!product!development!

!

Concurrent!with!the!primary!research!was!a!comprehensive!contextual!scan!(see!Chapter!2).!The!

scan!comprised!a!literature!review!in!evaluation!(particularly!in!the!domain!of!participatory!

evaluation)!and!community!arts!(including!community!partnerships)!and!review!of!practice*based!

resources!(including!industry!reports!and!documentation,!online!and!social!media!platforms,!peer*

recommended!evaluation!exemplars,!and!seminar/!conference!papers).!

!


! 47!

The!following!sections!provide!descriptions!of!the!key!research!methods!utilised!as!part!of!the!

forensic!reflective!practice!strategy.!The!methods!were!employed!for!data!collection!and!analysis!for!

my!process!of!appraising!the!performance!and!outcomes!of!the!READ!application,!and!facilitating!

the!collective!development!of!the!model!for!future!use.!!

3.3.1! Reflective#methods##

A!cognitive!program!for!forensic!reflective!practice!was!designed,!utilising!scheduled!and!reflexive!

sessions!for!data!collection,!analysis!and!knowledge!generation.!The!methods!selected!supported!

the!reflective!practice!cycle!shown!in!Figure!3.2,!and!were!chosen!based!on!the!quality!and!depth!of!

the!reflection!they!could!deliver,!which!is!argued!by!Moon!(2001)!to!be!of!critical!importance.!The!

multimodal!inquiry!approach!I!devised!was!inspired!from!the!Stepping%Outside%the%Circle!online!

resource,!a!reflective!practice!framework!for!creative!facilitators!created!by!Moffatt!(2014)!geared!

toward!supporting!transformative!learning!and!action.23!Across!the!READ!application!I!employed!

reflective!methods!at!various!times,!such!as!before,!during!or!post*events!or!key!evaluation!

milestones.!Loughran!(2002,!p.35)!states!that!a!mix!of!reflective!methods!employed!as!‘anticipatory,!

retrospective!and!contemporaneous!reflection’!achieves!Schon’s!reflection*in*action!and!reflection*

on*action!principles,!as!well!as!incorporates!reflection*before*action!to!rationalise!pre*existing!

judgments,!prejudices!and!actions!(Kotzee,!2012;!Finlay,!2008;!Loughran,!2002;!Schon,!1983).!!

The!documentation!of!reflective!practice!aimed!to!capture!progress,!learning,!problematic!blocks,!

failures,!strengths,!and!shifts!to!articulate!the!taken*for*granted!aspects!of!practice.!As!argued!by!

Boud!(2001,!p.9),!reflective!documentation!functions!as!‘a!lens!for!learning’.!Informed!by!techniques!

of!scholars!such!as!Bolton!(2010),!Gray!(1996)!and!Burgess!(2006),!the!methods!I!employed!included!

unstructured!and!structured!reflective!journaling!(20+!entries),!video!diaries!(10+!entries),!

production!of!one!digital!story,!anecdotal!notes,!online!posts,!and!archiving!of!visual!and!audio!

material.!The!systematic!use!of!such!methods!is!argued!to!result!in!grounded!research!analysis!and!

interpretative!learning!(Stock,!2013),!and!produce!an!emergent!language!able!to!blend!process!and!

product,!form!and!content,!creativity!and!rationality!(Doloughan,!2002).!

! !

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

23!Moffat’s!framework!can!be!retrieved!from:!http://www.steppingoutsidethecircle.com/#!build*your*own/c1qr4!!


! 48!

Journaling*and*video*diaries*

The!two!principal!reflective!methods!I!employed!were!written!reflective!journaling!and!recording!a!

video!diary.!Guiding!the!forensic!reflective!practice!for!both!of!these!methods!were!strategic/!

directed!questions!and!activities,!as!well!as!an!unstructured!approach.!Boud!(2001,!p.12)!describes!

the!unstructured!approach!as!being!characterised!by!a!‘focus!on!uncertainty,!perplexing!events,!and!

exploration!without!necessarily!knowing!where!it!will!lead’.!Throughout!my!reflective!practice,!

searching!questions!(reflective!questions)!were!used!to!stimulate!reflective!thinking!and!scrutinise!

my!practitioner!experience.!This!is!a!technique!supported!by!Bourner!(2003).!!

Asking!questions!of!ourselves!as!practitioner*researchers!not!only!produces!new!information!and!

perspectives,!but!can!stimulate!creative!and!innovative!shifts!in!practice!(Tharp,!2003;!Gray!1996).!

Haseman!and!Mafe!(2009,!p.225)!argue!‘questioning!is!the!key!to!deepening!reflexivity’.!Utilised!

were!emergent!questions!as!well!as!selected!structured!and!semi*structured!exercises/!templates/!

question*sets!designed!to!guide!my!reflective!exploration,!promote!skills!development!and!advance!

practice!(Bourner,!2003).24!The!results!of!this!type!of!reflective!practice!are!documented!in!Chapter!

6.!Combined,!the!journaling!and!video!diary!sessions!and!exercises!produced!vast!and!rich!data.!

Interestingly,!they!also!produced!many!questions!which!compelled!the!reflective!practice.!!To!assist!

with!organising!and!re*evaluation!through!the!reflective!practice,!reframing!techniques!were!used!

involving!the!consideration!and!analysis!of!alternative!perspectives!via!techniques!such!as!viewing!

data!through!the!lens!of!different!stakeholder!positions!and!the!meta*evaluation!techniques!

discussed!below.!

Critical*buddy*and*shared*reflective*discussion**

Self*evaluation!was!enhanced!through!critical!exchange!and!shared/collective!reflection!sessions!

with!selected!peers!and!colleagues!working!at!the!site!of!practice.!Such!sessions!are!defined!in!the!

literature!as!critical!friend!or!buddy!sessions!and!involve!review,!feedback!and/or!critical!discussion!

on!practice!between!individuals!(Bambino,!2002).!The!technique!of!sharing!reflective!dialogue!with!a!

critical!friend!(or!buddy)!is!argued!by!Lamont!(2011,!p.64)!to!help!‘colleagues!to!make!sense!of!their!

world!by!being!non*judgmental!and!trying!to!understand!their!perspective’.!During!the!Balaklava!

project,!I!conducted!critical!buddy!sessions!with!three!members!from!the!creative!team,!and!this!

proved!useful!given!the!issues!arising!from!the!evolving!READ!model!theorisation!and!application!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

24!Templates/!exercises!included!an!evaluators’!competency!framework,!evaluation!design!exercises!based!on!Davidson’s!

Actionable!evaluation!framework!and!meta*evaluation!criteria!(Davidson,!2005),!and!Michael!Scriven’s!meta*evaluation!

checklist!(Scriven,!2011).!


! 49!

transcended!my!own!individual!practice!and!affected!multiple!stakeholders!and!the!program!

management!and!art!practice!of!the!Nuclear!Futures!initiative.!!

Data!was!also!collected!from!small!and!medium!group!reflections!(ranging!from!3!*8!people),!and!

there!were!many!of!these!sessions!integrated!into!the!creative!residency!program!associated!with!

the!READ!application.!These!sessions!are!an!example!of!a!collective!reflection!method,!and!data!was!

captured!via!note!taking,!photographs,!and!observation.!The!sessions!were!a!mix!of!structured!

(facilitated)!and!self*governing!(un*facilitated)!reflective!dialogues.!

Digital*storytelling**

Digital!storytelling!was!utilised!early!during!the!research!as!a!media!production!process!for!analysis.!

In!general!terms,!digital!storytelling!is!a!short!multi*media!production!method!used!for!telling!stories!

digitally,!often!with!an!autobiographical!or!mini*documentary!narrative!(Centre!for!Digital!

Storytelling,!2014;!Lambert,!2012;!Burgess,!2006).!Research!demonstrates!digital!storytelling!

applications!can!be!an!effective!for!learning,!professional!reflection,!knowledge*generation!and!

communication!(Couldry,!2008).25!!For!the!research,!I!employed!digital!storytelling!as!means!to!

understand!and!articulate!the!complexities!I!faced!as!an!evaluator!practitioner!at!the!site!of!practice,!

which!was!central!to!the!work!involved!in!developing!the!READ!model.!In!particular,!I!utilised!the!

method!as!a!sensemaking!process!at!the!completion!of!the!contextual!scan.!The!digital!story!and!

transcript!can!be!viewed!at:!http://nuclearfutures.org/digital*story*community*arts*evaluation!

(Password:!READ).26!!

The!digital!storytelling!technique!used!for!the!research!is!an!adaptation!from!the!original!model!

introduced!by!the!Centre!for!Digital!Storytelling!(CDS)!–!a!model!involving!the!co*creative!practice!

where!ordinary!people!use!digital!media!tools!(images,!voiceover,!music)!to!produce!a!2*3!minute!

movie!of!an!autobiographical!story,!in!partnership!with!experienced!professionals!(Centre!for!Digital!

Storytelling,!2014;!Lambert,!2012;!Lundby,!2008;!Watkins!&!Tacchi,!2008).!Like!the!CDS!model,!the!

digital!storytelling!method!used!for!the!reflective!practice!placed!emphasis!on!process,!and!utilised!

techniques!such!as!the!personal!voiceover!and!combined!visual!and!audio!material!to!convey!an!

overarching!narrative!(Lambert,!2010;!Drotner,!2008).!The!application!for!reflective!practice!

however!aligns!more!closely!to!successful!digital!story!adaptations!developed!for!Action!Research!

and!Ethnography!(as!represented!by!Burgess!&!Klaebe,!2009),!where!digital!storytelling!is!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

25!See!Hlubinka!(2003)!for!an!example!of!a!successful!education*related!application!of!digital!storytelling;!or!Kathy!

Schrock’s!extensive!list!of!applications!across!curriculum!at!http://www.schrockguide.net/digital*storytelling.html!!26!The!digital!story!was!created!as!a!private!product!of!the!reflective!practice,!rather!than!produced!for!public!audiences.!


! 50!

renegotiated!from!a!workshop*based!learning!model!to!a!research!and!documentation!method!for!

knowledge*generation!(Hartley!&!McWilliam,!2009).!!While!still!using!a!self*representational!

narrative,!the!adaptation!is!not!strictly!autobiographical!and!uses!a!systematic!and!analytical!

approach!to!the!narrative,!with!the!researcher!centrally!positioned!in!this!process!(Hertzberg!Kaare!

&!Lundby,!2008).!!

The!digital!story!produced!is!an!example!of!how!forensic!reflective!practice!involved!cycles!of!

secondary!reflection!as!a!method!of!analysis.!Experimenting!with!the!digital!story!format!allowed!me!

to!re*examine!(reflect!upon)!the!research!problem!in!response!to!the!information!arising!from!the!

contextual!scan!and!preliminary!interviews!conducted.!As!a!secondary!reflective!activity,!the!digital!

storytelling!helped!me!organise!and!synthesise!data,!identify!relevant!themes!emerging!from!the!

scholarship,!industry!documentation!and!research!participants,!and!undertake!analysis!via!scripting!

the!narrative!and!editing!the!multi!media!content!such!as!photomedia,!film!and!sound!samples.!

Critical!reflection!and!analysis!became!creative!and!iterative!processes!leading!to!the!deep!

interrogation!of!both!primary!and!secondary!research!material,!and!the!research!topic.!!

Meta8evaluation**

Given!one!of!the!central!tasks!of!the!research!was!to!critically!review!the!READ!application,!and!

assess!the!emergent!READ!approach!as!a!potential!model!of!practice!for!community!arts!evaluation,!

the!forensic!reflective!practice!needed!to!generate!data!that!would!make!explicit!the!practical!

insights!and!learnings!arising!from!the!READ!trial.!This!related!to!understanding!and!assessing!data!

generated!about!my!own!evaluation!practice!as!READ!facilitator,!as!well!as!the!collective!evaluation!

practice!and!experience!of!the!READ!model!at!the!site!of!practice!with!stakeholders.!To!generate!

knowledge!about!the!model’s!functionality!and!to!determine!the!strengths,!weaknesses!and!

potential!areas!of!innovation!demonstrated!from!the!READ!approach,!I!employed!meta*evaluation!

and!situation!analysis!methods!for!critical,!reflective!appraisal.!

The!term!meta*evaluation,!in!the!field!of!evaluation,!refers!to!the!process!of!evaluating!the!

evaluation!(Scriven,!1991).!Davidson!(2005,!p.205)!describes!meta*evaluation!as!the!‘determination!

of!the!quality!and/or!value!of!an!evaluation’.!As!is!the!case!for!evaluation!(see!Chapter!2),!meta*

evaluation!can!also!be!ambiguous!and!contentious,!however!it!does!offer!a!means!to!appraise!

evaluation!in!reasoned!and!meaningful!ways!(Davidson,!2005).!!

!

!

!


! 51!

Davidson!(2005,!p.215)!outlines!the!basic!process!for!meta*evaluation!as!follows:!

At!a!bare!minimum,!meta*evaluation!should!involve!checking!in!regularly!with!clients!to!see!

how!useful,!valid,!credible,!cost*effective,!and!appropriately!conducted!the!evaluations!have!

been!to!them.!Also!helpful!for!self*directed!meta*evaluation!is!the!exercise!of!running!

through!one!or!more!evaluation!or!meta*evaluation!checklists!to!see!how!well!each!of!the!

relevant!checkpoints!or!standards!has!been!addressed.!The!meta*evaluation!checkpoint!in!

this!case!should!consist!of!an!honest!critique!of!the!strengths!and!limitations!of!the!

evaluation!and!what!can!and!cannot!be!inferred!from!it.!

The!above!text!is!also!summative!of!the!approach!I!adopted!for!the!research!problem!(see!Chapter!6!

for!findings!and!further!explanation!of!the!meta*evaluation!criteria!used).!!

Meta*evaluation!and!situation!analysis!techniques/methods!used!in!the!reflection!sessions!included:!

•! Davidson’s!(2005)!meta*evaluation!criteria:!(a)!validity!of!conclusions,!(b)!utility!to!relevant!

stakeholders,!(c)!conduct,!(d)!credibility,!and!(e)!cost.!!

•! Scriven’s!(2003b,!2011)!Key!Evaluation!Checklist!(KEC)*!a!prescriptive!evaluation!matrix!with!

sequenced!criteria!defined!by!Scriven!as!elements!for!evaluation!best!practice.!

•! Reflective!questions!arising!at!the!site!of!practice!(informed!by!stakeholder!perspectives),!

such!as!asking!why?!or!why!not?!areas!of!READ!worked!or!didn’t,!and!what!could!have!been!

done!differently!or!better?!(see!Chapter!6,!Section!6.1.2).!

•! SWOT,!a!situation!analysis!tool!to!examine!the!strengths,!weaknesses,!opportunities!and!

threats!(SWOT)!of!the!READ!model!(an!established!business!and!project!management!

technique!that!enables!the!user!to!take!different!perspectives!on!a!product/!idea!for!the!

purpose!of!development,!assessment,!and!improvement).!!

Reflective!exercises!based!on!the!above*stated!methods!and!techniques!systematised!and!

structured!the!reflective!analysis.!With!the!data!generated,!I!fed!the!results!and!emergent!ideas!into!

the!continuous!development!and!refinement!of!the!READ!model,!striving!to!overcome!identified!

weaknesses!or!limitations!in!the!conceptualised!and/or!Balaklava*based!iteration!of!the!model.!

Likewise,!I!sought!to!identify!those!attributes!of!the!developed!model!and!its!functioning!which!are!

most!likely!to!become!successful!innovations!for!community!arts!evaluation,!scrutinising!which!

would!be!relevant!for!future!applications!for!the!Nuclear!Futures!program!and/or!beyond.!As!part!of!

the!full!complement!of!techniques!used!during!reflective!practice,!meta*evaluation!allowed!me!to!

draw!both!on!academic!scholarship!and!tacit!knowledge!from!my!professional!practice!to!enable!the!

determination!of!effective!practice!for!the!Balaklava!application,!in!order!to!clarify,!explain,!and!

justify!the!findings!extrapolated!(see!Chapter!6!and!7).!

!

!


! 52!

3.3.2! Interviews#and#consultation##

The!interviews!and!in*depth!consultation!with!partners!aimed!to!generate!data!specific!to!the!

stakeholder!collective!and!the!Nuclear!Futures!initiative,!as!well!as!identify!general!themes!relevant!

to!the!community!arts!context.!The!interviews!and!consultation!sessions!were!undertaken!before,!

during!and!after!the!application!and!involved!thirteen!participants.!!

Data!was!collected!for!the!following!three!purposes:!

•! Gaining!understanding!of!the!broader!context!of!community!arts!and!challenges!for!

evaluation!(not!related!to!the!READ!application!or!Nuclear!Futures!site!of!practice).!

•! Scoping!with!stakeholders!to!identify!the!evaluation!needs!for!the!Balaklava!Community!Arts!

Project!specifically,!within!the!broader!Nuclear!Futures!program!(prior!and!during!the!READ!

application).!

•! Obtaining!feedback!and!analysis!from!stakeholders!participating!in!the!READ!application!

about!the!emergent!READ!model!and!their!experience!of!participation.!

The!interviews!and!consultation!conducted!generated!data!for!the!overarching!DCI!methodology!

and!as!an!embedded!part!of!the!READ!application!(see!Chapter!5).27!Interviews!and!consultation!

sessions!took!on!a!variety!of!modes!across!several!phases!of!the!READ!application,!with!varying!

numbers!of!questions,!and!adaptations!to!accommodate!interviewees!(an!approach!supported!by!

Rowley,!2012).!The!sessions!conducted!to!collect!feedback!about!the!READ!application!generated!

valuable!data!for!validating!content!arising!from!the!reflective!methods!(for!triangulation).!Sessions!

were!face*to*face!or!via!Skype,!and!were!either!recorded!and!transcribed,!and/or!done!with!note!

taking.!Approximately!five!group!sessions,!each!attended!by!2*8!participants,!were!facilitated!

(corresponding!with!interview!questions).!!

Sampling**

Research!participants!were!sourced!using!techniques!including!purposeful!sampling!(i.e.!involving!

well!regarded!industry!practitioners/experts),!snowball!sampling!(i.e.!referred!research!informants)!

and!criterion!sampling!(i.e.!Nuclear!Futures/Balaklava!project!partners!and!stakeholders)!to!ensure!

participants!were!site!relevant!and/or!had!a!detailed!understanding!of!the!field/context.!Sampling!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

27!Given!that!the!content!collected!in!these!interviews!spanned!both!the!DCI!and!READ!methodologies!I!have!included!

information!about!the!interviews!in!the!dissertation!sections!related!to!both!(see!Chapter!5!for!READ!application!details).!

!


! 53!

techniques!were!informed!by!research!by!DiCicco*Bloom!&!Crabtree!(2006),!Babbie!(1992)!and!

McNeill!(1985).!Theoretical!sampling!was!used!during!the!theory!development!process!where!it!was!

necessary!to!clarify!ambiguities!as!the!READ!model!developed,!involving!consultation,!interviews,!

and!feedback!processes!with!Alphaville!(host!organisation!for!Nuclear!Futures)!and!stakeholders.!!

The!research!participants!interviewed!were!representatives!from:!

•! Government!arts!and!funding!bodies!(state!and!federal)!!

•! Peak!arts!organisations!and!cultural!institutions!

•! Community!arts!practitioners/!creative!artists/!arts!workers/!administrators!

•! Community!partners!and!community!arts!participants!

•! Evaluator!practitioners!

•! Industry!leaders/!experts/!specialists!in!the!research!area!

•! Academic!researchers.!

Question*design*for*interviews,*consultation*and*group*sessions*

Consideration!was!given!to!the!structure!and!type!of!questions!used!in!the!sessions!and!their!

implications!for!the!information!gathered!in!response!(in!terms!of!accuracy!and!validity).!The!

questions!aimed!to!collect!individual!articulations!of!evaluation!to!provide!insight!into!the!

underlying!constructs!of!participant!definitions,!and!identify!evaluation!needs!and!interests.!This!

responds!to!evidence!from!the!contextual!scan!that!for!evaluations!to!be!utilised,!they!need!to!be!

able!to!answer!the!questions!of!interest!for!the!intended!users!(Kotvojs!&!Hurworth,!2013;!Patton,!

2008).!Information!was!also!collected!on!stakeholder!experiences!of!the!READ!approach!and!the!

utility!of!the!READ!outputs!for!their!needs.!Techniques!for!interview!design!included!funnelling!

(where!interviews!begin!with!general!or!open!questions!and!move!to!more!probing!ones),!story*

telling!(using!questions!that!elicit!stories!and!examples)!and!filtering!(so!respondents!are!directed!

only!to!the!pertinent!questions)!(Kendall!&!Kendall,!2001).!!!

Question!schedules!were!modified!as!necessary!to!suit!respondents!and!setting.!Probing!and!cross!

checking!during!the!interviews!(where!necessary)!allowed!the!perspectives!being!offered!to!be!

verified!for!consistency!with!responses!with!earlier!questions.!Having!a!consistent,!base!question!

schedule!also!aided!the!later!comparative!analysis!of!interview!responses!(see!Appendix!1!for!an!

example!of!a!question!schedule!used).!(RIPP:!Article!2,!p.8!also!includes!a!summarised!rationale!

and!example!question!schedule!as!related!to!the!READ!application.)!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!


! 54!

3.3.3! Observation##

The!observation!employed!was!inspired!by!ethnographic!data!collection!applications!and!was!used!

during!field!trips,!evaluation!practice!and!in!my!role!as!READ!facilitator.!Field!trips!related!to!the!

READ!model!application!were!conducted!in!Balaklava!(SA),!Sydney!(NSW),!and!Adelaide!(SA);!with!

benefit!arising!from!concurrent!Nuclear!Futures!READ*related!projects!in!Yalata!(SA)!and!Brisbane!

(QLD).!Field!observation!was!useful!for!corroborating!the!consistency!of!participant!interview!

responses!and!reflective!practice!findings.!For!example,!observation!during!evaluation*related!

meetings!played!an!important!role!for!verifying!stakeholder!evaluation!needs,!values!and!priorities.!

In!addition!to!producing!valuable!data!for!the!research,!the!method!generated!insights!relevant!to!

broader!partnership!development!and!project!management,!which!informed!the!approach!I!adopted!

for!partner!brokerage!and!project!management.!This!conduct!aligned!with!research!by!Hartley!(2008)!

on!effective!processes!for!project!management.!

3.3.4! Evaluation#theory#and#READ#product#development#

Reflective!analysis!undertaken!during!the!research!also!involved!the!process!of!examining!emergent!

evaluation!theory!related!to!the!READ!application,!and!via!my!own!journey!as!practitioner.!This!was!

an!iterative!process!of!data!collection,!organisation,!analysis,!theoretical!sampling!and!product!

development!–!which!was!enhanced!by!deepening!cycles!of!reflective!review.!The!aim!of!developing!

a!theory!of!practice!for!the!emergent!READ!model!was!to!establish!a!transferable!and!generalised!

READ!product!suitable!for!future!evaluation!practice!for!the!next!phases!of!the!Nuclear!Futures!

program.28!!Through!reflective!practice!and!continuous!improvement!during!and!following!the!READ!

application,!evaluation!theory!at!the!site!of!practice,!and!for!my!own!professional!practice,!was!

advanced.!These!findings!are!presented!in!Chapter!6!and!7.!

Figure!3.3!shows!the!simplified!hierarchy!of!the!theory!development!process!in!relationship!to!the!

levels!of!forensic!reflective!practice!employed!during!the!research.!Across!the!diagram!(Figure!3.3),!

each!level!represents!the!refinement!of!evaluation!theory!as!I!worked!towards!developing!the!READ!

model!of!practice,!with!data!being!progressively!examined!through!increasingly!critical!stages!of!

reflection.!As!data!was!analysed!and!refined!using!qualitative!and!reflective!methods,!through!the!

process!of!secondary!reflection!and!coding,!themes!and!theoretical!concepts!emerged.!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

28!A!product!is!defined!as!‘a!good,!idea,!method,!information,!object!or!service!created!as!a!result!of!a!process!and!serves!a!

need!or!satisfies!a!want.!It!has!a!combination!of!tangible!and!intangible!attributes’!(WebFinances,!2016).!


! 55!

The!transitioning!quality!of!reflection!can!be!explained!by!Hatton!and!Smith’s!(1995)!four!stage!

taxonomy,!where!reflection!progressively!deepens!from:!

1.!Descriptive:!where!focus!of!reflection!is!limited!to!description!of!events/!activities.!

2.!Descriptive!reflective:!predominantly!descriptive,!with!deeper!consideration!of!events.!!

3.!Dialogic!reflection:!stepping!back!occurs,!evaluation!of!judgments/!actions,!and!reflective!analysis!

links!factors!and!alternative!perspectives.!

4.!Critical!reflection:!frame!of!reference!for!reflection!is!responsive!based!on!new!information,!

emergent!insights!and!time!passing,!it!considers!the!views/!motives!of!others,!in!relation!to!the!

reflective!practitioner’s!own!judgments!and!experience.!!

!

!

Figure!3.3:!Hierarchy!of!theory!development!via!forensic!reflective!practice!

!

! !

!


! 56!

Through!the!theory!building!process!for!the!READ!model!and!for!advancing!my!own!practice!

(summarised!in!Figure!3.3),!large!quantities!of!qualitative!data!were!focused!and!integrated.!Critical!

reflection!and!in*depth!analysis!were!based!on!a!number!of!interpretative!techniques!(such!as!

coded,!textual!and!comparative!approaches)!to!identify!recurrent!threads!across!the!data!collected.!

One!of!the!key!integrative!techniques!employed!was!categorisation!of!information!by!emergent!

themes,!where!data!from!reflective!methods,!interviews,!and!observation!were!coded,!organised!

and!analysed!for!iterative!development!of!the!READ!model!and!evaluation!theory.!As!results!were!

produced,!I!continued!to!question!and!analyse!findings!through!the!deepening!levels!of!reflection.!

To!assist!with!organising,!testing!and!re*evaluation!during!the!reflective!practice!cycle,!analytical!

frameworks!and!reframing!techniques!were!used!involving!the!consideration!and!analysis!of!

alternative!perspectives!and!interpretations,!based!on!proof!of!concept!discussions!with!partners.!

Practice!and!site!specific!methods!were!also!utilised!to!enhance!critical!reflection!during!theory!

building!and!product!development!(extended!from!the!READ!application!itself),!as!well!as!the!

application!of!several!key!principles!from!systems!theory!to!help!determine!the!inter*relationships!

and!boundaries!between!elements!of!the!READ!model!as!it!developed.!The!systems!theory!approach!

that!I!employed!to!understand!and!improve!the!model!is!explained!in!Chapter!5,!and!can!be!

generally!defined!as!an!approach!to!‘describe!and!interpret!complex!interactive!phenomena…!

emphasizing!the!properties!of!the!relationships!between!the!phenomena!to!be!as!significant!as!the!

properties!of!the!phenomena!themselves’!(Davidson!&!Rowe,!2009,!p.562).29!!!

Finally,!the!timing!and!delivery!schedule!for!the!reflective!theoretical!development!of!the!READ!

model!proved!important.!Scheduling!was!important!for!ensuring!a!systematic!means!of!investigation,!

and!that!adequate!intellectual!space!was!allocated!between!practice!and!reflection.!Breaks!from!

both!reflection!and!practice!were!also!beneficial,!as!sometimes!significant,!spontaneous!insights!

would!arise!in!the!hours!following!a!reflective!session,!while!I!was!doing!non*research!related!

activities,!suggesting!the!value!of!complimentary!unconscious!and!semi*conscious!unstructured!

processes!of!reflection!and!theory!building.!!

! !

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

29!The!general!systems!movement!developed!in!the!early!1950s!and!was!influenced!by!leading!thinkers!such!as!Kenneth!

Boulding,!Ludwig!von!Bertalanffy,!Talcott!Parsons,!and!Anatol!Rapoport!amongst!others!(Richardson,!2004).!


! 57!

3.4! Research#phases#

The!DCI!research!methodology!outlined!in!this!chapter!(illustrated!in!Figure!3.1)!supported!my!

evaluation!practice!for!the!READ!application!and!enabled!the!critical!reflective!investigation,!review,!

and!iterative!modification!of!the!READ!model!based!on!its!performance!for!the!Balaklava!

Community!Arts!Project.!When!implemented,!the!research!methodology!spanned!a!number!of!

distinct!phases!across!the!two!years!of!the!DCI.!The!progression!of!the!DCI!consisted!of!the!following!

six!research!phases:!

1.! A!contextual!scan!covering!the!DCI!research!topic!and!community!arts!context!(Chapter!2).!

2.! Scoping!undertaken!with!Nuclear!Futures!and!industry!stakeholders!(Chapter!5).!

3.! Planning!and!design!of!the!tailored!READ!methodology!(system)!for!Balaklava!(Chapter!5).!

4.! READ!implementation!(data!collection!and!analysis!for!the!READ!application)!(Chapter!5).!

5.! Production,!dissemination!and!use!of!READ!outputs!for!the!READ!application!(Chapter!5).!

6.! DCI!research!analysis!of!the!data!collected!on!the!READ!application!(Chapter!6).!!

Of!these!six!research!phases,!the!middle!four!phases!(phases!2*5)!were!grounded!in!the!READ!

application!and!related!to!the!planning!and!the!collective!design!and!implementation!of!a!tailored!

READ!framework!at!the!site!of!practice.30!

3.5! Ethical#considerations#

The!research!was!classified!as!low!risk!and!was!implemented!under!QUT!Ethics!Approval!Number!

1400000001.!In!implementing!the!methodology,!measures!were!taken!to!limit!the!ethical!impacts!of!

the!research!(particularly!during!interviews!and!secondary!reflection)!such!as:!maintaining!

participant!confidentiality!and/or!anonymity!where!necessary;!developing!ethical!participation!

consent!and!withdrawal!procedures;!following!approved!protocols!(individual!and!group);!and!

ensuring!participant!safety!and!comfort.!

! !

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

30!See!Figure!1.2!in!Chapter!1!for!a!visual!representation!of!the!research!phases.!


! 58!

3.6! Conclusion#

For!the!DCI,!a!robust!practice*informed!research!methodology!was!designed!to!enable!data!

collection!and!analysis!relating!to!the!development!and!first!industry!application!of!the!READ!model.!

Forensic!reflective!practice!was!harnessed!as!the!principal!research!strategy,!and!with!a!tailored!and!

systematic!approach!devised!that!involved!reflective!journaling!and!video!diary!entries,!critical!

buddy!sessions,!digital!storytelling,!interviews,!and!observation.!!Guiding!the!critical!review!of!the!

emergent!READ!model!via!these!methods!were!a!number!of!meta*evaluation!and!situation!analysis!

techniques,!employed!to!assess!the!strengths,!weaknesses!and!areas!of!innovation!demonstrated!by!

the!model.!Data!analysis!utilised!theory!development!and!creative!documentation!processes,!in!

order!to!address!the!research!problem!of!the!DCI!and!develop!a!refined!READ!product!of!relevance!

for!future!evaluation!practice.!

The!DCI!methodology!outlined!in!Chapter!3!was!implemented!throughout!the!DCI!and!formed!the!

critical!lens!for!understanding,!developing!and!evaluating!the!READ!model!in!action.!The!following!

chapters!turn!to!focus!on!the!evaluation!practice!undertaken!for!the!READ!application.!!

In!Chapter!4,!I!provide!a!descriptive!account!of!the!site!of!practice!for!the!READ!application.!Profiled!

is!the!Balaklava!Community!Arts!Project,!the!first!in!the!series!of!Nuclear!Futures!projects!

implemented!by!Alphaville!arts!organisation!with!atomic!survivor!communities.!Given!the!context*

specific!nature!of!evaluation!practice,!Chapter!4!provides!the!essential!background!information!on!

the!project!and!partnerships!ahead!of!the!documented!READ!model!development!and!application!in!

Chapter!5.!The!thorough!account!in!Chapter!5!of!the!stakeholder*oriented!design!of!the!READ!

methodology!reveals!the!theoretical!underpinnings!and!the!diverse!suite!of!methods!implemented!

and!trialled!for!the!READ!application!during!the!Balaklava!Community!Arts!Project!that!respond!to!

the!defining!project!elements,!as!canvassed!next!in!Chapter!4.!!

!

! !


! 59!

4! Site#of#practice#

The!DCI!research!methodology!in!Chapter!3!was!implemented!at!the!site!of!my!professional!practice,!

negotiated!across!the!dual!roles!of!READ!facilitator!and!program!manager!for!the!Nuclear!Futures!

Partnership!Initiative!(Nuclear!Futures).!The!READ!model!(integrating!Reflection,!Evaluation,!Analysis!and!

Documentation)!is!central!to!the!research!enquiry!and!is!a!framework!for!learning!and!appraisal!that!was!

devised!in!response!to!the!evaluation!needs!of!the!Nuclear!Futures!program!and!partners.!Nuclear!

Futures!is!a!dynamic,!international!arts!and!cultural!program!using!community!arts!processes!to!develop!

and!showcase!original!creative!work!in!partnership!with!communities!of!interest,!enabled!by!multiple!

strategic!partnerships.!The!communities!of!interest!for!the!Nuclear!Futures!program!are!atomic!survivor!

communities!(affected!by!nuclear!weapons!testing,!and!use)!in!Australia!and!several!countries!overseas.!!

While!the!READ!approach!will!be!designed!and!tested!across!the!suite!of!Nuclear!Futures!projects!

(approximately!10!sub*projects!in!total),!the!scope!for!the!DCI!research!is!contained!to!the!critical!

examination!of!READ!application!for!the!first!Nuclear!Futures!project!*!a!twelve*month!community!arts!

initiative!delivered!in!regional!Balaklava,!South!Australia,!during!2014*15.!The!findings!from!research!on!

the!Balaklava!project!will!inform!the!ongoing!evaluation!practice!for!the!Nuclear!Futures!program.!

This!chapter!provides!an!overview!of!Nuclear!Futures!(the!site!of!practice)!and!describes!the!Balaklava!

Community!Arts!Project,!detailing!the!arts!activities,!relationships!and!creative!works!under!review!by!the!

READ!model.!The!Balaklava!project!aimed!to!promote!community!participation!and!increase!opportunities!

for!residents!within!the!Wakefield!region,!north!of!Adelaide,!to!engage!with!the!arts,!and!to!create,!

develop!and!present!high!quality!creative!works!on!community*led!and!atomic!legacy!themes.!The!result!

was!a!multi*arts!creative!residency!program!with!visiting!and!local!artists!that!culminated!in!a!creative!arts!

showcase!for!the!Adelaide!Fringe!Festival!in!February!2015.!

Given!the!significant!role!that!the!site!of!practice!has!played!in!defining!the!parameters!and!

considerations!of!the!READ!application,!which!transformed!the!original!READ!idea!into!a!working!model,!

an!understanding!of!both!the!Nuclear!Futures!program!and!the!Balaklava*specific!art*making!activities!is!

necessary!in!order!to!contextualise!the!research.!In!telling!this!story,!creative!documentation!from!the!

Nuclear!Futures!community!arts!practice!is!used!in!this!chapter!to!provide!rich!representations!of!the!

community!arts!processes!and!outputs!for!which!the!READ!model!has!been!devised!to!review.!

! !


! 60!

4.1! Nuclear#Futures#Partnership#Initiative#(Nuclear#Futures)#

4.1.1! Background#

Nuclear!Futures!is!a!three*year!collaboration!across!six!countries!involving!linked!multi*arts!and!

cultural!activity!with!atomic!survivor!communities,!to!create!original!story*based!artworks!exposing!

the!legacies!of!the!atomic!age!(Alphaville,!2015).31!!It!is!coordinated!by!Alphaville,!a!Sydney*based!

company!specialising!in!multi*platform!creative!arts,!originating!in!community!development!

processes.!Alphaville’s!program!documentation!describes!Nuclear!Futures!as!follows:!!

The!initiative!creates!new!partnerships!with!arts!organisations,!and!with!educational!and!

cultural!institutions,!research!facilities,!campaign!groups!and!atomic!survivor!and!nuclear!

veteran!communities!in!Australia,!Britain,!Japan,!Marshall!Islands,!Kazakhstan!and!India.!

Through!the!program,!communities!and!artists!use!theatre,!film,!paintings,!sculpture,!

photography,!digital!arts!and!other!art!forms!to!make!creative!works!that!reflect!both!the!

horror!of!living!with!nuclear!radiation,!and!the!resilience!of!communities!as!they!face!the!

nuclear!future.!(Extract!from!Nuclear!Futures!program!documentation,!courtesy!of!Alphaville)!

Led!by!Creative!Producer!Paul!Brown,!Alphaville!secured!support!for!Nuclear!Futures!in!2013!from!

the!Australia!Council!for!the!Arts!via!Creative!Communities!Partnership!Initiatives!(CCPI)!funding!for!

the!three*year!period!2014*2016!(see!Section!2.1.2).!Positioned!within!the!community!partnerships!

sector!of!the!community!arts!field,!the!funding!was!linked!to!program!objectives!to!promote:32!

!! individuals!and!communities!to!actively!participate!in!excellent!artistic!practices�!!! long!term!outcomes!through!arts!and!non*arts!partnerships!

!! capacity!building!in!communities�!

!! innovation!across!agencies!and!communities!in!the!use!of!the!arts!for!community!well!being�!!! understanding!of!the!contribution!of!the!community!arts!and!cultural!development!sector!

(CACD)!through!effective!evaluation!and!dissemination.!

Program!funding!was!also!secured!from!Arts!South!Australia!and!philanthropy,!and!extensive!in*kind!

support!was!received!from!arts!and!non*arts!organisations,!local!government,!industry,!and!

academic!institutions.!Partnering!organisations!for!Nuclear!Futures!includes!Maralinga!Tjarutja!

Council,!Yalata!Aboriginal!Community,!Global!Hibakusha!Project,!Atomic!Photographers!Guild,!British!

Nuclear!Test!Veterans!Association,!Australian!Nuclear!Veterans!Association!network,!Wakefield!

Regional!Council,!State!Library!of!South!Australia,!Allen!&!Unwin!publishing,!In!Place!of!War,!

Manchester!University,!Winchester!University,!Murdoch!University,!Queensland!University!of!

Technology,!University!of!Western!Sydney,!and!University!of!New!South!Wales!(Alphaville,!2015).!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

31!See!promotional!video!for!a!4!minute!summary!of!the!Nuclear!Futures!program:!http://nuclearfutures.org!!

32!This!content!is!sourced!from!program!documentation,!courtesy!of!Alphaville.!Please!note:!some!Alphaville!content!from!

Nuclear!Futures!documentation!and!in!the!public!domain!(website)!used!in!this!dissertation!has!been!authored!by!myself,!

on!behalf!of!Alphaville!in!my!role!as!program!manager,!and/or!in!collaboration!Paul!Brown,!the!other!major!author!of!

Alphaville!content.!


! 61!

4.1.2! Exposing#the#legacies#of#the#atomic#age#through#the#creative#arts##

While!international!themes,!activities!and!linkages!are!prominent!across!the!Nuclear!Futures!

projects,!the!program!is!primarily!focused!on!developing!new!creative!works!that!share!community!

stories!of!the!legacies!arising!from!atomic!testing!in!Australia!in!the!1950s!and!60s,!a!history!that!

remains!largely!unknown!to!most!Australians.!!Between!1952!and!1967,!with!the!agreement!of!Prime!

Minister!Menzies,!twelve!atomic!bomb!tests!and!hundreds!of!toxic!minor!trials!were!carried!out!by!

British!scientists!in!Australia!(McClelland,!1985).!The!British!run!tests!were!performed!in!the!Monte!

Bello!islands!off!Western!Australia,!and!at!Emu!Field!and!Maralinga!in!South!Australia!(National!

Archives!of!Australia,!2013)!(see!Figure!4.1).!The!impacts!of!these!tests!continue!to!play!out!in!the!ill!

health!of!the!Indigenous,!nuclear!veteran,!and!British!scientific!communities!exposed!to!or!involved!

in!the!tests,!and!have!left!a!legacy!of!long!term!environmental!contamination!(Yalata!&!Oak!Valley!

Communities!&!Mattingley,!2009;!Cross!&!Hudson,!2006).!!

!

!

Figure!4.1:!Map!of!South!Australia!and!relevant!place*names,!!

including!Emu!Junction!and!Maralinga!atomic!test!sites!

!

(Source:!Yalata!and!Oak!Valley!Communities!and!Mattingley,!2009,!p.68)!

!


! 62!

4.1.3! Nuclear#Futures#program#rationale##

The!impulse!to!work!with!atomic!survivor!communities!via!Nuclear!Futures!arises!from!the!urgency!

to!work!with!the!last!surviving!eye!witnesses!to!the!world’s!atomic!tests,!and!the!desire!to!contribute!

towards!capacity!building!and!recognition!for!nuclear!affected!communities.!It!fits!into!a!long!

tradition!of!artists!responding!to!nuclear!legacies,!atomic!weaponry,!and!issues!from!the!atomic!age!

(internationally!and!within!Australia)33.!The!Nuclear!Futures!initiative!also!builds!on!earlier!work!by!

Alphaville,!namely!a!verbatim!theatre!and!oral!history!project!with!nuclear!veterans!in!Britain!and!

Australia!that!culminated!in!the!script!development!of!the!play!Half%a%Life,!and!performances!in!

Australia!and!the!United!Kingdom!in!2005!and!2006!(see!Brown,!2010,!2007,!2006;!Arvanitakis,!2008).!!

As!stated!in!Alphaville’s!program!documentation,!the!Nuclear!Futures!initiative!has!the!following!

broad!aims:!

(i)!! To!make!the!story!go!far!

(ii)!! To!develop!the!storytelling!networks,!sharing!stories!of!resilience!

(iii)! To!make!the!link!between!‘white’!and!‘black’!experience!

(iv)! To!collaborative!with!and!support!initiatives!by!nuclear!veterans!and!their!families!

(v)! To!dramatise!the!role!of!British!Scientists!

(vi)!! To!build!new!international!collaboration!and!join!the!Global!Hibakusha!project!

(vii)!!To!explore!areas!of!interest!within!the!CACD!field!

(viii)!To!undertake!independent!evaluation!through!data!collection;!qualitative!and!rich!media;!

structured!feedback!and!review;!critical!reflection;!regular,!rich!reporting.!

Creative!Producer!Paul!Brown!explains,!‘Nuclear!Futures!seeks!to!explore!how!experience!of!the!

bomb!translates!into!community!development!and!international!campaigns!for!peace!and!

disarmament,!and!responds!to!the!atomic!legacies!through!community!arts!and!partnerships’.34!

Central!to!the!art*making!is!the!question:!‘What!creative!art!works!and!processes!can!endure!across!

the!deep*time!of!the!nuclear!future?’!(Alphaville,!2015).!

For!delivery,!Nuclear!Futures!was!devised!as!a!sequence!of!four!inter*related!CACD!activities:!

•! community!and!place!based!residencies!(in!Balaklava!and!Yalata/Oak!Valley,!South!Australia)!

•! integrating!arts!and!cultural!projects!across!the!life!of!the!initiative!(international)!!

•! showcases!of!the!program’s!linked!creative!works,!with!associated!events!(international)!

•! support!structures!for!program!engagement,!planning,!coordination!and!evaluation.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

33!There!is!a!significant!collection!of!creative!works!previously!produced!responding!to!atomic!issues.!Two!well*known!

Australian!examples!include!the!theatre!play!Ngapartji%Ngapartji!(Big!hART,!see!http://www.ngapartji.org/)!and!Maralinga!Sculpture!by!Lin!Onus!(Art!Gallery!of!Western!Australia). 34!Sourced!from!program!documentation,!courtesy!of!Alphaville.!


! 63!

These!four!key!program!activity!strands!are!summarised!in!Table!4.1,!with!the!Balaklava!Community!

Arts!Project!highlighted!in!relation!to!the!other!Nuclear!Futures!activities.!!

!

Table!4.1:!Key!activity!strands!for!the!Nuclear!Futures!program!(2014*16)!

!

NUCLEAR+FUTURES+++H+++KEY+ACTIVITY+STRANDS+

Community+and+place+

based+residencies+

Integrating+arts+and+

cultural+projects+

Public+presentation+

and+showcasing++

Support+structures+

Balaklava+project+

Regional!South!Australia+

(CACD!residencies,!

photography,!script!

development,!creative!

development!for!showcase)!

International+youth+

leadership+workshops++

(Digital!storytelling!skills!

development!and!!exchange!!

for!3rd!generation!atomic!

communities!participants)!

Community+

presentations+

(Presentations!of!creative!

works!to!and!in!communities!

of!interest)!

Nuclear+Futures+hub+

(Management!systems!for!

creative!team,!steering!and!

working!groups,!cultural!

liaisons/!advisers)!

Maralinga+project+

Yalata!&!Oak!Valley,!SA+

(CACD!residencies,!cultural!

exchange,!storytelling,!digital!

arts,!sculptures,!creative!

development!for!showcase)!

Online+community+

(Involves!social!forum!and!

exchange,!film/!webisodes,!

website,!games,!transmedia)!

Public+multimedia+events+

(International!workshops,!

film!screenings,!launches,!

festivals,!exhibitions,!

installations)+

Collaborative+networks+

(Communities!of!practice,!

and!circle!of!advisers,!

academics,!creative!artists!

and!other!researchers)!

!

SpinHoff+projects+

(Nagasaki!Peace!Park!

sculpture!gifting,!play!script!

development!with!Monte!

Bello!focus,!radio!play!with!

British!veterans,!Maralinga!

cultural!tourism)!

The+archive+

(Collection!development!

with!cultural!institutions!and!

creative!documentation)!

Integrated+showcases+

(National!and!international!

touring!of!curated!creative!

works/!showcases!to!

regional!and!metropolitan!

audiences)+

Program+and+partnership+

development+

(Includes!building!creative!

community!partnerships!

with!arts!and!non*arts!

partners!and!fundraising)!

! ! Scholarly+&+industry+

presentations+

(Conferences,!seminars,!

forums)!

READ+

(Implementation!of!the!

READ!model!integrating!

Reflection,!Evaluation,!

Analysis,!Documentation)!

!(Table!by!Ellise!Barkley.!Sourced!courtesy!of!Alphaville)!

Through!the!four!activity!strands!in!Table!4.1,!Nuclear!Futures!was!designed!to!produce!a!suite!of!

new!multi*platform!creative!works,!representing!a!register!of!voices!from!various!atomic!survivor!

communities.!New!creative!works!produced!as!a!result!of!the!program!will!include!digital!stories,!

theatre!plays,!books,!a!radio!play,!exhibitions/!installations,!sculpture,!online!community,!integrated!

showcase!and!an!enduring!archive!of!historical!and!creative!materials.!Community!artworks!and!

creative!works!produced!in!partnership!with,!and!independently!by,!professional!artists!will!be!

curated!into!an!international!showcase!planned!for!2016!onwards,!following!the!presentations!in!

communities!completed!in!2015.!(See!www.nuclearfutures.org!for!further!program!information.)!


! 64!

4.2! Balaklava#Community#Arts#Project#

The!Balaklava!Community!Arts!Project!was!the!first!of!two!major!community!and!place*based!

projects!implemented!for!Nuclear!Futures.!The!second!was!a!series!of!creative!projects!undertaken!

with!the!Yalata!Anangu!Aboriginal!community!in!far!west!South!Australia!(see!Figure!4.1).!!

4.2.1! Origins#and#inspiration#

Research!for!the!Balaklava!Community!Arts!Project!began!in!December!2013!with!nuclear!veteran!

Avon!Hudson!and!his!daughter!Kate,!both!residents!of!the!small!regional!community!of!Balaklava.!

Balaklava!is!an!agricultural!township!in!the!Mid!North!Region!of!South!Australia,!situated!92!km!

north!of!Adelaide.!It!has!a!population!of!under!2000!residents!and!is!a!cultural!hub!within!the!

surrounding!Wakefield!regional!area!(see!Figure!4.2)!(Wakefield!Regional!Council,!2015).!!

!

!

!

!

!

!

!

!

!

!

!

!

!

Figure!4.2:!Map!of!Balaklava!and!Wakefield!Regional!Council!area,!South!Australia!

!

(Source:!http://profile.id.com.au/wakefield/about?WebID=10)!!

!

!

!

Balaklava!


! 65!

The!project!began!in!Avon’s!Balaklava!living!room!amidst!his!private!collection!of!rich!and!rare!

archival!materials!relating!to!the!Australian!atomic!tests,!amassed!from!decades!of!social!justice!

campaigning!on!this!issue!(see!image!below).!Avon!had!served!in!the!Royal!Australian!Air!Force!

(RAAF)!at!Maralinga!in!the!early!1960s,!and!blew!the!whistle!in!the!1970s!when!he!publicly!revealed!

Maralinga’s!deadly!legacy,!an!act!that!contributed!to!the!establishment!of!the!Royal!Commission!into!

British!Nuclear!Tests!in!Australia!(1984*85).!Over!sixty!years,!Avon!has!been!a!prominent!antinuclear!

campaigner!and!advocate!for!Australia’s!nuclear!veteran!community,!and!was!a!leader!within!the!

Australian!Nuclear!Veterans!Association!(ANVA).!His!nationally!significant!private!archive!remains!an!

important!legacy!of!his!activism!and!was!a!cornerstone!resource!for!the!community!arts!project.!

!

Balaklava!community!partner,!nuclear!veteran!and!whistleblower!Avon!Hudson.!!

Photo!by!Jessie!Boylan.!

!

It!was!in!Balaklava,!Avon’s!hometown,!that!the!CACD!activities!and!community!showcase!for!the!

project!were!delivered.!Avon’s!energy!for!participating!in!the!proposed!project,!his!extensive!

knowledge!of!nuclear!veteran!related!subject!matter!and!networks,!and!his!generosity!for!sharing!his!

testimonies!via!oral!history,!were!formative!factors!for!grounding!the!project!in!Balaklava.!!

+

!


! 66!

4.2.2! Project#partners#and#stakeholders##

The!process!of!developing!the!creative!community!partnerships!for!the!Balaklava!Community!Arts!

Project!was!facilitated!by!arts!company!Alphaville!and!targeted!relationships!with:!relevant!partner!

organisations!and!communities!of!interest!(to!be!directly!associated!and!involved!in!the!development!

and!delivery!of!the!project);!funding!partners!(contributing!financially!to!the!project);!stakeholders!

(those!potentially!affected!by!or!with!a!stake!in!the!project!in!some!way);!and!potential!project!

champions/!allies!(strategic!alliances!to!help!promote,!support,!and!endorse!the!project).!The!project!

was!financially!supported!by!the!Australia!Council!for!the!Arts!and!Arts!SA,!and!involved!core!

partnerships!with!community!representatives!within!the!Australian!nuclear!veteran!network!in!South!

Australia!and!organisations!such!as!the!Wakefield!Regional!Council,!the!Council’s!Youth!Advisory!

Committee,!and!State!Library!of!South!Australia!(RIPP:!Article!28,!p.106!is!a!partner!summary).!!

Community!consultation!with!stakeholder!groups!such!as!local!government,!regional!arts,!

progress!committees,!and!local!cultural!and!youth!service!organisations!was!undertaken!during!

the!planning!phase!so!that!the!arts!activities!devised!aligned!with!regional!strategic!objectives!and!

local!priorities.!Contact!was!also!made!with!nuclear!veteran!communities!in!Australia!and!overseas,!

via!known!Australian!networks!and!the!British!Nuclear!Test!Veteran!Association.!It!was!established!

that!the!project!would!target!two!key!communities!of!interest:!

1.! South!Australian!nuclear!veteran!networks,!with!Avon!and!Kate!as!the!key!community!

partners.!

2.! Regional!participants!and!the!Balaklava!community,!with!a!focus!on!engaging!young!people!

in!the!Wakefield!region.!!

During!implementation,!partnership!brokerage!and!stakeholder!engagement!continued!across!all!

project!phases.!The!partnerships!with!local!and!regional!arts!and!non*arts!organisations!brought!

significant!in*kind!contribution,!with!an!estimated!value!of!$40,000.!This!included:!expertise!in!

community!cultural!development;!event!management;!oral!history;!archiving;!mentoring/training;!

and!access!to!local!arts!and!cultural!programs,!networks,!community!groups,!equipment!and!venues.!

In!addition!to!the!core!partners!listed!above,!organisations!contributing!in*kind!support!included!the!

Balaklava!Courthouse!Gallery,!Balaklava!Town!Hall,!Balaklava!Men’s!Shed,!Vitalstatistix!(Port!

Adelaide),!University!of!Adelaide,!British!Nuclear!Test!Veteran!Association,!and!the!Blackman!Gallery!

(Clare,!SA).!Avon!Hudson’s!established!veteran!and!media!networks!also!proved!very!valuable!for!

extending!partnership!development!and!advancing!networking!with!several!South!Australian!nuclear!

veteran!families!(initially!unknown!to!the!project).!!

!

!


! 67!

4.2.3! Project#overview#

The!Balaklava!project!aimed!to!promote!community!participation!and!increase!opportunities!for!

residents!within!the!Wakefield!region!to!engage!with!the!arts,!achieve!long*term!outcomes,!and!

encourage!innovation!between!arts!and!non*arts!partners!(see!objectives,!p.60).!For!the!project,!this!

involved!creating,!developing!and!presenting!quality!creative!works!on!community*led!and!atomic!

legacy!themes!(particularly!nuclear!veteran!themes!relating!to!South!Australia’s!atomic!history).!As!

per!the!funding!and!partner!agreements,!the!Balaklava!Community!Arts!Project!set!out!to!deliver:!

•! An!arts!residency!using!CACD!processes,!to!foster!local!cultural!development!

•! Collaborative!and!community*driven!art*making!!

•! Showings!of!the!work*in*progress!to!communities!and!stakeholders!

•! A!community!showcase!of!created!work!–!a!contemporary!multi*arts!work!of!excellence!

•! An!accessible!and!enduring!archive!of!the!evidence!on!which!creative!works!are!based!

•! Findings!based!on!Reflection,!Evaluation,!Analysis!and!Documentation!(READ).!

A!core!creative!team!was!assembled!to!work!with!the!communities!of!interest.!The!Nuclear!Futures!

team!included!Director!Teresa!Crea,!Photomedia!Artist!Jessie!Boylan,!Playwright!John!Romeril,!Digital!

Artist!Linda!Dement,!Projection!Designer!Nic!Mollison,!Sound!Designer!Luke!Harrald,!Creative!

Producer!Paul!Brown,!Production!Manager!Ellise!Barkley,!Audio*visual/Projection!Technician!Rowan!

Lee!and!Local!Coordinator!Carly!Friedrichs.!!

Conceived!were!a!mix!of!creative!residencies!to!be!based!in!Balaklava!involving!visiting!and!local!

artists,!as!well!as!creative!development!and!project!work!conducted!in!Adelaide!and!Sydney!to!

support!the!residencies!and!art*making.!Five!parallel!community!arts!initiatives!proceeded:!!

1.!Photography!skills!development!workshops!and!a!community!exhibition,!facilitated!by!

Photomedia!Artist!Jessie!Boylan!and!local!Photographer!Lisa!Redpath.!

2.!Script!development!for!a!new!play,!led!by!playwright!John!Romeril.!

3.!A!multi*media!immersive!projection!installation,!led!by!Director!Teresa!Crea.!

4.!A!co*curated!photomedia!and!archival!exhibition,!including!artworks!by!Jessie!Boylan.!

5.!Archive!development!and!oral!history!collection!with!State!Library!of!South!Australia!(SLSA).!

Across!March!2014!*!February!2015,!fifteen!professional!creative!personnel!were!contracted!for!the!

CACD!residencies!and!final!showcase!in!Balaklava!(nine!of!the!artists!and!technicians!were!South!

Australian).!Additional!personnel!were!also!involved!via!partner!organisations,!such!as!staff!from!

Wakefield!Regional!Council!and!Collection!Development,!SLSA.!


! 68!

The!creative!partnership!established!between!participating!artists!and!community!developed!over!six!

week*long!residencies,!as!well!as!extended!rehearsal!and!production!periods.!The!artist!in!residence!

program!utilised!community!arts!processes!such!as!co*creative!art*making,!storytelling,!public!

engagement,!skills!development,!community!research,!network!building,!mentoring!and!professional!

development*!involving!participants!of!all!ages.!Creative!development!was!undertaken!via!

workshops,!meetings,!oral!history!collection,!interviews,!field!research,!networking!and!immersion!in!

Avon!Hudson’s!private!archives.!Representatives!from!Australia’s!nuclear!veteran!community,!young!

people,!the!wider!Wakefield!community,!and!professional!artists!from!Balaklava,!South!Australia!and!

other!states,!were!engaged.!

The!creative!works!developed!were!presented!back!to!communities!of!interest,!and!publicly!to!

general!audiences.!Presentation!of!new!works!included!two!gallery!exhibitions:!Field%of%View%and!

Portrait%of%a%Whistleblower;!and!a!dynamic!multi*arts!showcase!10%Minutes%to%Midnight%for!the!2015!

Adelaide!Fringe!Festival,!featuring!an!assemblage!of!immersive!projection,!digital!artwork,!

photomedia,!sound!design,!archival!material,!interwoven!with!a!lively!speakers’!program.!Linked!to!

this!was!a!two*day!schools!program!to!engage!local!students.!In!Balaklava,!the!project!engaged!

approximately!400!residents!as!participants!and/or!audience.!!

The!aforementioned!project!activities,!outputs!and!partnerships,!resulted!in!myriad!personal,!social!

and!cultural!outcomes!and!impacts!via!the!community!arts!processes!and!exchange.!The!socio*

dynamic!pulse!underpinning!the!collaborative!creative!practice!was!driven!by!an!iterative!process!of!

learning,!sharing!and!responding!to!the!sensitive!political!and!emotional!subject!matter!of!Australia’s!

atomic!test!legacies,!as!well!as!reflection!on!locally!relevant,!place*based!themes!as!part!of!Field%of%

View.!The!result!was!a!sliding!scale!of!transformative!and!personal!impacts!for!participants!and!the!

creative!team,!as!well!as!benefits!for!the!wider!Balaklava!community.!

Across!the!project,!media!coverage!profiled!significant!milestones!and!events!(in!publications!such!

as!BAside%Magazine,!The%Advertiser,%Plains%Producer,%Northern%Angus,%The%Clothesline!(online)!and!on!

Radio!Adelaide!and!ABC!891).!Mainstream!media,!social!media!and!community!newsletters!were!

used!to!profile!the!creative!development!process,!regional!arts!events,!participant!stories,!and!

showcase!highlights.!Overall,!the!media!coverage!extended!the!public!storytelling!platform!for!

Nuclear!Futures’!community!partners!and!artists,!and!included!feature!articles!on!individual!

community!and!team!members,!and!journalism!on!Australian!atomic!test!history!and!its!

contemporary!socio*political!context.!

!


! 69!

4.2.4! Summary#of#the#Balaklava#community#arts#practice##

This!section!elaborates!on!the!art*making!processes,!activities!and!partnerships!at!the!site!of!practice!

in!relation!to!the!key!project!elements!for!the!Balaklava!Community!Arts!Project.!Creative!

documentation!from!the!arts!practice!of!visiting!and!local!artists!is!included.!

Summarised!over!the!following!pages!are!snapshots!of:!

I.! What’s%the%Story?!Creative!arts!pop*up!

II.! Photography!skills!development!and!community!exhibition!

III.! Script!development!

IV.! Portrait%of%a%Whistleblower,%curation!and!exhibition!

V.! 10%Minutes%to%Midnight!installation!

VI.! Public!showcase:!Adelaide!Fringe!Festival!!

VII.! Secondary!schools!program!

VIII.! Archive!development!and!oral!history!collection.!

+

!

! !


! 70!

i.! What’s+the+Story?+Creative+arts+popHup+(17H25+March,+2014)++

Activities!included!a!storytelling!workshop,!film!screening,!school!visits/!workshops!with!

students,!shop*front!art!displays,!community!archive!mentoring,!and!a!local!business!portrait!

series!where!professional!photos!were!taken!and!gifted!back!to!participants.!The!residency!

was!delivered!by!4!artists/art!workers,!community!partners,!and!SLSA!staff.!The!activities!

were!designed!to!initiate!community!engagement!via!meet*the*artists!events,!relationship!

building, skills!development!and!community!visioning.!

!

A.!Playwright!John!Romeril!with!senior!

Drama!students!at!Balaklava!High!School.!!!

B.!Example!from!the!pop*up!exhibition!of!

atomic!artwork!by!Jessie!Boylan!displayed!in!

Balaklava!shopfronts.!

C.!Creative!arts!pop*up!branding.!

Photos!by!Jessie!Boylan.!

!

!

!

A.+ B.+

C.+


! 71!

ii.! Photography+skill+development+and+community+exhibition+(June+–+November,+2014)+

Photographers!Jessie!Boylan!(visiting!artist)!and!Lisa!Redpath!(Balaklava)!teamed!up!to!run!

two!free!workshops!with!27!participants!for!photography!skills!development.!An!evening!

workshop!targeted!general!community!participants,!and!an!all!day!field!trip!for!young!people!

was!also!held.!Workshop!participants!were!invited!to!submit!entries!for!a!community!

photography!exhibition,!and!artists!Boylan!and!Redpath!co*curated!the!selected!works!

culminating!in!the!Field%of%View!exhibition!held!at!the!Courthouse!Gallery,!25th!September!*!

9th!November,!2014.!The!original!photography!by!local!and!young!creatives!included!diverse!

and!stunning!images,!responding!to!sites!that!have!been!part!of!their!field!of!view!while!

living,!or!growing!up!in!the!Wakefield!region.!The!exhibition!was!officially!opened!by!the!

Mayor!of!Wakefield!Regional!Council!and!was!strongly!supported!by!Wakefield!Regional!

Council!and!their!Youth!Advisory!Committee.!!

!

!

!

! !

A.!Excerpt!from!promotional!poster,!Field%of%View!photography!exhibition.!!Design!by!Ellise!Barkley.!

B.!Workshop!#1!participants.!!

Photo!by!Lisa!Redpath.!

C.!Workshop#2!participants!during!field!

excursion.!Photo!by!Carly!Friedrichs.!

!

!

!

A.+ B.+

C.+


! 72!

iii.! Script+development+(December+2013+–+March+2015)+

!

!

!

!

!

!

!

!

!

!

Writer!John!Romeril!and!Avon!Hudson!worked!together!over!several!residencies!in!2014!at!

Avon's!home!in!Balaklava!as!part!of!the!creative!research!and!script!development!process.!As!

part!of!the!fieldwork,!Romeril!accompanied!Hudson!on!the!2014!Radioactive!Exposure!Tour!

(organised!by!Friends!of!the!Earth)!of!nuclear!sites!in!remote!South!Australia,!to!hear!Hudson!

as!guest!presenter!sharing!his!testimony!and!extensive!historical!knowledge!of!nuclear!

testing!in!Australia.!Director!Teresa!Crea!and!artist!Jessie!Boylan!were!involved!in!the!early!

concept!development!for!the!script,!and!Creative!Producer!Paul!Brown!was!a!key!

collaborator!across!the!drafting!process.!In!developing!the!original!dramatisation,!

documentary,!verbatim!and!multi*media!art!techniques!were!used.!

The!draft!script!developed!was!a!dynamic,!operatic!piece,!inspired!by!the!personal!and!

intergenerational!stories!of!nuclear!veterans!and!their!families.!Early!iterations!of!the!script!

were!formative!in!the!concept!development!for!the!10%Minutes%to%Midnight!projection!

installation!(see!below),!and!closely!linked!with!the!SLSA!supported!community!archive!

activities.!The!script!was!workshopped!in!August!2015,!with!two!actors!and!a!Director,!

involving!Drama!students!and!staff!at!Queensland!University!of!Technology!(QUT)!as!part!of!a!

Nuclear!Futures!exhibition!and!event!program!at!The!Block,!QUT.!

!Creative!research!during!an!artist!residency!in!Balaklava.!Photo!by!Jessie!Boylan.!


! 73!

iv.! Portrait'of'a'Whistleblower,+curation+and+exhibition+(October+2014+–+February+2015)+%

%

%

%

%

%

%

%

%

%

%

%

%

% %

%

Portrait%of%a%Whistleblower!chronicles!Avon!Hudson’s!life,!combining!contemporary!

photographs!and!video!works!by!artist!Jessie!Boylan!with!pieces!from!Avon’s!private!archive!

of!rare!documentation,!reports,!testimonies,!news!clippings,!photographs,!maps!and!

artefacts!that!make!up!his!life’s!work.!The!exhibition!was!co*curated!over!several!residencies!

by!Jessie!Boylan,!Ellise!Barkley,!Avon!Hudson!and!Paul!Brown,!and!was!programmed!within!

the!2015!Adelaide!Fringe!Festival.!Officially!opened!on!Friday!13th!February,!2015,!the!

exhibition!ran!over!six!weeks!until!29th!March!and!was!well!attended!by!local,!regional!and!

interstate!audiences.!The!exhibition!is!planned!to!tour!in!Australia!and!internationally!from!

2015!onwards!via!the!Nuclear!Futures!program.!

! !

!

A.!Example!of!archival!

material!included!in!the!

exhibition.!Photo!of!Avon!

Hudson!in!front!of!

Lockheed!Neptune!

aircraft,!Darwin!Airbase,!

NT,!1958.!Photographer!

unknown.!

!

B.!Photomedia!Artist!

Jessie!Boylan!and!Avon!

Hudson!in!the!

Courthouse!Gallery,!

Balaklava,!sitting!

amongst!Portrait%of%a%Whistleblower.!Photo!by!Tania!Safi.!

!

!

!

A.+

B.+


! 74!

v.! 10'Minutes'to'Midnight+projection+installation+(July+2014+–+February+2015)+!

!

!

!

!

!

! !!

A.!Inside!the!immersive!

projection!installation.!Digital!

artwork!by!Linda!Dement.!!

Photo!by!Jessie!Boylan.!

B.!Projection!mapping!and!

technical!set*up.!!

Photo!by!Nic!Mollison.!

C.!External!view!of!custom!built,!

surround!projection!screen.!!

Photo!by!Tania!Safi.!

D.!Projected!video!art!by!Jessie!

Boylan.!Photo!by!Tania!Safi.!

!

! ! !

A.+

B.+ C.+ D.+

A!multi*media!projection!installation!was!developed!to!offer!audiences!an!immersive!

experience!of!South!Australia’s!atomic!test!stories,!integrating!rare!archival!footage!and!oral!

history!with!original!digital!artwork,!projection!and!sound!design.!The!24*minute!creative!

work!applied!cutting!edge!immersive!techniques!to!create!a!dramatic!‘countdown’!to!the!

atomic!bomb!tests!conducted!at!Maralinga!and!Emu!Field.!The!projection!installation!was!

developed!over!seven!months!with!a!team!of!leading!Australian!artists,!at!Balaklava!and!at!

Vitalstatistix!in!Port!Adelaide.!Creative!development!was!led!by!Teresa!Crea,!a!specialist!in!

audience!immersive!techniques,!and!evolved!in!association!with!the!script!development!and!

community!archive!projects.!!

The!creative!team!involved!Teresa!Crea!(Director,!Dramaturg),!Nic!Mollison!(Projection,!

Lighting!&!Set!Designer),!Linda!Dement!(Digital!&!Multi*media!Artist),!Luke!Harrald!(Sound!

Designer!&!Composer),!Jessie!Boylan!(Photomedia!&!Video!Artist),!John!Romeril!(Research)!

and!Avon!Hudson!(Community!Partner).!Production!team!included!Paul!Brown!(Creative!

Producer),!Ellise!Barkley!(Production!&!Event!Manager),!Carly!Friedrichs!(Local!Coordinator),!

Rowan!Lee!(Programmer!&!Technician),!Craig!Clifford!(Set),!and!Amanda!Brown!(Seamstress).!

The!work!created!constitutes!a!core!element!of!the!future!Nuclear!Futures!showcase.!

!


! 75!

vi.! Public+showcase:+Adelaide+Fringe+Festival+(13H14+February,+2015)++

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

A!large*scale!community!showcase!for!

Wakefield!regional!audiences!was!organised!

to!present!back!to!community!the!story*based!creative!works!developed!through!the!

Balaklava!Community!Arts!Project.!Titled!10%Minutes%to%Midnight,!the!2015!Adelaide!Fringe!

Festival!ticketed!event!ran!across!two!venues!and!included!the!immersive!projection!

installation,!Portrait%of%a%Whistleblower!exhibition!and!a!Speakers!Corner!involving!

representatives!from!atomic!survivor!communities,!experts!and!creative!artists.!The!

assemblage!of!original!creative!works!was!the!culmination!of!the!twelve*month!partnership!

between!the!creative!team,!community!partners!and!participating!organisations!across!all!of!

the!Balaklava!projects.!10%Minutes%to%Midnight!was!attended!by!over!140!people,!and!

attracted!local,!Adelaide!and!interstate!audiences.!The!event!provided!the!opportunity!for!

the!new!creative!works!to!be!presented!to!mainstream!community!audiences,!and!was!a!rare!

networking!opportunity!for!attending!atomic!survivors!and!families.! !

!

! !

!

!

A.!Speaker!panel.!

B.!Speakers!Corner!session.!

C.!Audience!visiting!Courthouse!Gallery.!

D.!Nuclear!veteran!and!descendent!exchange.!

E.!Archive!materials!were!brought!by!nuclear!

veteran!audience!members!for!sharing.!

Photos!by!Tania!Safi.!

!

A.+ B.+

C.+ D.+

E.+

D.+


! 76!

vii.! Secondary+schools+program+(10/12+February,+2015)+

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

Over!two!full!days,!a!tailored!education!and!engagement!program!was!offered!for!local!

secondary!schools.!The!program!engaged!220!local!students!from!two!Balaklava!high!schools,!

and!linked!with!the!curriculum!for!Design!and!Technology,!History,!Art!and!Tourism.!Students!

had!the!opportunity!to!view!the!projection!installation,!learn!about!Australia’s!nuclear!test!

history,!have!a!guided!tour!of!the!Whistleblower%exhibition,!hear!eyewitness!accounts!of!the!

atomic!experiments,!and/or!meet!members!of!the!creative!team!to!discuss!the!art*making!

and!technical!aspects!of!the!showcase.!Discussion,!questions!and!feedback!indicated!that!for!

many!students!this!was!the!first!time!they!had!heard!of!Australia's!atomic!test!history.!

!

!

Local!secondary!school!students!attending!10%Minutes%to%Midnight.!Photos!by!Tania!Safi.!


! 77!

viii.! Archive+development+and+oral+history+collection+(March+–+May+2015)+

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

Running!in!parallel!with!the!afore*mentioned!arts!activities!was!a!community!archives!

project,!supported!and!mentored!by!SLSA.!The!archiving!and!oral!history!work!initiated!the!

process!of!curating!Hudson’s!private!collection!to!determine!pathways!for!the!future!public!

storage!of!the!material.!It!was!one!of!several!legacy!initiatives!within!the!Nuclear!Futures!

program,!to!support!public!and!long*term!access!to!the!significant!material!and!resources!

unearthed!during!the!creative!research.!The!archives!project!facilitated!the!establishment!of!

a!new!collection!at!SLSA!to!house!material!donated!by!the!nuclear!veteran!community!and!

established!a!relationship!between!SLSA!and!the!Hudson!family!for!future!donation.!

!

!

A.!Avon!Hudson!with!State!Library!of!South!

Australia!and!Nuclear!Futures!staff.!!

B.!Slides!of!Maralinga!from!Avon!Hudson’s!

private!archive.!!

Photos!by!Jessie!Boylan.!

A.+

B.+


! 78!

4.3! Conclusion#

The!Balaklava!Community!Arts!Project!aimed!to!promote!community!participation!and!increased!

opportunities!for!residents!in!the!region!to!engage!with!the!arts.!Across!2014*15!the!project!

delivered!an!arts!residency!program!to!foster!cultural!development!and!collaborative,!community*

driven!art*making.!Local!activities!included!public!events,!school!visits,!skill!development!workshops,!

script!development,!community!archiving,!oral!history!collection!and!the!production!of!original!

creative!works!sharing!community!stories.!Public!presentation!of!new!works!included!two!gallery!

exhibitions:!Field%of%View%and!Portrait%of%a%Whistleblower%and!a!dynamic!multi*arts!showcase!as!part!

of!the!2015!Adelaide!Fringe!Festival.!The!showcase!was!an!assemblage!of!digital!artwork,!photo!and!

video!media,!sound!design,!rare!archival!material,!interwoven!with!lively!panel!and!speaker!sessions.!

Featured!was!the!20*minute!immersive!projection!installation!10%Minutes%to%Midnight.!

The!project!engaged!an!estimated!400!people!in!the!region,!via!workshop!participation!and!event!

attendance.!Media!coverage!extended!the!public!storytelling!platform!for!community!partners!and!

artists,!and!included!feature!articles!on!individual!community!and!team!members,!and!journalism!on!

Australian!atomic!test!history!and!its!contemporary!socio*political!context.!Some!project!outcomes!

have!long!term!implications,!such!as!the!resultant!State!Library!of!South!Australia!collection!on!

South!Australia’s!atomic!test!history.!Many!of!the!story*based!creative!works!generated!from!the!

project!will!also!tour!in!future!showcases!as!part!of!Nuclear!Futures!in!Australia!and!overseas.!

In!order!to!understand!and!appraise!the!activities,!outputs!and!impact!of!the!Balaklava!Community!

Arts!Project,!we!set!out!to!develop!an!integrated!methodology!for!our!evaluation!practice,!to!fit!the!

needs!of!the!project!and!the!Nuclear!Futures!program!more!broadly.!The!READ!model,!integrated!

Reflection,!Evaluation,!Analysis!and!Documentation!(READ)!to!meet!the!stakeholder!requirements!for!

accountability*driven!evaluation!findings!as!well!as!a!means!to!reveal!the!broader!implications!of!the!

creative!practice.!While!the!focus!of!this!research!is!the!Balaklava!application!of!the!READ!concept,!it!

was!envisaged!that!the!model!developed!during!the!DCI!would!be!applicable!for!the!suite!of!inter*

related!CACD!activities!across!Nuclear!Futures.!Different!READ!methodologies!have!been,!and!will!be,!

designed!and!implemented!per!community*based!project,!as!well!as!for!the!international!activities.!!

Next,!in!Chapter!5,!an!in*depth!account!of!the!READ!framework!designed!and!implemented!for!the!

Balaklava!arts!project!is!provided.!In!order!to!be!effective,!the!READ!methodology!needed!to!be!

engaging!and!credible,!dealing!with!the!epistemological!and!technical!challenges!raised!by!the!

complex!multi*arts!practice!and!partnerships,!as!well!as!extend!its!critical!lens!beyond!the!project!

processes!and!outcomes!so!as!to!generate!findings!on!the!impact!of!the!arts*based!activities.!


! 79!

5! The#READ#application,#Balaklava#(2014S15)#

“Feed!on!reality,!so!you!can!take!an!impression!of!it,!!

!so!we!can!stand!back!and!say!‘yeah!that!is!how!it!was’.”!

John!Romeril!(Playwright,!Nuclear!Futures!creative!team!member!for!Balaklava!Community!Arts!Project)!!

Chapter!5!examines!the!stakeholder*oriented!evaluation!practice!undertaken!for!the!Nuclear!

Futures!Partnership!Initiative!(Nuclear!Futures),!an!international!arts!and!cultural!program!

partnering!artists!with!atomic!survivor!communities,!funded!by!the!Australia!Council!for!the!Arts.!

Detailed!in!this!chapter!are!the!collaborative!processes,!activities!and!methods!that!were!developed!

and!implemented!for!the!first!industry!application!of!the!READ!model,!for!which!I!was!the!key!expert!

and!facilitator!in!my!role!as!researcher!for!the!Doctor!of!Creative!Industries!(DCI).!Arising!from!

professional!practice,!the!conceptualised!READ!model!aimed!to!provide!a!wide!summative!lens!for!

evaluating!community!arts,!by!extending!evaluation!practice!across!four!broader!strategies!of!

Reflection,!Evaluation,!Analysis!and!Documentation!(READ).!It!was!envisaged!that!an!integrated!

approach!to!evaluation!would!better!equip!practitioners!and!community!participants!to!reflect!on,!

represent,!and!appraise!their!art*based!activities!and!partnerships,!for!shared!learning!in!context.!

My!role!as!researcher!involved!leading!the!design!and!implementation!of!the!tailored!READ!

approach!for!the!Balaklava!Community!Arts!Project!with!stakeholders!at!the!site!of!practice.!In!this!

role,!I!was!responsible!for!the!brokering!of!collective!input,!consolidating!the!methodology,!

certifying!the!validity!and!robustness!of!the!emergent!methodology,!and!ensuring!ethical!and!

appropriate!conduct.!While!I!refer!to!myself!as!the!READ!facilitator!in!the!research,!it!is!important!to!

state!that!this!was!not!a!neutral!role,!and!was!rather!one!of!active!direction!and!participation,!

involving!significant!intellectual!contribution!from!me!as!the!key!evaluation!expert!and!practitioner*

researcher.!Further,!formulating!the!READ!methodology!based!on!stakeholder!input!required!me!to!

span!the!domains!of!creative!practice,!program!management!and!evaluation!practice.!

Documented!here!are!the!‘what’,!‘why’!and!‘how’!aspects!of!the!developed!READ!model!in!action.!

To!articulate!the!READ!model!as!it!was!enacted!for!the!Balaklava!project,!I!describe!and!systematise!

the!extensive!body!of!work!resulting!from!the!trial!and!explain!the!key!developments!in!the!model!

of!practice!developed,!based!on!the!observation,!stakeholder!consultation,!reflection,!and!analysis!I!

conducted!as!READ!facilitator/!DCI!researcher.!In!consolidating!the!first!READ!methodology!of!its!

kind,!I!was!able!to!establish!an!original!and!substantial!model!of!practice,!and!address!some!of!the!

persistent!issues!for!evaluation!of!community!arts,!as!identified!in!the!Contextual!Scan!(Chapter!2),!

by!exploring!the!integration!of!diversified,!mix!method!approaches!to!identify!and!represent!impact.!


! 80!

A*guide*to*the*READ*application*

Chapter!5!is!divided!into!two!parts.!Part!A!overviews!the!preliminary!activities!from!the!initial!phase!

of!the!READ!application!for!the!Balaklava!Community!Arts!Project,!and!Part!B!covers!the!tailored!

design!and!implementation!of!the!READ!approach!as!a!model,!collaboratively!devised!at!the!site!of!

practice.!Accompanying!the!chapter!is!the!READ%in%Practice!portfolio,!a!compilation!of!original!

documentation!from!the!evaluation!practice.!Combined,!the!chapter!and!READ%in%Practice%portfolio!

provide!insight!into!the!integrated!READ!model’s!scope!and!capability!in!the!community!arts!setting.!!

Part+A+–+Envisaging+the+READ+model+and+preliminary+activities+

The!chapter!begins!with!the!background!story!of!the!READ!concept!prior!to!this!research,!and!

introduces!a!basic!systems!explanation!of!the!READ!model!that!I!developed!in!preparation!for!the!

application!(Section!5.1).!Six!core!elements!of!the!READ!concept!are!identified,!and!these!become!

the!roadmap!for!the!chapter!as!each!is!discussed!in!turn!in!Part!B.!The!six!READ!system!elements!

charted!are:!the!READ!support!structures;!Reflection;!Evaluation;!Analysis;!Documentation;!and!the!

integration,!utilisation!and!dissemination!of!READ!outputs.!The!discussion!includes!preliminary!

assumptions!and!approaches!determined!for!the!Balaklava*based!application,!including!the!

interpretative!stance!and!the!internal!positioning!of!the!evaluation!practice!within!the!arts!program.!!

Part!A!also!positions!the!READ!application!within!the!overall!DCI!research!progression,!to!provide!a!

guiding!narrative!on!the!research!phases!that!underpinned!the!journey!of!my!evaluation!practice!as!

READ!facilitator.!The!section!concludes!with!a!discussion!on!the!findings!arising!from!the!first!phase!

of!the!READ!application,!the!process!of!scoping!with!stakeholders,!that!critically!informed!the!

collective!design!and!delivery!of!READ!in!action.!

Part+B+–+READ+methodology+in+action++

Part!B!covers!the!systems!investigation!of!the!applied!READ!model,!and!provides!rich!descriptions!

and!practical!examples!of!the!emergent!methodology,!methods,!and!deliverables!for!the!READ!

application.!Throughout!the!chapter!I!cross*reference!to!examples!in!the!suite!of!original!materials!

in!the!READ%in%Practice!portfolio!(RIPP),!sign*posted!by!the!icon!to!the!right.!Alternatively,!the!RIPP!

can!be!viewed!separately!as!a!standalone!document!of!industry!examples.!In!studying!the!READ!

model!in!action,!Part!B!opens!with!an!examination!of!the!organisational!scaffolding!necessary!for!

the!model’s!implementation!in!the!Balaklava!context,!which!I!refer!to!as!the!READ!support!

structures.!Next,!each!of!the!four!enquiry!strategies!for!READ!are!described!to!illuminate!how!each!

was!implemented!for!the!project.!Lastly,!the!processes!employed!for!integrating,!utilising!and!

disseminating!the!READ!data!and!outputs!are!canvassed.!Revealed!in!Part!B!are!the!operations,!

functionality!and!outcomes!of!the!READ!model!during!the!Balaklava!experience.!!

!


! 81!

Part#A:#Envisaging#the#READ#model#and#preliminary#activities#

5.1! Background#

5.1.1! The#READ#model##

The!story!of!the!READ!model’s!inception!starts!at!the!site!of!my!professional!practice,!twelve!months!

prior!to!the!commencement!of!the!Balaklava!Community!Arts!Project!(South!Australia).!The!

integrated!READ!concept!arose!from!the!community!arts!practice!of!Alphaville,!a!Sydney*based!arts!

company!specialising!in!community!arts,!cultural!development!and!production,!with!a!focus!on!

social!justice!and!sustainability!issues.!The!READ!concept!was!collaboratively!devised!and!

implemented!by!Alphaville’s!creative!team,!and!in!particular!through!developmental!work!by!Paul!

Brown!as!Creative!Producer!and!myself!as!Program!Manager.!This!took!place!in!2012!while!working!

together!on!the!program!and!partnership!development!for!the!Nuclear!Futures!Partnership!Initiative!

(Nuclear!Futures),!an!international!program!encompassing!a!suite!of!arts!projects.!It!was!during!the!

formative!stages!of!securing!funding!from!the!Australia!Council!for!the!Arts’!Creative!Communities!

Partnership!Initiative!(CCPI),!that!Brown!and!I!commenced!our!early!work!on!the!READ!approach.!

Nuclear!Futures!was!conceived!by!Brown!as!an!extension!of!Alphaville’s!previous!Half%a%Life%project!

which!developed!a!verbatim!theatre!play!with!nuclear!veterans!(See!Brown!&!the!Maralinga!

Research!Group,!2006).!Brown!and!I!had!collaborated!on!Half%a%Life!and!on!several!environmental!

education!initiatives!at!the!University!of!New!South!Wales!(UNSW).!Nuclear!Futures!built!on!

Alphaville’s!existing!networks!and!was!inspired!by!the!urgency!to!continue!work!with!the!last!

surviving!eye!witnesses!of!the!British!led!atomic!tests!in!Australia,!and!by!renewed!efforts!of!nuclear!

affected!communities!seeking!recognition.!It!developed!as!a!series!of!interrelated!arts!and!cultural!

activities!across!three!years,!partnering!international!atomic!survivor!communities!with!professional!

artists!to!create!an!original!showcase!of!story*based!artworks,!with!the!support!and!engagement!of!

a!broad!partnership!collective!involving!both!arts!and!non!arts!organisations!(see!Chapter!4).!!

In!the!planning!and!development!phase!of!Nuclear!Futures,!Brown!and!I!worked!with!program!

partners!to!design!an!international!program!of!creative!arts,!as!well!as!the!necessary!administration!

and!management!structures!to!support!the!activities!and!partnerships.!Included!in!this!

developmental!phase!was!the!identified!need!for!evaluation.!The!early!premise!was!that!program!

evaluation!of!Nuclear!Futures!would!need!to!service!the!stipulations!of!the!primary!government!

funding!agency,!the!Australia!Council!for!the!Arts,!as!well!as!embrace!the!varied!interests!and!values!

of!community!partners,!so!as!to!be!representative!and!meaningful!for!the!mix!of!culturally!diverse!


! 82!

participants!and!organisations.!The!evaluation!approach!also!needed!to!be!reflexive!and!

participatory!to!reflect!the!program’s!creative!community!partnerships!and!community!arts!setting.!!

In!the!development!of!the!READ!approach,!I!reviewed!many!inspiring!evaluation!methodologies!

including!the!rigorous!academic!and!analytical!approach!of!Somebody’s!Daughter!Theatre!

evaluations!conducted!by!Jan!Osmotherly,!the!practice*led!examples!from!Queensland!University!of!

Technology,!and!the!creative!documentation!style!of!Big!hART!for!Ngapartji%Ngapartji!(complete!with!

a!treasure!trove!box!of!stories,!symbolic!objects!and!novelties)35.!Participatory!oriented!and!arts*

relevant!frameworks!such!as!Arts!Victoria’s!Evaluation!Guide,!the!Most!Significant!Change!technique,!

and!the!BetterEvaluation!Rainbow!framework!were!useful!for!providing!methodological!guides,!some!

of!which!had!adopted!a!design*your*own!toolbox!approach.36!While!such!exemplars!were!insightful,!

my!realisation!culminating!from!the!contextual!scan!was!that!there!was!no!one!approach!readily!

applicable!to!suit!the!needs!and!scope!of!the!site!of!practice,!and!the!multi*platform!evaluative!

outputs!Alphaville!envisaged.!Further,!some!of!the!practitioner!guides!consulted!tended!to!demand!

over*simplification!of!evaluation!or!were!underpinned!by!empirical!methodologies!borrowed!from!

the!social!sciences!more!so!than!the!humanities!and!creative!arts!(Belfiore!&!Bennett,!2010).!

Consistently,!the!research!demonstrated!that!the!task!of!designing!effective!program!evaluation!for!

community!arts!is!best!done!using!a!tailored!approach,!on!a!project*by*project!basis!(Keating,!2002).!

In!conjunction!with!the!contextual!scan!to!identify!best!practice,!Brown!and!I!ruminated!on!our!own!

professional!experiences,!on!the!failures,!successes!and!lessons!from!past!evaluations,!and!reflected!

on!the!types!of!evaluative!enquiry!that!may!be!relevant!for!understanding!and!advancing!the!

creative!arts!practice!central!to!Nuclear!Futures.!We!also!considered!the!expertise,!arts!orientation!

and!strengths!of!the!Nuclear!Futures!creative!team!and!prospective!partners.!While!the!creative!

team!operated!across!a!range!of!art!forms!and!practices,!collectively!there!was!extensive!experience!

in!documentary!art*making.!Such!considerations!led!Brown!and!I!to!adopt!a!wide!frame!of!reference!

for!defining!what!might!constitute!Alphaville’s!evaluation!opportunity,!and!we!explored!a!number!of!

complementary!learning,!appraisal!and!exchange!methods!that!would!potentially!support!and!enrich!

evaluation,!program!management!and!creative!practice.!In!creating!an!original!concept!model!for!the!

Nuclear!Futures!program,!there!was!also!an!opportunity!to!consider!some!of!the!broader!issues!and!

problems!of!community!arts!evaluation!identified!within!the!field.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

35!In!respective!order,!examples!include:!Osmotherly!(2002),!Osfield!Consultants!(2007)!and!general!information!at!

http://www.somebodysdaughtertheatre.com;!Klaebe!et!al.!(2015)!and!Gattenhof!&!Seffrin!(2014);!and!Big!hART!(2010),!

Palmer!(2010)!and!general!information!at!http://www.ngapartji.org.!36!In!respective!order,!references!for!the!listed!examples!include:!Keating!(2002);!Dart!&!Davies!(2003);!and!

BetterEvaluation!(2013).!


! 83!

Collectively!Brown!and!I!had!decades!of!experience!in!arts!management!and!evaluation!to!draw!upon!

during!these!formative!conversations.!Over!three!decades,!Brown!has!amassed!a!wealth!of!

experience!in!evaluation!in!institutional!and!community!contexts,!from!the!perspectives!of!manager,!

producer,!and!creative!practitioner.!Brown’s!evaluation!experience!includes:!

•! Program!evaluation!and!staff!reviews!whilst!Head!of!the!School!of!Humanities!at!UNSW.!

•! Evaluation!of!the!impact!of!the!Arts!in!sectors!such!as!Health,!as!co*author!Art%and%Wellbeing!(Mills!&!Brown,!2004).!!!

•! Conducting!a!major!evaluation!of!Australia’s!hazardous!waste!management,!using!survey!and!

key!informant!interview!techniques.!

•! Assessment!and!peer!review!of!project!funding!applications!for!the!Australia!Council!for!the!

Arts.!

•! Reflective!practice/!evaluation!as!an!established!arts!practitioner!(playwright,!producer,!

filmmaker).!

In!complement!to!Brown’s!expertise!is!my!own!professional!experience!in!program!management!and!

evaluation!over!fifteen!years!in!community!development,!creative!arts,!education!and!sustainability.!

My!evaluation!experience!includes:!

•! Research!and!evaluation!of!a!sustainability!education!and!engagement!program!for!1,100!

staff,!in!a!NSW!local!government.!

•! Corporate!and!project!evaluations!as!Manager!for!the!Comboyne!Community!Technology!

Centre!(rural!NSW).!!

•! Program!evaluation!of!a!national!design!and!innovation!program!at!UNSW!involving!900!high!

schools,!and!staff!performance!appraisals!as!the!Program!Manager.!

•! Providing!strategic!evaluation!advice!and!planning!with!businesses!and!organisations,!as!a!

project!consultant!through!my!sole!trader!business!High%Ideals.!

Taking!inspiration!from!Alphaville’s!creative!arts!practice!we!explored!an!integrated!approach,!where!

a!series!of!complementary!review!strategies!relevant!to!community!arts!and!cultural!development!

(CACD),!might!work!together!to!extend!the!relevance!and!benefits!generated!during!program!

development!and!appraisal.!In!the!division!of!labour!for!Nuclear!Futures,!it!became!my!role!to!lead!

the!evaluation!elements!of!the!program,!and!one!of!my!early!tasks!was!to!enunciate!the!scope!and!

intent!of!READ.!Consultation!was!undertaken!with!early!partners!and!stakeholders!for!feedback,!

which!verified!interest!in!an!evaluative!approach!that!could!reach!beyond!an!evaluation!and!acquittal!

reporting!format.!We!also!undertook!negotiation!and!confirmation!with!funding!agency!partners.!!

Across!many!conversations,!the!collective!input!and!visioning!for!creating!a!fresh!approach!to!

evaluation!at!the!site!of!practice!contributed!significantly!to!the!emergent!READ!concept.!!

! !


! 84!

The!early!READ!concept,!as!included!in!the!Australia!Council!for!the!Arts!funding!application,!was:!

We!propose!to!develop!a+new+participatory+model+that+integrates+Reflection,+Evaluation,+

Analysis+and+Documentation!(the!‘READ’!model)!to!evaluate!the!program’s!processes,!

partnerships!and!creative!outputs.!!

•! We!will!use!a!partnership!model!based!on!participatory!evaluation!methodology!and!

reflective!analysis,!with!evaluative!goals!set!by!partners!to!inform!academic*supported!

framework!design!and!implementation.!

•! The!approach!blends!traditional!data!collection!and!analysis!(qualitative!and!quantitative)!

with!emergent!techniques!such!as!‘Most!Significant!Change’!process!(story*based!

evaluation).!It!involves!all!partners!in!asking!the!questions,!data!collection!and!analysis.!(It!

is!socio*dynamic.)!

•! READ!includes!formative!evaluation!and!analysis!for!ongoing!appraisal!and!summative!

evaluation!at!completion,!with!focus!on!CACD!processes,!the!creative!works!and!the!

quality!of!the!partnerships.!

•! Reflective!analysis!originates!in!individual!and!group!practice!within!individual!projects,!

and!proceeds!via!conferences,!seminars!and!publications!that!link!creative!artists,!

academics!and!communities.!!

Strengths:!

•! The!overall!approach!is!documentation*oriented!(written,!visual,!audio,!ephemeral/!rich!

media,!as!well!as!archival),!with!commitment!to!disseminate!READ!findings!to!

communities,!partners!and!industry!to!make!lessons!explicit!&!accessible!to!wider!field.!

Our!on*line!community!is!a!key!delivery!mechanism.!

•! READ!is!evidence*based,!continuous!over!multiple!years,!and!guided!by!socially!inclusive!

principles.!It!is!transferable!and!relevant!for!other!Community*based!Arts!and!Creative!

Industries!practitioners.!

•! READ!develops!a!culture!of!learning!and!reflection!that!will!support!professional!

development!and!is!a!key!strategy!for!sustaining!long*term!collaborative!partnerships!and!

capacity!of!communities!and!practitioners.!

•! Evaluation!is!supported!by!an!extensive!‘community!of!researchers’!committed!to!action!

research,!and!therefore!is!likely!to!generate!robust!evaluation!findings.!The!lead!institution!

is!QUT’s!Creative!Industries!Faculty,!which!will!host!two!doctoral!level!research!projects!

dedicated!to!Nuclear!Futures.!

(Extract!from!Alphaville!fundraising!documentation,!2012!describing!the!READ!model)!

As!outlined!in!Section!1.5,!the!summary!above!demonstrates!the!extent!to!which!the!READ!approach!

had!been!articulated!by!Alphaville!prior!to!this!research.!Brown!and!I!had!surmised!the!basic!frame!

and!overarching!strategies,!however!had!not!established!the!methodology!or!proof!of!concept.!In!

order!to!develop!the!early!READ!concept!into!a!model!of!practice!(as!an!applied/!working!system)!I!

turned!to!the!DCI!as!the!means!for!completing!this!work!as!academic!research!(distinct!from!Nuclear!

Futures!and!CCPI!funding).!The!DCI!professional!doctorate!offered!a!unique!opportunity!for!me!as!an!

industry!practitioner!to!develop!a!READ!product!and!to!substantiate!its!potential!value!using!

advanced!critical!reflection!of!practice!and!rigorous!scholarship.!The!following!sections!document!my!

process!of!leading!the!READ!model!application,!a!collective!design!and!implementation!endeavour!at!

the!site!of!practice,!to!advance!the!epistemology!and!methodology!of!the!READ!model.!


! 85!

5.1.2! A#systems#approach#

To!visualise!and!examine!the!conceptualised!READ!model,!at!the!commencement!of!the!research,!I!

used!a!systems!approach!to!identify!and!isolate!its!core!elements!and!boundary,!and!to!map!the!

anticipated!inter*relationships!of!the!key!elements!(Darabi,!2002).37!A!system!is!defined!by!Raynal!

and!Rieunier!as!‘a!set!of!elements!related!both!to!each!other!and!to!their!environment!and!organised!

in!accordance!with!a!goal’!(UNESCO!&!ICCROM,!2015).38!Exploring!the!model!as!a!system!was!the!

first!step!I!took!in!the!process!of!leading!the!development!of!a!READ!prototype!and!methodological!

structure!for!the!READ!Balaklava!application.!

The!six!core!system!elements!of!the!proposed!READ!model!I!identified!were:!

1.! Support!structures!–!administrative!and!operational!structures!developed!to!link!READ!

implementation!with!program!development!and!management,!and!guide!partnership!

brokerage!and!stakeholder!engagement/!participation.!

2.! Reflection!–!community!members!and!artists!reflect!on!projects!and!practice!through!

journaling,!interviews!and!other!personal!records!(individual/collective!reflective!practice).!

3.! Evaluation!–!a!range!of!formal!instruments!are!devised!and!implemented!in!

consultation/collaboration!with!program!partners!in!response!to!key!evaluation!questions.!

4.! Analysis!–!a!cohort!of!academics!work!with!artists!and!community!members!to!develop!

scholarly!publications,!reports!and!presentations,!for!broader!learning!and!dissemination.!

5.! Documentation!–!filmmakers,!multi*media!artists!and!participants!produce!multi*platform!

documentation!and!archives,!and!extracts!from!project!documentation!are!collated.!

6.! Integration,!utilisation!&!dissemination!–!!Data!from!each!READ!element!is!synthesised!to!

produce!a!portfolio!of!evaluative!and!non*evaluative!outputs!to!meet!end*needs!and!

preferred!formats,!in!consultation!with!partners,!artists!and!communities.!

The!above!six!elements!work!together!within!the!READ!system.!Figure:!5.1!Illustrates!how!I!visualised!

the!model!as!a!system!and!its!core!elements.!During!the!chapter,!each!key!element!will!be!discussed.!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

37!Systems!theory!can!be!defined!an!approach!to!‘describe!and!interpret!complex!interactive!phenomena…!emphasizing!

the!properties!of!the!relationships!between!the!phenomena!to!be!as!significant!as!the!properties!of!the!phenomena!

themselves’!(Davidson!and!Rowe,!2009,!p.562).!While!elaboration!on!systems!theory!and!system!approach!to!program!

evaluation!(Darabi,!2002)!is!beyond!the!scope!of!the!research,!techniques!from!the!literature!were!influential!for!how!I!

approached!the!READ!model!to!theorise!and!develop!it.!!38!The!original!source!for!the!quote!is!given!as:!Françoise!Raynal!and!Alain!Rieunier,!Définir%des%objectifs%pédagogiques,!

1987,!IPNETP*Les!Nouvelles!Éditions!Africaines,!Abidjan.!


! 86!

Feeding!into!the!system!are!inputs!and!information!arising!from!the!context,!partnerships!and!art*

making!at!the!site!of!creative!practice.!The!process!of!scoping!and!planning!defines!the!support!

structures!and!strategic!plans!necessary!for!delivering!READ,!including!handling!of!ethical!conduct,!

cultural!competency,!partnership!brokerage,!engagement,!administration,!content!management!and!

so!on.!!The!four!central!READ!strategies!(Reflection,!Evaluation,!Analysis!and!Documentation)!are!

illustrated!as!independent!yet!interrelated!modules,!and!running!across!all!is!an!exchange!of!

methods!and!data/information.!The!READ!strategies!generate!artefacts,!and!once!integrated!to!suit!

the!desired!partner!identified!end*uses,!a!portfolio!of!raw!and!processed!content!results!that!can!be!

used!for!evaluation,!management!and!creative!development/!arts!practice.!Interacting!with!the!

READ!system!are!companion!systems!that!drive!the!community!arts!practice!and!management,!

shown!on!the!left!of!the!diagram!where!the!spheres!overlap.!In!concurrent!operation,!the!creative!

practice,!management!and!READ!spheres!support!and!influence!each!other,!where!a!broader!

exchange!of!information!and!impact!occurs!across!the!site!of!practice.!See!Figure!5.1!below.!

!

+

!

Figure!5.1:!The!READ!model!as!system,!in!relationship!to!community!arts!practice!and!management!

!


! 87!

5.1.3! Research#phases##

In!guiding!the!READ!application!as!a!systems!approach,!a!number!of!phased!activities!were!

conducted.!The!phases,!undertaken!as!evaluation!practice,!fit!within!the!broader!progression!of!the!

DCI!research!(see!Chapter!3!for!full!DCI!methodology).!As!previously!stated,!the!READ!application!

constitutes!four!of!the!central!research!phases!for!the!DCI.!The!READ*related!research!phases!were:!!

1.! Scoping!with!stakeholders!

2.! Planning!and!design!of!the!Balaklava!READ!methodology!

3.! Implementation!(data!collection!and!analysis)!

4.! Production!and!dissemination!of!READ!outputs.!

Figure!5.2!is!an!elaboration!of!Figure!1.2!included!in!the!Introduction!(Chapter!1).!Figure!5.2!provides!

a!brief!explanation!of!each!of!the!READ*related!research!phases!stated!above.!!It!is!important!to!

note!that!while!represented!as!a!linear!sequence,!the!four!research!phases!summarised!in!Figure!5.2!

have!not!conformed!to!a!chronological!or!straightforward!sequence!as!the!diagram!suggests,!and!

were!iterative!processes!delivered!for!both!formative!and!summative!cycles.!!Across!the!READ*

related!research!phases!I!adopted!a!reflexive!approach!that!allowed!me!to!respond!to!emergent!

project!needs,!seize!unforeseen!opportunities!and/or!address!gaps!in!the!evaluation!research.!!

!

! !


! 88!

+

!

!

! !

1. Identify and consult with stakeholders to determine values, needs & opportunities. Get to know the context and partners. What are the strengths of the art practice and partnerships? What is the aim of READ for the project? Identify cultural and ethical considerations. What are the inhibitors and enablers for using READ?

2. Analyse what is unearthed during scoping. Set purpose and sub-questions within READ strategies that balance partner needs. Select methods. Establish indicators where necessary. Decide who facilitates and participates. Work out the when, how and the what. Define the audience and ideal outputs. Develop plans for action, including how to fund and resource activities. Identify what support structures best link READ with arts practice and management.

3. Undertake data collection, analysis and integration for formative, summative or ongoing use. Get permissions. Keep reflexive and opportunistic and refine READ plans as needed. Organise and manage data collected. Support implementation with training and resources. Reduce and check data. Create a flow of information between READ, arts practice and management. Analyse content to establish findings, lessons and conclusions, and process data if needed. Keep lines of feedback and communication with partners open to plan the outputs.

!

4. Integrate the emergent data as needed to create an array of end-products from READ (raw and processed content and outputs) that meet partner needs. What formats and mediums will deliver best results for the type of information/material requested? Disseminate READ outputs to partners and stakeholders. Make use of outputs for continuous improvement, wider learning or legacy outcomes.

READ application

Figure!5.2!:!The!research!phases!for!the!DCI!

!

(Note:!The!four!research!phases!related!to!the!evaluation!practice!for!the!READ!application!are!white)!

!


! 89!

5.2! READ#approach#and#assumptions#

Qualified!in!this!section!are!the!contextual!influences!and!application!assumptions!determined!for!

the!tailored!Balaklava!READ!framework,!such!as!the!internal!positioning!of!the!evaluation!practice!

(via!READ)!and!the!interpretive!stance!adopted!to!reflect!the!needs!and!ethos!of!the!site!of!practice!

(see!Chapter!4!for!full!details).!!

5.2.1! Context#and#parameters##

For!the!Balaklava!READ!application!there!are!three!notable!research!parameters!to!acknowledge.!

First,!while!I!refer!to!the!READ!model!design!and!implementation!process!as!experimental,!involving!

the!trial!and!error!of!methods!to!refine!the!working!model!(underpinned!by!forensic!reflective!

practice),!the!application!does!not!constitute!a!research!experiment!in!the!traditional!scientific!sense!

as!I!(as!researcher!and!practitioner)!did!not!control!the!variables!or!design!the!application!for!that!

purpose.!!

Second,!the!contextual!setting!and!the!Nuclear!Futures!project!selected!have!significantly!impacted!

on!how!the!conceived!READ!model!has!been!interpreted!and!developed.!Boud!and!Walker!(1998,!

p.169)!argue!that!context!is!‘the!single!most!important!influence!on!learning!and!reflection’,!and!this!

is!true!for!documented!community!arts!evaluation!design!(Fitzpatrick!et!al.,!2012;!Keating,!2002).!!

Given!the!importance!of!context!for!the!application!outcomes!presented!in!this!chapter,!it!is!worth!

reiterating!the!characteristic!features!of!the!Balaklava!setting!(refer!to!Chapter!4!for!further!

contextual!background).!Namely,!the!featured!project!was!a!creative!community!partnership!

initiative!positioned!within!a!community!arts!setting,!which!encompassed!broad!creative!and!

collaborative!practices!involving!multiple!partnerships!with!artists!and!communities!(see!Chapter!2).!

The!project*specific!and!geographical!context!for!the!application!was!the!small!regional!South!

Australian!community!of!Balaklava,!on!the!Wakefield!Plains!north!of!Adelaide.!Involved!were!mixed!

constituencies!of!targeted!participants!for!the!multi*arts!objectives,!involving!artists,!local!young!

people,!the!general!public!and!a!network!of!families!linked!with!Australia’s!atomic!test!history.!!

The!third!parameter!to!acknowledge!is!the!influence!of!the!scope!and!emphasis!of!the!READ!

application!for!the!Balaklava!project.!Namely,!the!application!presented!in!this!chapter!focused!on!

the!investigation!of!the!processes,!impacts!and!outcomes!of!the!community!arts!activities!for!

participants,!partners,!program!and!communities.!Figure!5.3!from!An%Evaluation%Guide%for%

Community%Arts!Practitioners,!produced!by!Arts!Victoria%(Keating,!2002),!provides!a!useful!matrix!for!

illustrating!the!areas!of!project!interest!that!came!under!review!of!the!READ!model!development.!


! 90!

!

Figure!5.3:!Key!elements!of!Community!Arts!project!to!be!reviewed!!

(Source:!An%Evaluation%Guide%for%Community%Arts!Practitioners!!produced!by!Arts!Victoria%(Keating,!2002,!p.vii))!

!

5.2.2! Internal#position#for#READ#facilitation#

For!the!READ!application,!my!role!as!READ!facilitator!was!internally!positioned!within!the!Nuclear!

Futures!program,!distinct!yet!interrelated!to!my!professional!position!as!program!manager.!This!

meant!that!the!evaluation!practice!for!the!READ!application!was!internally!positioned!within!the!

community!arts!program.!This!insider!positioning!was!a!strategic!decision!considered!as!favourable!

for!facilitating!the!potential!transformative!benefits!of!the!READ!processes.!Internal!design!and!

delivery!meant!I!was!able!to!harness!my!existing!relationships!within!the!Nuclear!Futures!network!to!

support!participation!and!partnership!brokerage,!and!take!advantage!of!my!detailed!knowledge!of!

the!arts!program.!Further,!I!envisaged!that!by!acting!in!the!roles!of!both!program!manager!and!READ!

facilitator!the!efficacy!of!implementation!and!information!sharing!between!READ,!the!creative!

practice!and!program!management!would!be!enhanced.!

5.2.3! Interpretative#stance#

To!compliment!the!internal!positioning!and!the!pluralism!underpinning!the!community!partnerships,!

a!social!constructivist!stance!was!adopted!for!the!Balaklava!application!of!the!READ!approach.!Unlike!

the!positivist!or!objectivist!theoretical!traditions,!an!interpretative!constructivist!stance!recognises!

that!individuals!and!groups!construct!their!perceived!reality!(Glaser!&!Strauss,!2009;!Charmaz,!2000;!

Cresswell,!1994).!Schwandt!(2000,!p.197)!states!‘constructivism!means!that!human!beings!do!not!

An Evaluation Guide for Community Arts PractitionersPrepared by Effective Change for Arts Victoria; Darebin City Council; City of Whittlesea and VicHealth vii

W H O I S T H E G U I D E F O R ?

The guide is for community arts practitioners, rather than specific organisations, as community artsprojects can be located across a variety of organisations.

H O W S H O U L D I T B E U S E D ?

The guide essentially ‘talks’ to a generic community arts project worker and assumes that thisperson will drive the evaluation. However, it can be used in a range of ways. For example, allparticipants in a community arts project could share roles in undertaking an evaluation. Or itcould be used an external student or consultant to undertake an independent evaluation.

C O M M U N I T Y A R T S – N O T J U S T C O M M U N I T Y , N O T J U S T A R T S

This guide recognises the unique place of community arts within community and cultural activity.Case stories abound of community arts projects producing stunning and lasting creativeachievements. There are also numerous examples where extraordinary life changes occurred forparticipants while the creative outcomes were less memorable. Other times, community artsprojects have contributed enormously to their local community, through a range of strategies,such as engaging disenfranchised members of the community, or producing valuable resourcesfor a particular community. Sometimes exceptional community arts projects produceextraordinary results across all areas. Other times, people just feel good by being involved andtaking risks. This guide has been developed to work across all areas – from the creative output,to the creative journey, to the individuals and communities involved. This model of evaluationwill allow you to identify the areas of achievement and success. It works even when, inconventional terms, the project may not appear to be that successful. It will also allow you torecognise extraordinary achievements across this range of factors when they do occur.

The diagram below shows the key elements to be evaluated, using this model of evaluation.

Diagram 1: Key Elements of Community Arts Projects to be evaluated

participantsproject/

organisationcommunity

processhow participants are

involved in theproject

how the project ismanaged

how the community isinvolved in the

project

process

impactwhat happens to, or

for, participantsthrough the project

what happens asresult of the project

what happens to orfor the communitythrough the project

impact

outcomewhat happens to, or

for, participants in thelong term as a result

of the project

what happens in thelong term, as a result

of the project

what happens to, orfor, the community in

the long term as aresult of the project

outcome

participantsproject/

organisationcommunity


! 91!

find!or!discover!knowledge!so!much!as!we!construct!or!make!it’.

In!terms!of!its!relevance!for!evaluation!practice,!the!selected!interpretative!stance!aligns!with!Guba!

and!Lincoln’s!position!that!evaluation!needs!to!extend!beyond!measurement,!description!and!

judgment*!to!develop!methodologies!that!cater!for!the!multiple!realities!and!stakeholder!views!

necessary!to!produce!authentic!and!representative!findings!(Guijt!et!al.,%2011).!The!stance!meant!

that!emphasis!and!power!was!withdrawn!from!the!notion!that!evaluation!is!a!mechanism!for!

producing!objective!summative!judgments,!widening!READ’s!terms!of!reference!to!accommodate!

multiple!knowledge!systems!and!value!paradigms!of!partners,!and!the!pursuit!of!both!evaluative!and!

non*evaluative!outputs/outcomes.!!

It!is!worth!noting!that!for!future!applications,!it!is!envisaged!that!the!READ!model!has!the!flexibility!

to!be!delivered!from!a!range!of!different!stances,!across!positivist,!realist!or!constructivist!

philosophies.!

5.2.4! ParticipatorySoriented#approach#

To!best!align!with!the!community!arts!and!cultural!development!(CACD)!principles!governing!the!site!

of!practice!(which!include!participation,!representation,!democracy,!and!inclusion)!a!participatory!

approach!was!adopted!for!the!READ!application.!As!defined!in!Chapter!2!participatory!evaluation!is!

defined!by!the!level!of!engagement!and!contribution!by!participants!and!partners!in!the!process.!For!

Balaklava,!a!stakeholder*oriented!participatory!approach!was!devised!where!as!facilitator!I!

consulted!and!collaborated!with!stakeholders!and!community!representatives!who!had!a!stake!in!

the!project.!Partners!were!invited!to!be!involved!and/or!contribute!in!the!formulation!of!the!

evaluation!questions,!goals/measures,!methods,!as!well!as!with!the!delivery!and!implementation.!

While!partners!in!the!Balaklava!project!had!varying!capacities!and!interest!to!be!involved,!the!READ!

design!and!implementation!did!draw!on!the!collective!knowledge,!skills,!perspectives,!and!where!

possible,!active!participation!of!partners!during!the!process!(see!Section!5.3!and!5.4.1).!The!methods!

and!results!of!the!invitation!to!participation!in!the!READ!application!are!included!within!the!

documentation!of!the!READ!application!in!this!chapter.!

! !


! 92!

!

5.3! Scoping#

The!scoping!and!preparation!phase!for!the!READ!application!(summarised!in!Figure!5.2)!involved!

preliminary!research,!consultation!and!brokerage!with!stakeholders!and!partners!to!establish!the!

information!and!documentation!needs,!guiding!questions,!evaluation!goals!and!criteria,!and!

opportunities!for!participation!and!learning!exchange.39!Below!are!are!some!of!the!findings!

generated!from!the!scoping!process!I!facilitated!for!the!Balaklava!Community!Arts!Project.!!

(RIPP:!Section!A,!p.6!has!some!examples!demonstrating!aspects!of!the!scoping!research.)!

5.3.1! Stakeholders#identification#and#consultation#

The!first!step!of!the!scoping!process!was!to!identify!relevant!stakeholders!and!groups!for!the!project!

and!for!the!READ!application.!The!stakeholders!for!the!Balaklava!project!I!identified!were:!!

Host!organisation!and!personnel:!Alphaville!and!Nuclear!Futures!staff!and!creative!team.!

Delivery!partners:!Youth!Advisory!Committee!(Wakefield!Regional!Council,!South!Australia),!

freelance!artists/practitioners.!

Community!partners:!Australian!Nuclear!Veterans!Association!(and!Nuclear!Veteran!

communities!and!families),!British!Nuclear!Test!Veterans!Association!(BNTVA).!

Funding!partners:!Australia!Council!for!the!Arts,!Arts!South!Australia.!!

Participants:!community!members!and!stakeholders!participating!in!community!arts!processes!

and!projects;!Balaklava!High!School,!Horizon!Christian!College.!

Audiences!(public):!members!of!the!public!or!communities!of!interest!who!view,!interact!and!

experience!the!creative!works,!showcases,!promotion!and!affiliated!research.!

In*kind!partners/!institutions:!Wakefield!Regional!Council,!State!Library!of!South!Australia,!In!

Place!of!War,!Manchester!University,!Atomic!Photographers!Guild,!University!of!Adelaide.!

Academic!partners:!Queensland!University!of!Technology,!University!of!Western!Sydney,!

University!of!New!South!Wales.!

Venue!partners:!Balaklava!Courthouse!Gallery,!Balaklava!Town!Hall.!

Regional!organisations!and!representatives!of!relevance!(community,!arts!and!media):!

Balaklava!Community!Arts,!local!progress!associations,!youth!and!health!organisations,!

community!leaders,!Carclew!Arts,!local!artists.!

From!the!list!above,!I!determined!those!with!primary!interests!in!the!project!and!those!affected!

directly!by!the!proposed!READ!processes!and!dissemination!to!guide!the!extent!of!consultation!

required.!I!surmised!that!in!particular,!the!READ!activities!and!outputs!would!principally!represent!

the!interests!of!Alphaville/Nuclear!Futures!and!the!program’s!community,!delivery!and!funding!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

39!As!a!directional!guide,!to!keep!the!READ!application!grounded!within!the!DCI!research!progression,!I!have!included!the!

graphic!from!Figure!5.2!at!the!beginning!of!the!three!major!sections!to!signpost!the!related!research!phase.!!

!

!


! 93!

partners.!Beyond!this,!READ’s!formative!and!summative!outputs/!artefacts!were!also!identified!to!be!

of!potential!relevance!to!special!interest!and!niche!audiences,!such!as!CACD!practitioners!and!

evaluators.!It!was!representatives!from!these!primary!stakeholder!groups!that!became!the!key!

informants!for!the!scoping!and!consultation!research.!

The!proposed!READ!approach!was!discussed!with!partners!and!selected!stakeholders,!as!part!of!an!

intensive!period!of!research!and!consultation!at!the!commencement!of!the!project.!For!this!process!I!

collected!information!from,!and!about,!partners!and!stakeholders!using!interviews,!communications,!

workshops,!organisational!documentation,!and!meeting!notes.!During!the!needs!assessment!I!

gathered!input!on!partner!areas!of!interest,!evaluation!and!documentation!needs!(short!and!long!

term)!and!preferred!outcomes!to!establish!the!types!of!questions!the!evaluation!study!would!

address!for!the!focus!of!the!evaluation.!Ahead!of,!and!during!sessions,!communication!tools!were!

used!to!inform!stakeholders!about!the!project!vision!and!other!partners!as!part!of!this!process.!

(RIPP:!Article!8,!p.16!is!an!example!of!a!summary!document!used!for!this!purpose.)!!

5.3.2! Methods#and#analysis###

Semi*structured!and!unstructured!interviews!were!conducted!with!individuals!and!groups!as!part!of!

the!scoping!phase.!The!preliminary!interviews!unearthed!industry!and!stakeholder!interpretations!of!

evaluation!in!the!community!arts!context,!and!specifically!the!evaluation!and!broader!needs!of!

stakeholders!for!the!Nuclear!Futures!READ!application.!In!many!cases,!as!a!prelude!for!the!interview,!

participants!were!asked!to!complete!a!short!profile!questionnaire!(via!email!or!phone)!to!generate!

preliminary!qualitative!and!quantitative!information!about!them!or!their!organisation!which!

informed!the!question!design!of!the!interview.!I!interviewed!thirteen!representatives!from:!

•! Government!Arts!and!funding!bodies!(state!and!federal)!!

•! Peak!arts!organisations!and!cultural!institutions!

•! Community!Arts!practitioners/!creative!artists/!arts!workers/!administrators!

•! Community!partners!and!community!arts!participants!

•! Evaluator!practitioners!

•! Industry!leaders/!experts/!specialists!in!research!area!

•! Academic!researchers.!

!


! 94!

Two!principal!schedules!of!probe!questions!were!designed!to!suit!the!industry!representatives!and!

Nuclear!Futures!partners.!The!methods!for!sampling!and!question!design!used!for!the!interviews!

is!included!in!the!READ%in%Practice!portfolio!(RIPP:!Article!2,!p.8).40!!

A!number!of!meetings!with!partners!were!conducted!between!December!2013!–!April!2014!(with!

discussion!questions!reflecting!the!themes!from!the!interview!schedule)!and!one!workshop!with!ten!

representatives!from!the!creative!team!and!institutional!partner!organisations!held!in!February!2014!

was!undertaken.!Such!sessions!allowed!for!READ!goals!to!be!collaboratively!defined,!core!values!to!

be!shared!and!articulated,!and!brainstorming!for!READ!related!strategies!that!would!maximise!the!

links!to!the!arts!practice!and!program!management.!The!collected!information!was!integrated!into!

both!READ!and!program!management!planning.!(RIPP:!Article!4,!p.11!is!an!example!of!READ!and!

program!management!plans!being!integrated.)!

I!undertook!data!analysis!to!identify!emergent!themes!and!categorised!the!list!of!evaluation!needs,!

examining!the!commonalities!and/or!contrast!of!content.!Predetermined!and!open!categories!

(thematic!coding)!were!used!for!processing!the!large!volume!of!data!generated!via!the!scoping!and!

design!phases!of!the!READ!application,!predetermined!categories!were!used!for!the!stakeholder!

needs!analysis!and!for!developing!an!overarching!evaluation!plan!(with!categories!such!as!evaluation!

questions,!stakeholder!end*uses,!scheduling!needs,!preferred!format),!which!later!informed!the!

READ!theory!development.!The!data!collected!in!the!questionnaire,!interviews,!group!meetings!and!

observation!was!organised!and!synthesised!with!information!sourced!from!secondary!sources!such!

as!program!and!organisational!documentation!and!informal!communications.!

In!order!to!gain!insight!into!the!different!constructs!of!evaluation!operating!in!the!READ!application,!

and!consequently!in!the!collective!process!of!refining!the!READ!methodology,!a!process!of!

comparative!analysis!and!deconstruction!was!employed!via!a!two*tiered!process.!The!first!stage!

involved!examining!the!information!manifest!in!the!responses!to!identify!evaluation!priorities!and!

principles!held!by!the!researcher!and!stakeholders.!The!second!phase!examined!any!evident!

underlying!epistemologies!and!ideologies!(the!latent!content!inherent!in!stakeholder!articulations),!

as!related!evaluation!trends/approaches!identified!in!the!contextual!scan!that!would!be!relevant!the!

theory!development!of!a!more!widely,!transferrable!model.!The!deconstruction!technique!allowed!

responses!to!be!broken!down!at!the!site!of!practice,!and!in!some!cases!revealed!subtle!underlying!

discourses!providing!insight!into!stakeholder!evaluation!agendas!and!priorities!more!broadly.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

40!Data!collected!in!the!interviews!spanned!both!the!DCI!and!READ!methodologies.!I!have!included!information!about!the!

interviews!in!both!dissertation!sections.!See!also!Section!3.3.1!in!Chapter!3.!

!

!


! 95!

5.3.3! Results#for#the#scoping#phase#

During!the!stakeholder!interviews!I!asked!participants!about!their!definition!of!evaluation!and!its!

purpose/role!for!community!arts.!Definitions!predominantly!described!a!formal!assessment!process!

of!assigning!merit!or!judgment.!For!example,!one!stakeholder!described!evaluation!as!‘a!discrete!

highly!fenced!approach!to!examining!the!successes,!failures,!impacts,!benefits,!etc.!of!a!particular!

project’.!Overall,!the!articulations!of!evaluation!presented!by!the!participants!during!the!scoping!

interviews!represented!diverse!interpretations!and!definitions!of!evaluation.!As!part!of!the!data!

analysis,!I!deconstructed!the!articulations!to!gain!insight!into!the!evaluation!values!underlying!

partner!needs!and!agendas!for!participating!artists,!organisations!and!community!participants.!

The!stakeholder!evaluation!needs!and!expectations!expressed!by!participants!were!invariably!

influenced!by!past!experience!of!evaluation!and/or!the!values!of!the!organisation!stakeholders!

represented.!One!interesting!theme!arising!from!the!data!was!the!trepidation!expressed!by!multiple!

stakeholder!participants!towards!using!community!arts!evaluation!at!all.!One!creative!practitioner!

for!instance!questioned!how!formal!evaluation!could!contribute!meaningfully!towards!professional!

learning!and!future!practice,!while!another!rejected!outright!the!legitimacy!of!evaluation!practice!for!

community!arts!altogether:!‘I!don't!like!evaluations,!I!don't!like!those!standard!evaluations!and!I!

particularly!don't!like!arts!funding!body!evaluations’.!

Included!were!questions!relevant!for!establishing!baselines!for!appraising/!gauging!the!project’s!

impact,!outcomes!and!success.!What!would!success!look!like!to!different!stakeholders?!Interestingly,!

this!area!of!the!consultation!demonstrated!that!many!participants!lacked!clarity!about!how!the!

evaluation!could!best!serve!them!or!their!organisation!and!how!to!articulate!project!indicators!for!

success.!In!other!cases!however,!particularly!for!the!institutional!partners,!some!stakeholders!were!

able!to!clearly!describe!what!evaluation!areas!were!important!for!them!and/or!their!organisation.!

Consider!these!three!examples!from!different!stakeholder!participants.!I!have!anonymised!the!

participants,!and!each!of!the!following!statements!are!from!different!interviewees:!!

#1+Extract+from+interviewee+1+

…and!this!means!concentrate!on:!A*!the!quality!and!the!vibrancy!of!the!work!that!is!produced;!

B*!the!relevance!to!the!community,!and!I!mean!that!in!its!broadest!possible!terms,!and!in!this!

case!relevance!to!the!constituency;!C*!the!developments!in!capacity!of!participants.!!


For!me!it’s!the!transformational,!it’s!the!transaction!and!the!transformation!that!comes!from!

that!transaction.!If!people!have!been!through!a!process,!or!witness!an!artwork!and!it!has!left!

them!in!a!different!place,!that!transformative!exchange!is!success.!!!


! 96!


Evaluation!is!only!relevant!in!terms!of!what!were!the!objectives!and!how!do!you!measure!

against!those!objectives.!!!

!

Another!purpose!of!scoping!was!to!gain!insight!into!the!potential!utility!and!value!READ!processes!

and!outputs!may!have!for!partners.!Through!the!interviews!and!consultation,!I!asked!partners!about!

a!wider!range!of!considerations,!beyond!evaluation*driven!processes!and!outputs,!for!example!

exploring!how!reflective!practice,!academic!analysis!or!creative!documentation!might!be!relevant!for!

their!practice!or!organisation.!Preferred!end*products!and!content!formats!were!discussed,!including!

anticipated!partner!needs!for!confidentiality!or!anonymity!for!content!expected!to!be!generated!

through!READ.!It!was!identified!that!not!all!partners!were!interested!in!traditional!evaluation!

findings!or!reporting!(based!on!the!notion!of!attributing!merit!or!worth!about!the!activities).!For!

example,!the!Balaklava!community!partners!from!the!nuclear!veteran!community,!some!of!the!

creative!team,!and!the!regional!institutional!partners!fell!in!this!category.41!!Partners!were!instead!

interested!in!story*based!outputs!able!to!capture!the!unique!or!powerful!moments!for!participants!

or!that!offered!short!case!examples!of!any!outstanding/!transformative!aspects!of!the!collaboration.!

Collectively,!there!were!a!number!of!needs!and!preferences!arising!from!the!scoping!that!informed!

the!design!of!the!READ!methodology!for!Balaklava,!including!stakeholder!requests!for:!

•! formative!and!summative!outputs,!including!project!reporting,!documentation,!findings!

•! stories!from!the!project,!story*based!examples!

•! evaluation!and!documentation!for!acquittal!

•! use!of!practice*informed!methods!arising!from!the!creative!practice!

•! community!archiving,!broader!learning!and!dissemination!outcomes,!so!information!is!

shared!beyond!site!of!practice!

•! end*products!and!information!that!can!be!used!for!continuous!improvement,!decision*

making,!new!creative!work,!future!funding!applications!and!to!enhance!partnership!

development!and!capacity!building!!

•! creative!documentation!and!web*based!outputs!that!link!with!community!stories.!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

41!More!generally!for!Nuclear!Futures,!another!example!was!the!Yalata!Aboriginal!community!partner,!not!interested!in!

evaluation!reporting!rather!creative!documentation!and!co*ownership!of!raw!data,!of!the!stories!that!arise!out!of!

community.!


! 97!

Using!forensic!reflective!practice,!I!analysed!the!site*specific!needs!and!issues!revealed!during!the!

consultation!for!the!Balaklava!Community!Arts!project!to!identify!the!following!list!of!key!challenges!

that!I!would!look!to!address!through!evaluation!practice!via!the!READ!application:!

•! Lack!of!clarity!by!some!stakeholders!about!their!evaluation!needs!and!measures!of!success,!

and!the!integration!of!divergent!stakeholder!evaluation!values!and!goals.!

•! Potential!lack!of!evaluation!engagement!capacity,!as!identified!by!some!institutional!and!

community!partners!due!to!policy,!resource!or!time!constraints!in!relation!to!the!project.!

•! Prevalent!trepidation/!scepticism!by!some!artists!about!the!value!and!meaningfulness!of!

formal!and/or!traditional!evaluation!(especially!if!driven!by!bureaucratic!reporting!agendas).!

•! The!need!to!negotiate!the!funding!agency’s!independent!evaluation!requisite!with!the!in*

house!experimental!approach!under!development!by!Alphaville.!

•! Legal!and!ethical!considerations,!such!as!ensuring!permissions!for!participation!in!

documentation,!questionnaire!and!interviews!for!under!18!years!of!age!participants.!!

•! Time!and!resource!management!of!evaluation!implementation,!partnership!brokerage,!

community!engagement/!participation,!and!the!dissemination!of!evaluation!findings.!

•! The!need!to!reflect!and!embody!the!creative!practice!through!the!evaluation!practice,!to!

diversify!evaluative!outputs!beyond!a!written/reporting!format.!

In!synthesising!the!feedback!from!the!early!consultation!and!analysis,!it!was!evident!that!the!READ!

premise!to!create!a!wide!summative!lens!for!understanding!and!representing!Nuclear!Futures’!

stories!and!impact!was!highly!relevant.!The!register!of!voices!within!the!partnership!collective!called!

for!a!creative!and!engaging!framework!that!extended!beyond!traditional!evaluation!reporting.42!!

The!findings!arising!from!the!scoping!phase!with!stakeholders!critically!informed!my!evaluation!

practice!and!the!collective!design!and!delivery!of!READ!in!action!for!the!Balaklava!Community!Arts!

Project.!The!resulting!READ!methodology!was!tailored!to!suit!the!partner!needs,!concerns!and!

preferences!identified!for!the!creative!community!partnerships,!and!is!presented!next!in!Part!B!of!

the!chapter.!

!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

42!The!terminology!‘register!of!voices’!became!terminology!within!Alphaville!for!the!interviews,!images,!creative!works,!

documents,!press!material!collated!in!the!Nuclear!Futures!archive!and!documentation.!


! 98!

Part#B:#READ#methodology#in#action#

5.4! Design#and#implementation#

!

The!information!generated!from!scoping!phase!allowed!me!to!understand,!articulate,!share!and!

negotiate!with!stakeholders!the!emergent!ideas!for!evaluation!practice!within!the!collective.!This!

process!helped!me!to!establish!a!shared!purpose!and!short*list!of!the!collective!evaluation!needs!for!

the!READ!application!for!the!Balaklava!Community!Arts!Project.!Based!on!the!scoping!findings,!I!

devised!a!process!for!further!engaging!stakeholders!to!input!into!the!design!of!a!reflective!and!

reflexive!methodology!for!the!Balaklava!project,!covering!how!we!could!approach!data!collection,!

analysis,!knowledge!generation!and!sharing!within!and!across!each!of!the!READ!strategies.!!In!terms!

of!the!overall!DCI!research!progression,!this!methodological!work!on!the!READ!concept,!included!in!

this!section!(Section!5.4),!constitutes!the!planning,!design,!and!implementation!phases!of!the!READ!

application!within!the!broader!structure!of!my!DCI!research.!

Discussed!first!are!the!support!structures!that!were!designed!to!scaffold!the!four!enquiry!strategies!

of!the!READ!model!–!Reflection,!Evaluation,!Analysis,!and!Documentation.!To!illustrate,!I!have!

included!examples!canvassing!stakeholder!engagement!and!partner!brokerage,!information!and!

management!systems,!and!conduct!for!ethical!practice!(Section!5.4.1).!!!

Next!each!of!the!READ!strategies!are!reviewed,!to!reveal!how!each!was!approached!as!a!

complementary!yet!independent!platform!for!capturing,!monitoring,!understanding!and!assessing!

the!partnerships!and!creative!outputs.!!In!summary,!the!READ!strategies!are!as!follows:!

•! Reflection!was!directed!toward!establishing!a!culture!of!reflective!practice.!

•! Evaluation!was!driven!by!a!set!of!key!evaluation!questions.!

•! Analysis!via!academic!research!aimed!to!generate!broader!learning!and!dissemination!of!

original!knowledge.!

•! Documentation!was!guided!by!creative,!program!and!legacy!goals!set!at!the!site!of!practice!

for!community!story*telling!and!art*making!purposes.!!

These!concurrent!strategies!meant!that!the!emergent!epistemology!and!methodological!design!for!

the!implementation!of!the!READ!approach!supported!more!than!the!traditional!aspects!of!evaluation!

practice!(often!narrow,!merit*driven!and!empirically!based)!while!extending!the!scope!of!practice!to!

explore!and!embrace!constructivist!approaches!for!collective!learning!and!project!review.!

!


! 99!

5.4.1! Support#structures#

Underpinning!the!READ!application,!across!the!four!concurrent!READ!strategies!(Reflection,!

Evaluation,!Analysis!and!Documentation),!were!a!number!of!agreed!principles,!organisational!

systems/!procedures!and!strategic!plans!necessary!for!delivery.!I!refer!to!these!as!READ’s!support!

structures!as!they!constituted!the!scaffolding!for!the!delivery!of!the!READ!strategies!(however!within!

the!list!of!inclusions!are!non*structural!elements!such!as!a!code!of!conduct).!To!provide!insight!into!

the!supports,!I!have!listed!a!selection!below!that!I!initiated!anew!for!the!Balaklava!project!or!

tailored/refined!from!Nuclear!Futures!program!management!systems.!The!selected!examples!are!

grouped!into!three!categories:!

•! Stakeholder!engagement,!participation!and!partnership!brokerage!

•! Information!and!management!systems!

•! Procedures!for!ethical!practice.!

Below!are!examples!of!the!types!of!support!structures!that!we!put!in!place!for!the!application.!

Stakeholder*engagement,*participation*and*partnership*brokerage*

•! A!program!of!education!and!engagement!supported!participation!and!inclusion!in!READ!

activities!via!education!and!training,!induction,!and!professional!development.!For!example,!

mentoring!relationships!were!facilitated!between!established!and!emerging!artists!for!skills!

development!dedicated!to!improving!documentation!and!data!collection!techniques/skills.!

•! Selected!internal!champions!were!asked!and!supported!to!promote!READ!and!to!directly!

participate!in!the!design!and!delivery!of!activities,!to!encourage!community!and!institutional!

partner!involvement.!For!example,!Alphaville’s!Creative!Producer,!several!Nuclear!Futures!

artists,!a!SLSA!representative!and!the!local!Balaklava!coordinator!helped!to!secure!wider!

involvement.!

•! Appointment!of!a!local!project!coordinator!(for!permanent!part*time!work!across!the!project!

including!on!READ!activities!in!Balaklava)!advanced!local!relationships!and!trust!for!data!

collection.!

•! The!READ!communication!strategies!were!linked!with!those!from!project!management!based!

on!the!partner!needs!identified!in!the!scoping!consultation!(such!as!the!timing,!format!and!

content!of!communications!with!partners/stakeholders).!For!example,!when!significant!READ!

findings!were!identified!or!key!outputs!created!for!one!partner,!where!appropriate,!this!

material!was!released!for!feedback!via!internal!dissemination!and!public!sharing!(where!

appropriate)!using!partner!networks,!website,!social!media,!and!email!groups.!


! 100!

Information*and*management*systems*

•! Digital!content/data!system!and!procedures!for!managing!data!retention,!storage,!archive!

register,!ownership,!security!and!confidentiality.!For!example,!both!centralised!and!

decentralised!digital!archiving!systems!for!READ!data!were!created!to!ensure!reliable,!high!

quality!and!durable!data!formats!and!processes!for!back*ups,!with!medium!and!long*term!

storage.!For!example,!a!long!term!repository!for!selected!documentation!was!established!

with!the!SLSA!collection!development!team!to!create!public!access!to!some!material,!and!

a!comprehensive!internal!archiving!system!was!managed!catering!for!the!large!quantity!

of!READ!data!generated!which!linked!with!the!Nuclear!Futures’!overall!content!

management!plan.!(RIPP:!Article!5,!p.12!is!the!developed!Digital!content!and!storage!plan.)!

•! Setting!up!and!moderating!online!network!and!forum!applications/platforms!that!supported!

partner!exchange!and!enabled!communications,!document*sharing!and!collective!review!

processes!(such!as!Dropbox,!wiki!page,!email!group!lists,!Facebook,!Skype).!

•! Stakeholder!contact!database!with!necessary!status!and!information!on!staff,!partners,!

volunteers,!and!working!groups!(such!as!the!project’s!advisory!group!of!artists,!nuclear!

veterans,!researchers!and!project!coordinators)!was!established.!

(RIPP:!Article!3,!p.10!includes!extracts!from!databases!and!profile!summaries.)

•! Management!records!for!READ*related!finances,!copyright,!permissions/!contracts,!legal!

matters!and!overall!logistical!aspects!of!READ!operation,!linked!to!the!project’s!management!

record!system.!!

Procedures*for*ethical*practice*

•! Code!of!conduct!was!based!on!Australasian!Evaluation!Society’s!(2010)!Guidelines%for%the%

Ethical%Conduct%of%Evaluations.%For!example,!consent!forms!and!protocols!for!ethical!

research!and!evaluation!conduct!ensured!READ!participants!had!sufficient!explanation!of!the!

purpose,!end*uses!and!audience!for!data!collected!(and!implications!of!the!research),!and!

handling!of!participants!under!18!years!of!age/parental!consent!processes,!and!privacy!and!

confidentiality!needs/preferences.!

•! Permission!protocols!for!decentralised!co*authorship!and!distribution!was!required!for!the!

intended!inclusion!of!the!project’s!register!of!voices!to!ensure!appropriate!acknowledgment!

and!source!identification,!unless!anonymity!was!requested,!for!short!and!long*term!

purposes.!Different!approaches!were!used!for!public!and!partner!participants.!(RIPP:!

Article!7,!p.15!is!an!example!of!a!Release!form!created!for!documentation!activities.)!!

!

!

!


! 101!

5.4.2! Reflection#

Approach*

The!Reflection!strategy!aimed!to!inspire!a!culture!of!reflective!practice!for!artists,!community!

partners!and!participating!stakeholders,!to!make!explicit!practitioner!and!participant!knowledge!

generated!through!the!arts!practice.!Through!reflective!practice,!the!implicit!knowledge!embedded!

in!professional!practice!was!able!to!be!unearthed,!critically!examined,!and!made!applicable!as!new!

knowledge.43!Underpinning!READ’s!reflective!enquiry!is!Schon’s!notion!that!‘our!knowing!is!in!our!

action’!(Schon,!1983,!p.49)!and!that!reflective!practice!is!valuable!in!the!collective!context!(Finlay,!

2008;!Schratz,!1993).44!Schratz!(1993)!describes!how!reflection!in!the!collective!context!has!the!

potential!to!span!three!levels!of!application:!micro!analysis!for!individual!professional!development;!

medium*range!reflection!for!formative!evaluative!and!collective!benefit;!and!macro!analysis!as!a!

summative!instrument!for!organisational!learning!and!sustaining!collaborations!in!the!long!term.!!

For!the!application,!artists,!art!workers!and!community!partners!were!invited!to!provide!reflection!

during!different!stages!or!milestones!of!the!projects!paying!critical!attention!to!the!‘practical!values!

and!theories’!of!the!arts!practice!and!experience!(Bolton,!2010,!p.!xix).!In!this!way,!reflections!

generated!both!formative!and!summative!data.!An!example!of!formative!data!arising!from!READ!is!

demonstrated!in!Article!10,!where!an!artist!reflects!on!a!training!session!and!includes!valuable!

ideas!relevant!to!the!unfolding!creative!development!and!arts!practice!(RIPP:!Article!10,!p.27).!

This!differs!to!the!summative!data!in!Article!11,!from!reflection!content!submitted!at!the!

completion!of!the!public!presentation!of!the!artwork,!which!produced!information!of!relevance!

for!the!arts!organisation!and!for!future!productions!(RIPP:!Article!11,!p.30).!!

The!reflective!practice!sought!to!capture!and!scrutinise!the!progress!of!the!arts!practice,!examining!

activities!and!achievements,!problematic!blocks,!strengths,!failures,!impact!of!practice!and!the!

articulation!of!the!innovative!aspects!of!CACD.!Reflections!were!undertaken!individually!via!

journaling!and!video,!and!as!a!shared!process!through!facilitated!meeting!and!semi*structured!group!

review/debrief!on!practice.!Reflection!was!also!compatible!for!community!partners,!and!able!to!be!

incorporated!into!a!variety!of!the!art*making.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

43!!See!Chapter!3!for!full!theoretical!discussion!of!reflection,!as!part!of!the!DCI!methodology!chapter;!and!see!Bourner!

(2003)!and!Schon!(1983).!44!There!were!a!number!of!useful!participatory!frameworks!for!collective!reflective!practice!relevant!that!I!consulted!during!

the!design!phase,!such!as!Finlay’s!2008!model!‘5!variants!of!reflexivity’!and!Schratz’s!1993!model!detailing!three!strategies!

for!collective!self*reflection,!coming!from!the!field!of!education!research.!

!

!


! 102!

The!Reflection!strategy!was!also!underpinned!by!Alphaville’s!organisational!values!and!ethos,!

including!the!assertion!that!the!project’s!creative!team!are!the!experts!of!their!own!practice,!and!

that!their!insights!about!the!participants,!the!art*making!and!relationships!would!be!essential!for!

understanding!and!representing!the!project.!Artists!and!artworkers!know!who!showed!up,!how!

often,!and!how!participants!interacted!with!the!CACD!processes!and!other!participants!(Keating,!

2002).!In!terms!of!the!creative!process!itself,!Carter’s!influential!work!on!‘material!thinking’!explains!

the!role!of!reflection!when!creating!art,!an!inherent!process!integral!to!the!art*making!and!learning*!

that!'creative!research!is,!in!itself,!an!act!of!reflection!and!invention'!that!produces!knowledge!

(Carter,!2004,!p.191).!While!critics!and!researchers!can!describe!or!evaluate!the!creative!process!

from!the!outside,!based!on!the!final!outcomes!produced!(the!artwork!work),!it!is!the!artists!and!

community!partners!making!the!creative!work!who!are!able!to!productively!reflect!on!the!creative!

thinking!and!impact!of!their!processes.!!

For!the!creative!team!leading!the!community!arts!residencies!in!Balaklava,!the!Reflection!strategy!of!

READ!tapped!into!the!internal!and!tacit!evaluative!processes!in!operation!during!the!art*making,!to!

generate!insights!that!informed!the!creative!decisions!about!the!arts!practice!and!engagement/!

participatory!processes!with!community!(what!might!work!or!not,!and!why).!Making!such!reflection!

explicit,!using!systematic!processes!and!methods,!became!a!means!to!utilise!the!'intellectual!work!

that!usually!goes!missing!in!translation'!during!creative!development!(Carter,!2004,!p.xiii).!

To!build!a!culture!of!reflection,!I!invited!the!creative!team!members!and!community!partners!to!

participate!in!a!voluntary!program!of!reflective!practice!from!the!project’s!commencement,!to!be!

undertaken!during!or!following!the!art*making.!Where!necessary,!I!provided!information!and!

examples!to!show!how!reflective!practice!could!be!undertaken!and!provided!ongoing!support!(and!

reminders)!to!facilitate!reflection!being!conducted.!Early!in!the!project,!a!shared!understanding!was!

established!amongst!reflection!participants!about!when!and!how!reflective!practice!could!be!

conducted!and!utilised.!

At!the!organisational!level,!a!system!of!interrelated!and!iterative!processes!was!created!to!support!

reflection,!participation!and!information!management!(see!Sections!5.4.1!and!5.4.5).!Of!importance!

to!the!success!of!the!collective!reflective!practice!was!the!early!establishment!of!clear!protocols!for!

how!collected!reflections!would!be!used!and!handled!to!create!a!safe!and!ethical!environment!

based!on!consent.!I!was!able!to!use!my!dual!roles!as!READ!facilitator!and!program!manager!to!

encourage!and!support!collective!reflection,!and!became!the!central!agent!connecting!READ!with!

the!arts!practice!and!management.!The!early!benefits/relevance!of!the!reflective!processes!were!

immediately!evident!to!myself!and!several!core!participants,!and!this!created!shared!momentum.!


! 103!

Methods*and*analysis*

Data!collection!via!reflection!began!as!an!organic!process,!that!was!refined!and!systematised!as!the!

READ!application!proceeded.!Multiple!methods+were!used!by!those!participating!in!reflective!

practice,!selected!based!on!their!suitability!for!participants!and!participant!preferences,!and!for!the!

quality!and!depth!of!the!reflection!they!could!deliver.!Reflective!methods!were!applied!to!individual!

and!collective!reflections!as!a!technique!used!for!planning!ahead!of!activities,!as!an!evaluative!tool!

during!creative!residencies,!and!as!a!summative!lens!following!an!event/activity.!Reflections!were!a!

mix!of!pre*planned,!guided!(probe!questions),!spontaneous!and!self*guided!responses!made!by!the!

reflective*practitioners.!Together!the!reflection!methods!generated!13!written!reflections,!many!

hours!of!video!content,!extensive!photomedia!documentation,!online!and!website!content,!and!a!

collection!of!quotes/!extracts!from!speeches,!project!documentation!and!communications.!!!

(RIPP:!Section!B,!p.25!includes!a!photo!overview!and!reflection!data!examples.)!

Through!my!own!reflective!practice,!I!analysed!the!collective!reflection!undertaken!during!the!

project’s!reflective!cycles!to!gain!an!understanding!of!the!suite!of!methods!being!applied!and!

outputs!generated.!I!identified!the!two!main!types!of!data!collection!for!Reflection!as:!

•! Active!reflection!collection:!Self!guided,!semi*structured!and/!or!facilitated!reflection,!

including!written!reflections!(post!practice),!website!blogs!(prior,!during,!or!post!practice)!

video!interviews!and!vox!pops!(during!and!post!practice),!facilitated!collective!sessions!(pre,!

during!and!post!practice),!creative!documentation!(photography!and!video)!and!excerpts!

from!oral!history.!

•! Passive!reflection!collection:!capture!of!informal!reflective!comments!arising!from!arts!

practice!or!project!management!(from!communications,!speeches,!events/workshops,!

meeting!notes,!oral!history,!project!documentation,!field!notes,!observation,!media!and!

social!media.)!

Of!the!methods!which!I!have!categorised!as!active,!the!most!significant!and!productive!were!the!

written!reflection,!interview!and!collective!reflection.!Written!reflections!were!submitted!from!nine!

members!of!the!Nuclear!Futures!creative!team!(artists!and!managers/!administrators),!two!

community!participants,!and!one!collaborating!external!artist.!Reflection!was!invited!at!the!

completion!of!a!residency!or!event,!with!the!insights!generated!being!fed!into!the!subsequent!

program!of!activities!and!concurrent!management!practices.!In!practical!terms,!reflection!invitations!

and!reminders!were!mainly!sent!via!email!or!through!phone!communication.!The!entries!received!

were!predominantly!unstructured/!self!guided!or!semi*structured/guided!written!pieces!(between!1*

!


! 104!

6!pages!in!length)!with!two!the!product!of!oral!history!interviews!that!were!collected!and!

transcribed!by!community!partners.!Where!appropriate,!and!if!consent!was!given,!the!written!

reflections!were!shared!with!others!in!the!creative!team!and/or!delivery!partners.!

For!the!reflective!interviews,!project!documenters!(Nuclear!Futures!artists)!conducted!video!

interviews!with!all!in!the!creative!team,!as!well!as!community!and!delivery!partner!representatives.45!

The!video!interviews!were!an!effective!method!for!collecting!oral!reflections!from!community!

partners!and!from!key!players!not!participating!in!the!written!reflection.!Not!only!did!it!save!

community!members!and!artists!the!task!of!writing!down!their!reflection,!the!documented!content!

became!immediately!relevant!for!documentary!art*making!and!evaluation!purposes.!Video!

reflections!meant!reflective!practice!was!linked!with!creative!documentation!and!art*making,!and!

was!particularly!engaging!for!creative!team!members!as!a!means!of!reflective!story*telling.!Overall,!

the!video!method!generated!hours!of!reflective!content!from!over!15!stakeholder/partner!

representatives.!Reflective!content!was!also!extracted!from!shorter!vox!pop!videos!undertaken!

with!community!participants!and!audiences.!(RIPP:!Article!12,!p.32!includes!an!excerpt!from!a!

guided!interview!reflection.)!

For!the!reflection!content!that!I!refer!to!as!being!passively!collected,!the!quality!and!quantity!of!data!

captured!from!this!collection!was!significantly!aided!by!my!internal!positioning!as!READ!facilitator!

combined!with!my!role!as!program!manager.!Such!positioning!afforded!me!access!to!a!wide!range!of!

content,!across!most!project!areas!and!from!diverse!sources,!which!resulted!in!the!capture!of!

valuable!embedded!reflective!content!from!the!arts!practice!and!program!management.!(RIPP:!

Articles!13*4,!p.33*34!are!passive!examples!from!a!blog,!an!interview!and!a!public!speech.)!

Results**

Reflective!practice!offered!those!participating!in!the!community!arts!project!a!range!of!methods!that!

encouraged!them!to!stand!outside!the!arts!practice!to!gain!different!perspectives,!in!a!safe!and!

structured!way.!Feedback!from!community!and!artist!participants!indicated!that!reflective!practice!

produced!benefits!at!the!individual!level,!and!promoted!organisational!learning!for!Alphaville!and!

the!Nuclear!Futures!team.46!For!example,!two!artists!stated!the!READ!structure!encouraged!them!to!

reflect*on!practice!in!a!new!way,!and!where!they!otherwise!would!not!have!made!time,!and!that!

they!appreciated!the!opportunity!to!articulate,!analyse!and!share!their!reflections.!For!another,!a!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

45!While!the!reflective!video!interviews!have!been!categorised!as!in!this!section!as!reflection,!they!do!overlap!in!several!

ways!with!interviews!conducted!as!part!of!the!vox!pop!method!outlined!in!Evaluation!section,!considered!separately.!46!Based!on!post*project!consultation!and!interviews!conducted!for!this!research.!

!

!


! 105!

community!participant,!the!invitation!to!conduct!reflective!practice!was!seen!as!a!positive!

reinforcement!that!her!thinking!and!contribution!were!acknowledged!and!valued!in!the!arts!practice!

within!Nuclear!Futures.!

A!significant!proportion!of!the!reflection!content!generated!by!community!and!artist!participants!

was!of!immediate!relevance!for!the!arts!practice,!management!and!partnership!brokerage!for!the!

Balaklava!project.!Through!reflection,!community!participants!brought!unique!perspectives!to!the!

community!arts!practice!and!the!storytelling!of!their!personal!journey!was!rich!and!insightful!(see!

excerpt#1!below).!Feedback,!issues!and!ideas!raised!in!the!reflections!(about!technical,!creative,!

inter*personal,!philosophical!or!production!related!aspects!of!project)!were!able!to!be!incorporated!

into!collective!domain!and!decision*making!as!necessary!(see!excerpt!#2!below).!In!particular,!the!

information!and!perspectives!forwarded!through!collective!reflection!sessions!with!the!creative!

team,!stimulated!creative!and!innovative!shifts!in!practice!on!many!occasions,!which!Haseman!and!

Mafe!(2009)!acknowledge!as!one!of!the!benefits!arising!from!practice*informed!reflective!enquiry.!

For!me,!as!program!manager!and!READ!facilitator,!having!real*time!access!to!the!data!arising!from!

reflection!was!valuable!and!productive,!and!the!reflection!content!was!frequently!relevant!to!the!

other!enquiry!goals!of!READ!for!evaluation,!analysis!and!documentation!(see!excerpt!#3!below).!

Below!are!content!excerpts!derived!from!two!different!written!reflection!methods!and!are!

illustrative!of!the!rich!evaluative!content!arising!from!the!Reflection!strategy.!The!first!two!extracts!

are!from!unstructured,!self*guided!reflections!where!participants!approached!the!reflection!‘without!

necessarily!knowing!where!it!will!lead’!(Boud,!2001,!p.12).!The!third!is!written!in!response!to!a!series!

of!semi*structured!evaluative!questions!that!I!had!circulated!after!one!of!the!public!events!in!

Balaklava,!abbreviated!by!the!reflective!practitioner!to!guide!reflection,!again!producing!relevant!

evaluative!data.!They!are!included!to!demonstrate!the!diversity!of!themes!raised!during!collective!

reflective!practice,!and!the!varying!levels!of!depth!and!consideration!exhibited.!

#1+Excerpt+from+written,+selfHguided+reflection:+Community+participant+

Once!again!I!have!learnt!more,!met!more!people,!and!have!been!able!to!connect!other!

descendants!…!and!find!emotional!support.!It!never!surprises!me!that!when!the!descendants!

get!together!we!can!compare!similar!health!issues!and!find!a!common!thread!among!us.!I!

met!more!women!with!health!issues!as!a!result!of!their!father’s!involvement!with!the!tests...!

It!is!consistently!the!daughters!with!health!issues.!!I!listened!as!the!women!discussed!not!

being!able!to!have!children!and!wondered!how!can!that!just!be!a!coincidence!when!there!

are!only!a!handful!of!us!in!the!room!and!the!stories!are!so!similar?!!

To!know!that!there!are!people!in!the!community,!who!through!their!talent,!compassion!and!

creative!arts!want!to!tell!the!story!of!the!atomic!survivor!communities!and!the!generational!

impact!of!the!British!Atomic!Tests,!helps!to!ease!the!burden!on!the!veterans!and!their!

descendants.%


! 106!

#2+Excerpt+from+written,+selfHguided+reflection:+Artist,+Nuclear+Futures+creative+team+1!

From!the!decision!to!create!the!circular!set!design!to!actually!building!it!took!more!out!of!me!

than!I!imagined!it!would.!I!was!really!excited!that!we!had!the!opportunity!to!create!the!

circular!projection!space.!The!work!involved!did!out!weigh!the!economic!benefits!as!far!as!

time!for!me!but!creatively!and!technically!it!was!extremely!worth!it.%%

#3+Excerpt+from+written+semiHstructured+reflection:+Artist,+Nuclear+Futures+creative+team+2!

What!are!your!impressions!of!how!the!material!worked!with!audiences?:!110%!successful.!

Everyone!loved!it,!including!the!sceptics!and!semi!nuclear*pro.!Some!attended!the!talks!and!

were!immediately!defensiveness,!but!all!eventually!opened!up!and!…!seemed!to!be!taken!

aback.+

Ideas!for!improvement?:!I'm!really!impressed!with!the!running!of!everything,!the!

organisation,!and!the!level!of!enthusiasm!and!cooperation!from!everyone.!Keep!it!up,!and!all!

will!be!well.!

What!did!you!gain!professionally!or!personally?:!Avon!and!the!NF![Nuclear!Futures]!team!

reminded!me!of!the!importance!of!speaking!out,!standing!up,!and!the!importance!of!

creating!art!that!captures!the!story!of!people!who!are!brave!enough!to!do!that.!!!!!!!!!!!!

!

While!some!content!needed!sensitive!handling!and!confidentially,!the!reflection!data!was!

predominantly!shared!where!possible,!especially!between!the!creative!team.!What!evolved!were!

distinct!cycles!of!reflection!linked!to!the!delivery!of!the!creative!residency!program!and!iterative!

feedback!loops!with!program!management!and!arts!practice.!These!reflection!rounds!are!

summarised!in!Figure!5.4.!Aligned!with!Schratz’s!(1993)!three!tiered!notion!of!collective!reflection,!

the!reflective!cycles!identified!for!the!Balaklava!project!produced!varying!levels!of!analysis,!

knowledge!generation!and!utility!(referred!to!in!the!diagram!as!the!primary,!secondary!and!tertiary!

cycles)!producing!real*time!and!longer*term!impacts!at!the!site!of!practice.!Impacts!included:!

improved!decision!making!and!program!management;!enhanced!creative!development;!individual!

and!collective!benefits!for!reflection!participants;!and!development!of!a!learning!culture!

(professional!development!and!organisational)!for!continuous!improvement.!By!the!end!of!the!

application,!Reflection!had!become!an!accepted!and!respected!protocol!of!the!community!arts!

practice!for!the!Nuclear!Futures!program.!

! !


! 107!

+

!

!

Figure!5.4:!Mapping!Reflection!within!READ!for!the!Balaklava!Community!Arts!Project!

!

!

Practitioner, Participant, Partner Reflections Active and passive reflective content is collected from individual or collective self guided, semi-structured, facilitated reflection methods (written, oral, video, documentation). Systems for skills development, support, participation, collection and feedback are in place to create safe, confidential/ permission-based context. Content is copyright of reflective practitioner with levels of permission granted for use. Valuable project relevant information is generated from different stakeholder perspectives.

Collection, Analysis, Reflection Information systems for ethical content handling are established & made transparent. Protocols for record keeping & content analysis are administered by program manager to identify actions and utility for stakeholders, project and broader program. Review/ analysis may be collective as needed. Emergent themes, issues and information are organised, & immediate to long-term actions are identified. Content (raw or adapted) is distributed (as appropriate) for feeding into arts practice or project management.

Sharing, Incorporation and Impact Active response to, & incorporation of, information/ new knowledge into creative practice, production and project management at site of practice and into broader program strategies. Distribution pathways are created, and information is tailored for recipients and purpose. Information may remain internal (confidential), be shared with selected partners/stakeholders, or be included in public outputs. Internal engagement and feedback on the generated findings is encouraged (as appropriate) and iterative refinement of emergent project knowledge is undertaken.

Enriched understanding into technical, creative, socio-dynamic & methodological aspects of the project & arts practice are gained from multiple perspectives. Findings have formative and summative relevance, and generate iterative cycles of reflection by reviewers & managers. Data relevant for evaluation, documentation & analysis are identified & archived contributing to the record of creative practice.

ITERATIVE PROCESSES OF REFLECTION & USE

SYSTEMS & INPUTS

Outputs from incorporated reflective information included new project knowledge, creative content, evaluation data, project documentation/ data; and led to iterative cycles of further primary & secondary reflection to emerging information. Impact at site of practice included benefits such as improved: project & program management decision-making; community arts practice; personal & professional development; participation & engagement; trouble-shooting; continuous improvement; organisational learning; quality of communication outputs; partnership brokerage; resource allocation; & human resource management.

OUTPUTS & IMPACT

Reflective practice generated: formative information for enriching/advancing arts practice and understanding of practice; professional and personal development; opportunity for participant feedback & engagement; opportunity for shared reflection; documentary content & project documentation; relevant evaluation data. The collective and repetitive cycles build a culture of reflection within the arts practice. !


! 108!

5.4.3! Evaluation#

Approach*

The!Evaluation!element!of!the!READ!methodology!employed!a!strategic!mix!of!evaluation!

instruments!to!respond!to!the!key!evaluation!questions!set!for!the!Balaklava!Community!Arts!

Project,!devised!and!implemented!in!consultation!with!the!program’s!contributors!and!partners.!The!

evaluation!questions!driving!the!appraisal!process!focused!on!project!impact,!how!well!the!project!

met!the!stated!objectives,!the!quality!and!depth!of!the!community!arts!practice!and!partnerships,!

and!areas!of!innovation!for!the!CACD!sector.!The!evaluation!activities!aimed!to!produce!

representative!and!defensible!findings!in!line!with!the!host!organisation!and!partner!needs.47!

Underpinning!this!strategy!was!a!conventional!definition!of!evaluation,!based!on!the+‘systematic!

collection!and!analysis!of!information!to!make!judgments,!usually!about!the!effectiveness,!efficiency!

and/or!appropriateness!of!an!activity’!(Australasian!Evaluation!Society,!2010,!p.3).!!

The!process!of!designing!and!delivering!the!Evaluation!strategy!of!the!READ!model!involved!a!

number!of!distinct!steps!that!I!facilitated!and!directed!in!collaboration!with!participating!

stakeholders!representing!Alphaville,!the!creative!team,!and!delivery!and!community!partners.!

Given!the!Balaklava!Project!was!the!first!of!the!Nuclear!Futures!projects!to!be!implemented,!there!

was!no!pre*existing!evaluation!framework!at!the!program!level!to!align!with!or!follow.!As!part!of!my!

evaluation!practice!it!was!first!necessary!to!facilitate!the!design!of!an!overarching!evaluation!plan!for!

the!Nuclear!Futures!program!(not!just!at!the!Balaklava!project!level)!so!that!consistency!across!

projects!was!achieved!for!the!later!summative!program!evaluation!for!Nuclear!Futures.!Below!are!

the!steps!I!followed!for!this!process,!as!part!of!the!DCI!research:!

•! Identify!and!consider!relevant!pre*existing!models,!methods!and!approaches!from!the!

Contextual!Scan!(discussed!in!Chapter!2).!

•! Synthesise!evaluation*related!information!from!the!scoping!research!(stakeholder!needs!

assessment!process,!questions!of!interest)!and!document!analysis!of!partner!agreements.!

•! Define!the!program!logic!and!hierarchy!of!planned!outcomes!for!the!Nuclear!Futures!

program.48!!

•! Formulate!the!key!evaluation!questions!(KEQs).!49!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

47!I!use!the!term!defensible!findings!to!mean!that!the!evaluative!conclusions,!findings!or!claims!arising!from!the!READ!

methodology!need!to!based!on!credible,!prudent!and!ethical!processes!of!corroborating!information,!to!ensure!the!

findings!are!valid!and!able!to!be!explained!and!supported!if!required!by!partners!or!external!parties.!48!The!objectives!and!hierarchy!of!outcomes!for!the!Nuclear!Futures!program!can!be!viewed!in!the!evaluation!plan!(READ%in%

Practice!portfolio)!and!further!information!about!the!project!is!contained!in!Chapter!4:!Site!of!Practice.!49!Key!evaluation!questions!are!a!short!list!of!questions!that!intended!users!of!the!evaluation!want!or!need!addressed!by!

the!evaluation!process!(Kotvojs!&!Hurworth,!2013;!Fitzpatrick!et!al.,!2012).!!


! 109!

•! Develop!a!framework!logic!and!methodology!for!the!overarching!evaluation!of!the!Nuclear!

Futures!program.!

•! Produce!the!Nuclear!Futures!evaluation!plan!(discussed!in!next!section).!

•! Tailor!a!project!plan!for!evaluation!suited!to!the!unique!evaluation!goals!of!the!Balaklava!

Community!Arts!Project,!based!on!the!Nuclear!Futures!evaluation!plan.!

•! Deliver/!implement!evaluation!over!the!Balaklava!Project!via!the!READ!framework.!

•! Disseminate!evaluation!findings!from!Balaklava!with!partners,!and!more!broadly!in!the!field.!

•! Refine!and!improve!the!overarching!Nuclear!Futures!evaluation!plan,!informed!by!ongoing!

feedback!and!results!of!the!Balaklava!evaluation!process.!

•! Undertake!ongoing!consultation/communications!between!evaluation!stakeholders,!and!

facilitate!participation,!for!all!of!the!above!milestones.!

For!the!Evaluation!strategy!within!the!READ!model!I!adopted!a!stakeholder!and!participatory*

oriented!approach!that!embraced!the!ethos!and!creativity!of!the!program’s!CACD!practice.!This!

meant!that!for!all!evaluation!activities!that!I!facilitated,!I!explored!how!the!activities!could!be!done!

to!draw!on!the!knowledge,!skills,!perspectives!and!participation!of!interested!partners.!To!support!

this,!I!provided!information!and!training!for!some!participants.!Varying!levels!of!participation!

ensued,!and!personnel,!artists!and!community!partners!were!actively!involved!in!four!ways:!

•! administrating!data!collection!(i.e.!delivery!of!planned!evaluation!instruments,!being!

opportunistic!to!lead!unplanned/!scheduled!data!collection,!using!arts!practice!to!capture!

data)!

•! participating!in!the!evaluation!research!(i.e.!being!interviewed,!doing!reflection,!giving!input)!

•! analysing!the!data!collected!(i.e.!reviewing/!assimilating!data,!debating/!establishing!

meaning)!

•! communicating!and!refining!findings!(i.e.!discussing!and!sharing!patterns,!themes!and!

information!arising!from!evaluation!data!internally!and!externally,!and!reflecting!on!

findings).!

(RIPP:!Article!15,!p.36!has!some!photos!and!short!descriptive!content!that!details!further!aspects!

of!the!approach.)!

Before!presenting!the!project*specific!methods!selected!for!Balaklava,!it!is!necessary!to!frame!

the!evaluation!practice!within!the!overarching!Nuclear!Futures!evaluation!plan.!The!next!section!

explains!some!of!the!methodological!details!of!the!plan,!before!discussion!turns!to!focus!on!the!

Balaklava!evaluation!and!results.!

! *

!


! 110!

Nuclear*Futures*evaluation*plan*

The!Nuclear!Futures!evaluation!plan!constitutes!one!of!several!significant!research!and!strategic!

planning!activities!undertaken!for!the!READ!application.!The!plan!is!included!in!the!READ%in%

Practice!portfolio!(RIPP:!Article!16,!p.37).!

The!comprehensive!evaluation!plan!included!a!guiding!evaluation!methodology!and!delivery!

schedule!for!the!Nuclear!Futures!Partnership!Initiative!between!2014*2017!for!evaluating!the!

multiple!projects!within!the!initiative,!of!which!Balaklava!Community!Arts!Project!was!the!first.!The!

evaluation!plan!(and!methodology)!sat!within!the!READ!framework!and!set!out!the!program’s!key!

evaluation!questions!(distinct!from!those!questions!driving!the!Reflection,!Analysis,!or!

Documentation!strategies).!While!the!evaluation!plan!was!a!stand*alone!document,!it!was!designed!

to!interrelate!with!the!other!READ!strategies,!and!within!the!suite!of!evaluation!methods!listed!in!

the!plan!were!methods!drawn!from!Reflection,!Analysis!and!Documentation.!

I!produced!the!Nuclear!evaluation!plan!primarily!as!an!in*house!guide!and!communication!tool,!and!

in!several!instances!I!re*formatted!or!abbreviated!the!versions!distributed!to!suit!different!partner!

information!needs.!The!plan!was!intended!to!be!dynamic!and!reflexive%and!has!had!several!rounds!of!

changes!and!refinements!over!the!past!two!years,!influenced!by!emergent!partner!input!and!

program!progress!(such!as!the!lessons!from!the!Balaklava!experience).!This!approach!to!the!

evaluation!plan!as!a!living!document!aligned!with!the!experimental!and!evolving!nature!of!Nuclear!

Futures,!and!the!ongoing!partnership!development!influencing!the!community!arts!practice.!#

As!was!outlined!in!the!section!on!scoping!(Section!5.3),!the!design!process!for!the!evaluation!plan!

involved!me!facilitating!a!process!of!collaborative!planning!and!development,!conducting!interviews!

and!integrating!data!from!the!contextual!scan!and!analysis!of!other!exemplar!program!evaluations.!

This!involved!wider!consultation!beyond!only!those!partners!relevant!to!the!Balaklava!project,!as!it!

sought!to!address!the!evaluation!needs!of!the!larger!Nuclear!Futures!collective!during!the!design!of!

the!evaluation!plan!methodology.!As!Jonassen!(1992)!points!out,!the!constructivist!approach!and!the!

contextual!influences!of!the!site!and!participants!involved!are!strong!determining!factors!in!

methodology!design,!which!was!the!case!for!the!Balaklava!project.!As!outlined!in!Chapter!4,!the!

wider!program!partners!included!diverse!cultural!groups!(Aboriginal,!non*Aboriginal,!and!

international!communities)!making!it!paramount!for!the!developed!READ!model!and!the!evaluation!

plan!to!integrate!a!cultural!lens!in!relation!to!evaluation!for!working!with!multicultural!groups!with!

different!needs.!The!resulting!epistemology!of!evaluation!was!thus!inclusive!of!community*based!

!


! 111!

understandings!of!program!impacts!and!these!include!indigenous!knowledge!and!the!hybrid!forms!

arising!from!collaborations!between!artists,!scientists!and!communities.!

While!it!is!beyond!the!scope!of!the!dissertation!to!comprehensively!discuss!the!evaluation!plan,!

noteworthy!are!the!four!key!evaluation!areas!identified!for!the!Nuclear!Futures!program.!

Specifically,!the!overarching!evaluation!methodology!sought!to:50!

1.! Assess!program!impact!

2.! Measure!program!implementation!against!stated!objectives!and!outcomes!

3.! Provide!a!critical!for!lens!for!community!arts!practice!!

4.! Encourage!skills!development,!capacity!building!and!information!sharing.!!

I!reduced!and!analysed!the!vast!scoping!data!and!discussed!a!short!list!of!key!evaluation!questions!

with!colleagues!to!identify!the!following!key!evaluation!questions!for!the!Nuclear!Futures!program:!

•! To!what!extent!did!the!program!meet!its!stated!objectives?!

•! How!effective!was!the!program!in!terms!of!generating!opportunities!for!individuals!and!

communities!to!actively!participate!in!excellent!artistic!practices?!

•! What!was!the!quality!and!depth!of!new!partnerships!formed!and!developed?!

•! In!what!ways!did!Nuclear!Futures!impact!upon!the!delivery!partners,!participating!groups!

and!the!wider!community?!

•! How!does!the!Nuclear!Futures!increase!understanding!of!the!contribution!of!the!CACD!

sector,!in!the!stated!areas!of!interest?!

Considered!in!the!methodological!design!process!for!the!evaluation!plan!was:!the!selection!of!

methods;!allocation!of!resources,!roles!and!responsibility;!delivery!time!schedule;!stakeholder!

engagement;!communications!plan!for!dissemination;!and!the!integration!of!the!evaluation!practice!

with!other!READ!and!arts!practice!activities.51!While!broad,!the!directives!and!key!evaluation!

questions!included!in!the!Nuclear!Futures!evaluation!plan!informed!the!unique!evaluation!needs!

identified!for!the!Balaklava!Community!Arts!Project.!A!process!of!tailoring!the!overarching!

methodology!set!out!in!the!evaluation!plan!was!undertaken!to!adapt!and!simplify!the!plan!to!meet!

the!needs!of!the!Balaklava!project.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

50!The!full!details!of!each!of!these!important!aims!for!the!evaluation!is!included!in!the!evaluation!plan!(p.10*11)!and!

provides!an!elaborate!definition!and!discussion!of!each.!51!The!length!of!the!evaluation!plan!is!longer!than!a!standard!Plan.!Abbreviated!versions!have!been!developed!to!suit!

partner!needs!during!consultation.!Alphaville!preferred!a!comprehensive!master!version!be!provided.!


! 112!

Evaluation*strategy*for*READ*in*Balaklava*

For!the!Balaklava!project,!the!evaluation!focused!on!how!well!the!proposed!activities!were!delivered!

and!the!value,!quality,!and!impact!of!outcomes!achieved.!As!detailed!in!Chapter!4,!a!central!project!

objective!was!to!create!regional!opportunities!for!arts!engagement!and!participation,!via!the!

production!of!creative!works!of!excellence!in!collaboration!with!community!and!other!partners.!!

In!addition!to!the!broader!program!evaluation!objectives!of!the!principal!funding!partner,!the!

Australia!Council!for!the!Arts,!of!relevance!were!also!project*specific!goals!for!Balaklava,!such!as!the!

committed!outcomes!in!the!project!funding!agreement!with!Arts!SA,!which!included!to:!

•! Involve!communities!in!the!creative!process,!direction!and!management!of!the!project!

•! Develop!and!support!the!skills!development!of!artists!working!in!and!with!the!community!

and!the!development!of!art!form!practice.!

•! Increase!opportunities!for!South!Australians!to!engage!with!the!arts!through!creative!

participation!in!line!with!social!inclusion!principle.!

•! Deliver!innovative,!community!based!arts!programs!with!long!term!outcomes!through!arts!

and!non*arts!partnerships.!!

•! Create,!develop!and/or!present!high!quality!work!that!is!recognised!for!its!artistic!merit!

•! Demonstrate!the!capacity!for!strong!operational!and!financial!management,!effective!

planning,!evaluation!and!use!of!resources.!!!!!!!

(Sourced!from!program!documentation,!courtesy!of!Alphaville).!

Unique!project!evaluation!goals!were!informed!by!the!community!partners!central!to!the!project’s!

development!and!delivery.!The!project!and!evaluation!goals!sourced!from!community!members!

were!not!couched!in!the!bureaucratic!language!of,!or!focus!on,!community!partnerships!but!rather!

connected!with!the!creative!and!social!justice!story*telling!capacity!of!the!project.!For!example,!in!

early!interviews!with!nuclear!veteran!Avon!Hudson,!when!asked!what!would!constitute!project!

success,!he!identified!his!want!for!the!stories!to!go!far!*!for!mainstream!audiences!to!be!engaged;!

for!public!awareness!about!Australia’s!atomic!test!history!to!be!increased;!and!that!public!

recognition!for!impacted!communities!be!promoted.!This!objective!to!make!the!story!go!far!was!

important,!and!was!incorporated!into!the!broader!program!and!Balaklava!project!rationale.!

In!assimilating!the!broader!Nuclear!Futures!evaluation!needs!and!unique!evaluation!goals!for!the!

project,!I!identified!five!priority!domains!for!appraisal!(i.e.!How!well!did!the!project!perform!for…):!

•! creative!community!partnerships,!

•! collaborative!art*making!and!cultural!development,!

•! artistic!vibrancy,!

•! engagement!of!regional!audiences!with!the!arts,!especially!young!people,!

•! making!the!story!go!far.!


! 113!


Across!March!2014!*!May!2015!participatory!processes!were!used!where!possible!with!core!team!

members!(artists!and!administrative!personnel)!to!employ!quantitative!and!qualitative!methods!

(including!rich!media)!for!the!multi*arts!activities!delivered.!Data!collection!methods!included!semi*

structured!interviews,!reflective!practice,!survey!instruments,!observation,!story*based!techniques,!

and!creative!documentation!such!as!film!and!photo!essays.!In!summary,!the!evaluation!findings!

were!based!on!the!analysis!of!data!captured!from!sources!such!as:!

•! Audience!numbers,!ticket!sales,!participant!registrations,!media!outreach!etc.!

•! Written!and!video!reflections!by!the!creative!team!and!community!partners.!

•! Feedback!forms!and!surveys!completed!by!participants!and!secondary!school!audiences.!

•! General!audience!post*event!feedback!forms!collected!during!the!season.!

•! Vox!pop!video!interviews!with!audiences,!post!event.!

•! Communications!and!informal!feedback!from!local!partner!organisations.!

•! Community!representation!for!management!and!working!groups.!

•! Observation,!reflective!practice!and!academic!analysis!by!the!lead!evaluator.!

•! Creative!documentation!(film,!photography,!video!interviews!etc.).!!

•! Story*based!techniques!(oral!history).!

(RIPP:!Article!17,!p.68!includes!examples!of!some!instruments!and!data!collection!methods.)!

The!methods!selected!were!in!keeping!with!the!envisaged!epistemology!of!the!READ!concept,!that!

seeks!to!foster!a!strengths*based!approach!utilising!the!multimedia!creative!documentation!skills!

and!processes!(such!as!documentary!photography!and!film!making)!embedded!in!Nuclear!Futures’!

art*making.!The!suite!of!creative!documentation!methods!employed!for!the!Evaluation!strategy!

naturally!overlapped!with!the!activities!of!the!Documentation!strategy!(see!Section!5.4.5).!This!is!

a!demonstration!of!the!way!evaluation!spanned!across!the!other!READ!strategies!(and!visa!

versa).!(RIPP:!Article!18,!p.69!provides!links!to!three!video!documentation!examples.)!!!

Information!generated!about!the!project!during!the!data!collection!phase!was!treated,!organised!

and!synthesised!to!generate!meaningful!and!robust!findings!in!response!to!the!KEQs.!My!role!

involved!managing!and!organising!the!data,!and!the!preliminary!task!of!reducing!the!data!collected!

prior!to!consultation!and!shared!processes.!I!also!used!triangulation!techniques!to!verify!data!

generated!from!different!sources.!The!aggregate!analysis!and!shared!review!undertaken!by!myself!

and!other!participants!(at!least!five)!used!processes!such!as!evaluative!reasoning,!impact!rankings,!

performance!rubrics,!and!partnership!mapping.!The!Artistic!Vibrancy!Framework!developed!by!the!

!

!


! 114!

Australia!Council!for!the!Arts!was!the!guiding!criteria!for!claims!and!evaluation!findings!on!the!

artistic!merit!of!the!works!produced.!The!analysis!also!included!consideration!of!the!broader!

implications!arising!from!the!work,!such!as!industry!accolades!and!feedback!from!prospective!future!

venues.!

For!each!of!the!five!domains!that!I!identified!as!priority!areas!for!appraisal,!I!drafted!a!descriptive!

account!of!what!was!achieved!and/or!delivered!through!the!project!and!integrated!the!evaluative!

findings!demonstrating!the!extent!or!quality!of!what!was!achieved!and!impacts!evidenced.!In!the!

domain!narratives!I!substantiated!the!emergent!evaluative!claims!with!specific!reference!to!data!

sources,!demonstrable!examples,!specific!quotes,!statistics!or!other!evidence.!I!shared!these!

evaluation!narratives!and!findings!for!feedback!with!core!creative!team!members!and!community!

partners.!In!doing!so,!the!content!reflected!my!own!interpretation!of!the!evaluation!information!

generated,!and!the!shared!values!and!consent!from!others.!My!intention!was!that!the!evaluation!

narratives!would!assist!in!profiling!the!project!outcomes!and!impact!of!most!interest!to!the!partners,!

and!create!material!relevant!for!the!program’s!summative!evaluation.!For!the!Balaklava!project,!no!

external!review!involving!an!independent!panel!and/or!evaluation!specialist!was!undertaken.!

An!example!of!the!domain!narratives!is!included!in!the!Acquittal!Report!for!Arts!SA!(RIPP:!Article!

36,!p.140).!Below!are!a!two!examples!of!the!evaluative!claims!resulting!from!the!process:!

The!collaborative!workshop*based!cultural!development!approach!employed!for!the!Field%of%View!exhibition!for!example,!was!highly!praised!by!participants,!community!partners!and!Wakefield!

Regional!Council.!Prevalent!from!the!results!of!the!participant!feedback!survey!was!consistent,!

positive!review!of!the!art*making,!training!and!mentoring!processes!facilitated!by!the!artists,!with!

‘skills!development’!and!the!‘opportunity!to!publicly!exhibit’!recorded!as!frequently!stated!

benefits!of!participation.!Short!reflective!interviews!with!participants!revealed!personal!and!

sometimes!transformative!impact!for!participants,!including!increased!confidence!for!art*making,!

advancement!of!individual!practice,!and!personal!rewards!from!having!their!work!celebrated!

within!the!community!context!(for!many!participants!this!was!the!first!time!their!work!had!been!

exhibited!in!a!Gallery).!

As!stand!alone!creative!works![10%Minutes%to%Midnight!showcase],!the!artworks!produced!fit!the!Australia!Council!for!the!Arts’!classification!for!‘great’!art.!As!defined!in!the!Australia!Council!for!

the!Arts!Artistic!Vibrancy!framework,!the!attributes!of!‘great’!art!is!that!it:!‘demonstrates!

integrity!of!process;!embodies!excellence!of!craft!and!skills;!demonstrates!imagination,!

distinctiveness!and!originality;!contributes!to!artistic!practice;!engages!with!the!diversity!and!

complexity!of!contemporary!life;!and!is!relevant!in!a!local,!national!and!global!context’!(taken!

from!the!e*book!introduction!http://www.australiacouncil.gov.au/strategies*and*

frameworks/artistic*vibrancy/).!These!attributes!are!evidenced!in!the!evaluation!findings,!and!

recognition!of!the!high!levels!of!artistic!merit!achieved!was!affirmed!by!audience,!partner!and!

team!member!reflections,!interviews,!feedback!forms!and!vox!pop!reviews.!Community!

sentiment!from!attending!nuclear!veterans!was!consistently!affirming!–!‘You!got!it!right’.!

(Extracts!from!the!Balaklava!Community!Arts!Project!Acquittal!Report!for!SA,!2015,!courtesy!of!

Alphaville.)!

!


! 115!

Results**

The!mixed!evaluation!methods!employed!generated!data!from!qualitative,!quantitative!and!

performative!sources.!!Where!possible!I!encouraged!collective!sensemaking!and!analysis!of!the!data!

collected,!to!ensure!different!perspectives!within!the!community!partnerships!were!included!in!the!

process!of!reviewing!and!synthesising!data.!Some!evaluation!findings!were!useful!immediately,!and!

other!data!got!analysed!through!longer!term!processes.!For!example,!the!results!from!the!school!

student!evaluation!for!the!10%Minutes%to%Midnight%showcase!were!used!in!real*time!by!the!creative!

team!to!adjust!the!contextual!information!provided!for!the!art!and!also!to!finalise!some!creative!

aspects!ahead!of!the!programmed!Fringe!festival!events.!(RIPP:!Articles!19,!p.70)!!

Evaluation!findings!included!statistical!statements!evidenced!on!survey!data!as!well!as!rich,!

qualitative!feedback!by!community!participants,!audiences!and!artists.!The!evaluation!outputs!and!

outcomes!generated!from!this!READ!strategy!were!multi*faceted!and!multiplatform!to!suit!the!key!

evaluation!audiences!and!partners!and!included:!

•! the!Nuclear!Futures!evaluation!plan!and!tailored!project!plan!for!Balaklava!

•! tested!survey!instruments!for!future!projects!(i.e.!audience!feedback!forms)!

•! video!documentary!content!of!artists,!partners,!community!participants,!audiences!

•! statistical!information!based!on!audience!feedback!(RIPP:!Article!20,!p.72!for!example.)!

•! testimonies,!letters!of!support,!case!studies!and!evaluative!findings!of!relevance!for!

future!arts!practice,!funding,!partnership!development!

•! report!documents:!monthly!reporting!for!Australia!Council!for!the!Arts!(included!evaluation!

findings!as!produced),!Acquittal!report!for!Arts!SA,!and!partner!summaries/!presentations!

to!local/regional!institutional!and!community!partners!

•! infographics!summarising!evaluation!findings!

•! skills!development!and!working!knowledge!of!group!processes!for!evaluation!practice.!

The!evaluation!outputs!were!distributed!as!necessary!to!the!partners!and!participants!involved.!

Feedback!and!consultation!following!the!dissemination!of!evaluation!findings!indicated!that!the!

outputs!and!content!met!partner!needs!and!expectations!for!the!evaluation,!and!for!several!

partners!evaluation!outputs!exceeded!expectations.!There!was!also!evidence!that!the!evaluation!

findings!were!of!immediate!use.!For!example,!a!number!of!compelling!themes!and!insights!emerged!

from!the!evaluation!that!were!the!basis!of!formative!discussions!by!the!creative!team!for!

subsequent!Nuclear!Futures!arts!practice!and!projects,!and!influenced!the!approach!and!

development!of!the!10%Minutes%to%Midnight!showcase!delivered!at!The!Block,!at!the!Queensland!

University!of!Technology,!later!in!2015.!!!

!

!


! 116!

It!is!important!to!note!that!the!evaluation!outputs!distributed!to!partners!in!2015!are!not!the!end!of!

the!READ!cycle!for!this!project.!Data!has!and!will!continue!to!emerge!about!the!project!beyond!the!

date!of!the!partner!closure!meetings!for!the!Balaklava!project.!Subsequent!to!the!dissemination!of!

the!evaluation!outputs!(like!the!Arts!SA!Acquittal!Report)!some!of!the!longer!term!and/or!broader!

implications!of!the!Balaklava!work!have!been!revealed,!and!these!have!a!bearing!on!how!the!project!

should!be!evaluated.!A!few!examples!are:!photomedia!artist!Jessie!Boylan!won!a!major!photography!

prize!for!artwork!generated!at!Balaklava,!the!projection!installation!10%Minutes%to%Midnight!was!

award!as!a!finalist!in!an!international!moving!image!competition,!and!audience!feedback!of!the!

artworks!continue!to!be!generated!at!following!exhibitions.!This!rich!data!will!continue!to!be!

collected!by!Alphaville!and!fed!into!the!summative!evaluation!for!the!Nuclear!Futures!program,!

however!it!does!highlight!how!timing!of!final!READ!activities!will!have!a!significant!influence!on!the!

range!of!data!able!to!be!included.!

5.4.4! Analysis#

Approach*

The!aim!of!the!READ!Analysis!strategy!is!to!promote!scholarship,!research!and!dissemination!of!

original!knowledge!arising!from!community!arts!in!areas!of!significance!about!the!practice!and/or!

emergent!creative!works.!This!READ!strategy!is!a!research*oriented!and!analytical!enquiry!approach!

for!learning,!driven!by!a!community!of!researchers,!to!provide!a!complimentary!critical!lens!for!

Reflection,!Evaluation!and!Documentation.!Analysis!focuses!on!the!wider!implications!arising!from!

CACD,!using!systematic!and!multi*step!processes!of!investigation!(documentation,!description,!

analysis,!discovery,!interpretation,!or!theorisation)!to!advance!knowledge!in!this!sector!of!

community!arts.!

For!Nuclear!Futures,!Analysis!involved!a!cohort!of!academics!working!with!artists!and!community!

members!to!develop!scholarly!publications,!reports!and!presentations.!Alphaville!refers!to!the!

collaborators!as!a!‘circle!of!advisers,!academics,!creative!artists!and!other!researchers’.52!As!well!as!

generating!original!knowledge,!the!approach!seeks!to!promote!broader!learning!via!the!wider!

dissemination!of!action!and!practice*led!research.!It!is!grounded!in!the!premise!that!facilitating!

collaboration!between!academics,!researchers!and!arts!practitioners!would!enrich!the!endeavours!

of!each!other!(National!Ideas!Summit!&!Australia!Council,!1990).!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

52!Sourced!from!Nuclear!Futures!documentation,!courtesy!of!Alphaville.!


! 117!

Driving!the!Analysis!strategy!was!Alphaville’s!interest!in!unlocking!the!academies+for!collaboration!

between!creative!artists!and!researchers!for!CACD.!The!notion!of!‘unlocking!the!academies’!first!

gained!traction!in!the!early!1990s!following!the!release!of!an!Australia!Council!discussion!paper!

entitled!Unlocking%the%Academies,!prepared!as!part!of!the!follow*on!from!the!1990!National!Ideas!

Summit!(National!Ideas!Summit!&!Australia!Council,!1990).!In!its!early!iteration,!attention!was!on!

reducing!the!cultural!and!professional!gaps!between!academics,!journalists!and!media!publishers,!

however!the!need!to!engage!and!collaborate!with!the!academies!is!also!relevant!for!other!sectors!

(Curthoys,!1991).!It!was!argued!that!too!often!academic!knowledge!is!not!accessible!for!the!wider!

public!or!non*specialist!audiences!due!to!a!lack!of!a!common!language,!preventing!knowledge!

exchange!and!learning!between!industry!practitioner*researchers!and!academics!(National!Ideas!

Summit!&!Australia!Council,!1990).!Based!on!observation!during!my!professional!experiences,!this!

remains!true!for!the!CACD!sector.!With!this!goal!in!mind,!Alphaville!was!well!placed!to!explore!how!

the!CACD!sector!might!progress!toward!‘unlocking!the!academies’!as!a!large!number!of!individuals!

connected!with!the!Nuclear!Futures!network!had!academic!roles,!interests!or!previous!research!

experience.!!


During!the!developmental!stage!of!Nuclear!Futures,!Creative!Producer!Paul!Brown!approached!

potential!academic!partners!to!research!the!unfolding!community!arts!activities!of!the!creative!

community!partnerships!initiative.!Academics!from!the!following!institutions,!representing!multiple!

disciplines,!partnered!with!the!Nuclear!Futures!program:!

•! Queensland!University!of!Technology!(QUT)53!

•! University!of!Western!Sydney!(UWS)!

•! University!of!New!South!Wales!(UNSW)!

•! Murdoch!University!

•! Hiroshima!Peace!Institute,!Hiroshima!University!

•! Manchester!University!

•! Winchester!University.!

For!the!Balaklava!project,!the!partnerships!with!QUT,!UNSW!and!UWS!have!been!most!relevant.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

53!QUT!has!been!involved!via!my!role!as!a!DCI!researcher!and!while!independent!to!the!Analysis!approach!described!in!this!

section,!is!still!contributing!to!the!Analysis!strategy!and!outcomes.!

!


! 118!

The!following!text!demonstrates!the!way!that!Alphaville!framed!the!academic!scholarship!in!relation!

to!the!community!arts!practice:!

The!Nuclear!Futures!project!brings!together!streams!of!theory!and!practice,!from!theatre!to!

activism,!to!wider!social!movements!and!politics,!to!memory,!loss,!survival,!history!of!

imperialism!and!displacement,!etc.!It!is!about!utopian!and!dystopian!futures,!capturing!

important!historical!developments!in!Australia,!United!Kingdom,!Japan!the!United!States!and!

Pacific!nations.!The!stories!of!atomic!survivors!are!too!often!neglected!and!ignored,!glossed!

over!and!obscured,!even!purposefully!recast!or!excluded.!!

Multiple,!dynamic!and!diverse!arts!practices!have!driven!the!project!so!far,!alongside!

scholarship!from!institutions!in!several!countries.!It!is!now!time!for!those!of!us!seeking!to!

make!sense!of!the!research!side!to!take!stock!and!ask:!what!are!the!theoretical!and!wider!

academic!implications!of!this!work?!(Courtesy!of!Alphaville!program!documentation)!

A!research!plan!for!the!Analysis!strategy!for!the!READ!model!was!devised!in!2014!by!Alphaville!and!

partnering!academics!from!University!of!Western!Sydney,!Professor!James!Arvanitakis!and!Dr!Ingrid!

Mathews.!A!staged!program!of!research!and!publication!was!designed.!The!first,!a!descriptive!phase,!

aimed!to!shed!light!on!the!arising!testimonials!and!emergent!register!of!voices,!with!delivery!of!a!

small*scale!seminar!and!publication!of!documentation.!The!second!phase,!scheduled!for!2016,!

planned!for!further!conceptual!work!and!dissemination!of!analysis!via!wider!conferences!and!

forums,!with!the!objective!to!publish!and!disseminate!a!second!compilation!of!material!generated!

on!Nuclear!Futures!and!related!topics!on!‘art!and!the!bomb’.!At!the!time!of!writing,!only!the!first!of!

these!two!phases!had!been!undertaken.!(RIPP:!Article!22,!p.!75!details!the!Research!Plan.)!

Alphaville!and!partners!were!interested!in!multi*disciplinary!research!areas!encompassing!CACD!as!

well!as!other!non*arts!disciplines!such!as!Science!and!Technology!Studies!(STS)!studies!and!Political!

History.!In!February!2014,!the!core!Nuclear!Futures!creative!team!and!participating!scholars!from!

UNSW,!UWS,!QUT!and!Murdoch!University!attended!a!visioning!workshop!to!canvas!potential!

research!topics.!The!ideas!generated!by!this!collective!process!identified!the!following!prospective!

themes:!Memory!and!narrative;!Arts*Science!interactions;!The!history!of!military!and!civil!nuclear!

programs;!Imperialism,!the!Cold!War,!and!other!key!historical!dimensions;!Wastes!and!

contaminated!sites;!Health!and!radiation!impacts;!The!role!of!the!arts!(and!of!community*based!

arts);!Art,!protest,!and!disruption;!Gift!theory!and!reciprocity;!Justice!and!the!arts;!Resilience!and!

survival;!Eco*criticism;!and!Future!Studies.!These!themes!were!driven!by!the!research!interests!of!

the!network!of!engaged!academics!and!the!creative!team,!and!as!is!evident,!were!not!related!to!the!

key!evaluation!questions!or!limited!to!areas!covered!by!the!program!objectives.!

!

!


! 119!

Results**

To!date,!the!Analysis!strand!of!READ!has!involved!academic!partners!and!collaborators!working!with!

creative!artists!(some!of!whom!are!also!academics)!and!communities!across!a!range!of!disciplines.!

The!first!phase!of!Analysis!has!resulted!in!the!following!scholarly!research!and!presentations:!

•! Conference!Presentation:!Arts!and!the!deep!Nuclear!Future:!a!prospectus!by!P.!Brown,!!

A.!Hudson,!E.!Barkley,!J.!Arvanitakis!at!the!Eco*Arts!Conference,!Wollongong!12*13!May,!

2013.!(RIPP:!Article!23,!p.77!includes!Abstract!and!Article!21,!p.74!shows!image.)!

•! Conference!Presentation:!Harnessing!evaluation!through!integration:!A!participatory!

model!for!Reflection,!Evaluation,!Analysis!and!Documentation!(the!‘READ’!model)!in!

Community*based!Arts!by!E.!Barkley!at!the!Australasian!Evaluation!Society!Conference,!

Darwin!10*12!September,!2014.!(RIPP:!Article!24,!p.78!includes!Abstract!and!slides.)!

•! Conference!Presentations:!Nuclear!Futures!by!P.!Brown!and!testimony!by!M.!Goodwin!at!

British!Nuclear!Test!Veteran!Association!(BNTVA)!Conference,!Manchester,!23*26!May!2014.!

•! Academic!Seminar:!Creative!Arts!and!Nuclear!Futures,!The!Glasshouse,!QUT!organised!by!

Alphaville!6*7!August!2015!(two*day!presentation!program!involving!20!participants).!

(RIPP:!Article!25,!p.96!includes!presentation!slides!for!my!presentation!on!READ.)!

!

Of!the!above,!of!particular!relevance!was!the!QUT!Seminar.!The!seminar!was!a!chance!to!‘take!stock!

and!ask:!what!are!the!theoretical,!academic!and!wider!social!implications!of!this!work?’!(taken!

from!Seminar!program/Alphaville!documentation).!Across!the!two!days!17!researchers!and!

practitioners!were!involved!in!the!program!of!presentations,!panel!discussions,!screenings,!

podcasts!and!live!streaming.!(RIPP:!Article!21,!p.74!for!images,!and!Article!34,!p.122!for!program.)!!

In!addition,!there!have!also!been!several!academic!events!and!forums!at!which!I!have!disseminated!

research!with!industry!peers!and!networks!about!Nuclear!Futures!and!the!READ!approach,!including:!

•! Evaluation!workshop!to!CSIRO!social!research!teams,!Brisbane,!December!2013.!

•! Sharing!of!research!at!the!masterclass!for!postgraduate!students!prior!to!the!Spectres%of%Evaluation%conference!in!Footscray,!February!2014.!

•! Dissemination!of!materials!to!Nuclear!Futures!stakeholders/!partners!(partner!summaries,!

online!publication,!sharing!of!DCI!research!outputs).!

Based!on!the!results!so!far!from!the!academic!led!Analysis,!it!is!anticipated!that!the!Analysis!outputs!

when!integrated!with!the!evidence!of!program!impact!generated!from!the!other!READ!strategies,!

will!greatly!contribute!to!an!increased!understanding!of!the!contribution!of!the!program,!and!also!

more!generally!of!the!CACD!sector.!This!is!however,!an!area!of!READ!that!is!still!being!developed!and!

necessarily!extends!beyond!the!DCI!research.!For!example,!we!are!still!exploring!how!best!to!

engineer!feedback!loops!over!the!medium!and!long*term!between!the!academic!research!produced!

and!the!communities!involved;!the!preferred!means!of!distribution;!and!issues!relating!to!co*

authorship!between!academic!researchers!and!participants!involved!in!the!art*making!and!research.!! !

!

!

!

!


! 120!

5.4.5! Documentation#

Approach*

Documentation!aimed!to!enhance!knowledge!and!understanding!through!the!organised!collection!of!

records!and!artefacts!arising!from!community!arts!practice!and!sourced!from!private!archive!

collections!of!community!participants.!For!the!READ!application,!Documentation!served!both!as!a!

creation!and!organisation!strategy.!This!was!consistent!with!prescriptions!advocated!by,!among!

others,!UNESCO!and!ICCROM!(2015).!Art*based,!administrative!and!archival!methods!were!used!to!

chronicle!the!story!of!the!community!arts,!and!embedded!community!and!artist!stories,!for!

immediate!and!long*term!purposes.!This!included!the!collection!of!printed!and!online!

documentation,!digital!and!photo!media,!oral!history,!archival!and!education!resources,!and!raw!and!

produced!creative!documentary*art!content.!The!documentation!captured!key!project!turning!

points,!emergent!themes/tensions,!stories!of!broad!public!relevance,!creative!works,!partnership!

exchanges!and!participant!experiences.!Creative!processes!such!as!curation,!production,!editing!and!

archiving!offered!an!interpretive!lens!for!understanding!the!artistic,!social,!economic,!political!and!

environmental!implications!of!the!work.!Multi*platform!and!dynamic!representations!of!the!arts!

practice!and!people!involved!resulted.!!(RIPP:!Article!26,!p.100!presents!a!photo!overview).!

Influential!to!the!approach!was!Alphaville’s!intent!to!create!opportunities!and!partnerships!that!

would!support!public!accessibility!to!the!documentation!and!long*term!storage!beyond!the!three*

year!Nuclear!Futures!program.!For!Balaklava,!one!example!of!this!is!the!partnership!developed!with!

State!Library!of!South!Australia!(SLSA)!where!SLSA!staff!mentored!community!members,!artists!and!

arts!workers!in!the!archiving!of!Nuclear!Veteran!related!materials!unearthed!during!creative!

research.!The!Library!initiated!a!new!collection!on!South!Australia’s!atomic!history,!so!that!archival!

materials!of!state!significance!and!oral!histories!generated!could!be!donated!to!the!institution!for!

long*term!public!access.!To!date,!two!nuclear!veteran!families!involved!in!the!Balaklava!project!have!

contributed!to!this!collection!and!it!is!anticipated!further!materials!will!be!donated!in!later!phases.!

Like!the!other!READ!strategies,!the!design!and!implementation!of!the!Documentation!strategy!was!

facilitated!as!a!collaborative!process,!with!Alphaville,!community!partners!and!Nuclear!Futures’!

artists!with!filmmakers!and!multi*media!artists.!Driving!the!strategy,!were!the!stakeholder!

needs/interests!identified!in!the!scoping!phase!as!well!as!goal*free!creative!development!using!

documentation!art!processes!led!by!artists.!The!resultant!approach!was!a!strength*based!process!for!

recording,!organising!and!reviewing!data!to!produce!multi*platform!documentation!and!archives.!

The!methods!used!are!discussed!below.!

!


! 121!

Methods*

For!community!arts,!it!is!both!the!processes!and!objects!created!that!convey!significant!messages!

about!the!arts!practice,!partnerships!and!cultural!development!outcomes!(Goldbard,!2006).!In!the!

first!instance,!process!documentation!was!conducted!to!chronicle!and!support!the!community!arts!

process!itself!and!this!involved!internal!documentation!(by!artists,!administrators!and!community!

members)!during!the!project!for!the!purpose!of:!capturing!stories!from!the!implementation!

experience;!as!art*making!in!and!of!itself;!and!for!continuous!improvement.!This!linked!with!the!

ongoing!project!monitoring!and!represents!the!‘how’!of!implementation!(Stauffer,!2015).!Secondly,!

attention!was!given!to!the!emergent!outputs,!artefacts!(such!as!the!creative!artworks!generated)!

and!the!outcomes!of!the!CACD!processes.!Focus!was!on!the!‘what’!of!the!art*making.!Lastly,!

reflective!methods,!inherent!in!the!creative!and!organisational!processes!used!during!the!processing!

phase!for!the!documentation!collected!(such!as!art*making,!publication,!and!production),!were!a!

means!for!exploring!the!‘why’!and!‘what!next’!aspects!of!the!community!arts!being!undertaken.!

Methods!included:!photography;!video!and!film;!audio!recording;!reporting;!publication;!script!

writing;!field!notes;!transcripts;!blogging;!infographics/!visual!summaries;!data!base!and!record!

keeping;!communications;!interviews;!media!and!documentation!scans;!oral!history;!archiving!and!

collection!development.!For!Nuclear!Futures’!Balaklava!Community!Arts!Project!I!surmised!that!

there!were!four!principal!types!of!documentation!data!sets!resulting:54!

•! Creative!documentation:!Art*based!and!storytelling!content!informed!by!documentary!arts!

traditions!that!culminated!in!vast!quantities!of!photomedia!and!video!content,!a!Nuclear!

Futures!promotional!video,!photo!essays,!a!photo*story!book,!video!interviews,!website!

material/blogs!and!planned!applications!for!film.!Creative!documentation!was!carried!out!by!

visiting!and!local!artists,!including!photomedia!artists!and!the!project’s!playwright,!and!

community!participants.!Managerial!and!administrative!staff,!including!myself!and!the!local!

coordinator!for!the!Balaklava!project,!also!collected!and!produced!creative!materials.!

Such!art*based!methods!used!visual!and!audio!stimuli!to!‘understand!and!communicate!

about!the!world’!(Patton,!1981,!p.!217).!(RIPP:!Article!27,!p.101!has!examples.)+

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

54!While!delineated!in!the!above!documentation!strands,!content!from!each!was!often!related!to!each!other!and!the!larger!

suite!of!READ!outputs,!and!organised/!edited/!archived!in!multiple!ways!(UNESCO!and!ICCROM,!2015).!This!is!also!true!for!

the!application!of!some!documentation!methods!for!the!other!READ!strategies.!

!

!


! 122!

•! Administrative!and!managerial!documentation.!This!involved!the!ongoing!process!of!

scanning!and!extracting!materials!from!Alphaville’s!program!development!and!production!

management!(including!operation!systems).!Sourced!were!predominantly!written!content!

such!as!guidelines,!delivery!summaries,!reports,!promotional!records!(including!branding),!

grant!submissions,!partner!agreements,!registration!records!(participants,!audiences),!media!

records,!communications,!meeting!notes!and!so!on.!My!dual!role!as!program!manager!

offered!an!excellent!vantage!point!to!access!and!intercept!relevant!content.!!

(RIPP:!Article!28,!p.106!includes!4!examples!and!Article!30,!p.111!has!a!media!report.)!

•! Nuclear!Futures!archive!of!creative!works!and!artefacts.!This!included!the!collection,!

storage,!and!digital!register!of!artworks!produced.!Data!was!sourced!from:!creative!team!

biographies!(artists,!community!partners,!participants,!art*workers);!art!form!specifications!

(interpretative!information);!artist!statements;!background!information;!descriptions!or!

representation!of!artworks/!processes;!publication!or!exhibition!records;!community!

information!(geographic,!demographic,!historical);!video!and!photography!of!arts!practice;!

thematic!coding;!and!cataloguing!of!the!content!files!and!copies!for!durable,!safe!

storage.!(RIPP:!Article!29,!p.109!includes!documentation!about!creative!works/artists.)!

•! Community!archives!and!resources!(external).!Inspired!by!the!intent!to!make!accessible!

significant!findings!unearthed!or!generated!during!the!project,!including!historical!and!

education!materials,!this!type!of!documentation!refers!to!the!oral!history!and!collection!

development!of!information/!artefacts!of!public!value.!Archiving!aimed!to!preserve!

documentation!held!by!those!who!have!witnessed!atomic!tests!and!those!who!have!

campaigned!for!recognition!and!reconciliation.!These!were!resources!distinct!from!the!

creative!works,!and!official!records!in!the!public!domain,!and!included!material!held!in!

communities!that!provided!unique!personal!perspectives!in!the!form!of!annotated!reports,!

medical!case!notes,!personal!maps,!photographs,!artefacts!and!documents.!Community!

archiving!was!a!collaborative!activity!by!the!creative!team!(including!myself),!community!

members!and!in!partnership!with!institutional!partners.!

•! Academic!research!and!documentation.!This!included!research!papers,!presentation!

documentation!and!materials!generated!through!linked!academic!scholarship!(see!Section!

5.4.4)!by!academic,!community!and!practitioner!researchers!on!the!CACD!activities!and!

themes,!exploring!the!wider!implications!of!the!community!arts!work!undertaken.!

As!summarised!above,!the!data!collection!and!analysis!methods!involved!creative!development!and!

research!(for!art’s!sake)!as!well!as!the!recording!of!the!CACD!processes,!and!cataloguing!and!archive!

development.!Analysis!of!documentation!data!was!done!via!a!shared!processes!of!curation!that!used!

!

!


! 123!

administrative,!reflective!and!creative!methods!(such!as!editing,!publication!and!review!processes)!

to!produce!online!content,!blogs,!a!photo!book,!video!bites!and!case!studies.!The!interpretative!lens!

of!the!documentation!analysis!is!akin!to!Paul!Carter’s!(2010)!notion!of!‘reflective!archiving’*!where!

souvenirs!are!produced!from!projects!based!on!reflection!and!from!going!over!documentation!in!

order!to!re*evaluate,!to!discover!new!meanings!within!or!between!projects!and!to!reveal!lengthy!

exposure!of!the!arts!practice.!By!reviewing!past!documents,!photos,!and!artefacts!in!reflective!and!

creative!ways,!meaning!and!patterns!are!identified!(Carter,!2010).!Similarly,!Boud!describes!these!

documentary!processes!as!‘a!lens!for!learning’!(Boud,!2001,!p.9)!able!to!generate!grounded!analysis!

and!interpretative!knowledge.!(RIPP:!Article!27,!p.101*5!has!photos!related!to!the!archiving).!!

Results*

The!products!of!the!Documentation!strategy!from!the!Balaklava!project!included!raw!(unprocessed)!

content!and!produced!(processed)!outputs,!such!as!historical,!creative,!educational,!political,!

technical!and!administrative!data,!artefacts!and!resources.!Specifically!the!Documentation!outputs!

were:!video!and!audio!recordings;!extensive!photography!collections!and!artefacts;!data!and!log!

files;!administrative!data!base!contents;!an!online!cultural!repository!for!intergenerational!learning!

at!the!SLSA;!an!oral!history!collection,!testimony!and!artefacts!for!social!justice!campaigns;!original!

creative!works;!education!resources;!promotional!and!media!records;!and!bulk!raw!creative!content!

valued!as!a!future!resource!for!participating!communities!and!artists!to!own!and!use!in!beyond!the!

Balaklava!project.!Materials!were!disseminated!via!stakeholder!networks,!online!and!social!media!

channels,!traditional/mainstream!media!and!SLSA’s!public!archive!collection.!

The!involvement!of!photographers!and!film!makers!in!the!Balaklava!creative!team!meant!that!the!

use!of!video!and!photography!was!an!immediate!and!familiar!practice!to!be!used!for!understanding,!

exploring,!analysing!and!communicating!emergent!themes!arising!from!the!creative!arts!practice.!

Utilising!the!readily!accessible!expertise!from!the!creative!team,!combined!with!external!factors!

(such!as!the!affordability!and!access!to!good!quality!audio!and!video!equipment,!software!and!

online!media!platforms!even!from!the!regional!location!of!Balaklava),!enhanced!the!scope!and!

sophistication!of!the!documentation!collected!and!produced.!Creative!documentation!in!particular,!

helped!relate!READ!as!an!evaluation!framework!with!the!art*making!and!creative!development!

activities!of!the!project,!which!was!engaging!for!artists!and!community!members.!The!Nuclear!

Futures’!Balaklava!artists!recognised!the!rich!details!able!to!be!captured!through!documentary!art!

practice!and!because!the!ownership!of!material!remained!with!them,!co*owned!in!some!cases!with!

partners!and!Alphaville,!documentation!creative!content!had!the!potential!to!be!used!for!future!art*

making!outside!of!the!project!(in!accordance!to!with!the!authorship!and!consent!protocols).!!

!


! 124!

The!end*products!were!of!short!and!long*term!relevance!for!the!project!and!partners.!Short!term!

needs!were!met!through!the!Documentation!deliverables,!with!both!documentary!(verbatim)!and!

story*based!representations!of!the!project’s!participants,!activities!and!impacts.!Documentation!as!a!

storytelling!device!brought!the!community!experiences!to!life,!representing!the!many!views!and!

voices!involved!in!the!project,!and!in!many!cases!provided!valued!gifts!for!community!partners.!

Community!partners!retained!access!and/or!copies!to!all!raw!footage!and!the!produced!outputs,!

such!as!the!published!photo*story!book!gifted!to!Avon!Hudson!and!family!as!a!project!keep*sake!

(RIPP:!Article!33,!p.121!has!photographs!of!the!book!to!demonstrate).!!

The!Documentation!strategy!resulted!in!the!emergent!language,!and!vibrancies!of!the!community!

arts!practice!to!begin!to!be!reflected!and!integrated!through!‘the!discursive,!the!pictorial,!the!

persuasive,!and!the!instrumental’,!and!challenged!the!privilege!traditionally!given!to!written!

language!in!the!research!and!evaluation!context!(Doloughan,!2002,!p.62).!Creative!documentation!in!

particular!was!relevant!to!the!community!partnerships!context!and!accommodated!the!different!

cultural!preferences!and!needs!for!some!stakeholders!as!identified!in!scoping.!Utilising!arts*based!

methods!was!valuable!as!a!means!to!represent!multiple!stakeholder!perspectives.!In!other!studies,!

researchers!argue!such!methods!‘evokes!different!ways!of!knowing!and!understanding!the!values!of!

the!program’!(Simons!&!McCormack,!2007,!p.!292;!Patton,!1981).!Art*based!methods!also!worked!

well!for!documenting!the!community!presentation!of!the!community!arts,!particularly!for!preserving!

ephemeral!aspects!of!the!showcasing!such!as!the!speakers!corner!for!the!10%Minutes%to%Midnight%

showcase!and!aspects/encounters!not!able!to!be!articulated!in!words!or!measured!by!metrics.!!

Lastly,!another!area!of!interest!arising!from!the!application!was!in!respect!to!the!embedded!long!

term!and!legacy!objectives!to!make!the!story!go!far.!The!documentary!art*making!and!storytelling!

methods!from!the!site!of!practice!produced!a!multi*platform!collection!of!nationally!significant!

documentation!as!part!of!the!creative!development!of!the!arts!practice,!and!the!READ!processes,!

including!rich!verbatim!narratives!and!documentary!textual!materials!that!were!engaging!for!READ!

collaborators!and!audiences.!In!the!medium!term!across!the!ongoing!arts!practice!of!Alphaville,!the!

creative!documentation!such!as!video!interviews,!enable!the!preservation!of!ongoing!creative!

conversations!within!the!broader!Nuclear!Futures!program!which!will!be!of!use!across!the!program’s!

three!years,!and!beyond.!Further,!it!is!envisaged!that!the!long*term!storage!and!public!access!to!

atomic!survivor!community!materials!donated!to!the!SLSA!collection!will!be!a!legacy!resource!for!

future!generations.!In!doing!so,!the!relevance!and!accessibility!of!project!artefacts!has!been!

extended!beyond!primary!participants!and!audiences!to!the!wider!public,!researchers,!historians!

and!artists!now!and!in!the!future.!(RIPP:!Article!6,!p.14!shows!an!early!plan.)!

!

!


! 125!

!

5.5! READ#outputs,#sharing#and#use#

Integration*

While!the!READ!strategies!have!been!presented!here!as!discrete!methodologies,!in!reality!many!

methods!and!data!collection!processes!were!relevant!for!multiple!strategies.!As!is!outlined!through!

this!chapter,!mixed!data!was!generated!across!the!four!READ!strategies!of!Reflection,!Evaluation,!

Analysis!and!Documentation.!Overall!the!sheer!volume!of!data!meant!as!an!evaluator!and!manager,!

I!considered!this!outcome!as!a!data*utopia.!Processes!to!integrate!and!synthesise!data!were!used!in!

real*time!across!the!four!READ!strategies!(denoted!by!the!grey!arrow!in!previous!Figure!5.1),!and!at!

the!processing/!production!stage!when!findings!and!products!were!generated!to!meet!the!partner*

identified!end*uses.!The!integration!meant!reviewing!different!types!of!data!(quantitative!and!

qualitative)!together,!to!produce!findings!or!published!outputs!of!the!READ!process.!The!following!

discussion!spanning!the!integration,!utilisation!and!dissemination!correlates!to!the!fourth!research!

phase!undertaken!for!the!READ!application!(READ!outputs,!sharing!and!use).!

Spontaneous!and!planned!sessions!were!held!to!review!the!READ!data!for!both!formative!and!

summative!rounds!of!integration.!I!used!facilitation!techniques!to!encouraged!participation!in!the!

review!and!integration!of!the!READ!material,!for!example!coordinated!shared!collective!review!

sessions!(involving!presentation!and!discussion!of!READ!data!at!group!meetings!with!artists,!

administrators!and!community!partner!representatives)!and!individual!consultations.!The!

integration!process!was!semi*structured,!and!was!intended!to!facilitate!triangulation!of!the!data,!

where!data!from!different!sources!were!reviewed!to!verify!validity,!themes!and!consistency!or!

tensions!in!the!findings.!Integration!was!about!synthesising!data!to!create!sound!and!credible!

findings!of!use!and!meaning!to!partners.!In!the!process!of!integration,!I!reduced!the!data!into!

manageable!chunks!so!that!themes!and!findings!became!clear.!I!used!visual!approaches!including!

tables,!graphs!and!graphics!for!quantitative!responses.!!

! !

!


! 126!

The!following!analysis!techniques!were!used:!

•! Thematic!analysis:!examining!information!for!synthesis!potential!and!identifying!points!of!

divergence/!contrast;!includes!textual!analysis!and!comparative!analysis,!and!feedback!of!

analysis!via!peer!review.!

•! Reflective!analysis:!process!of!reviewing,!interrogating,!re*evaluating!and!making!tacit!

knowledge!explicit.!

•! Evaluative!reasoning:!framed!for!KEQs!and!using!rubrics/!performance!criteria.!

•! Quantitative!analysis:!generating!statistics!and/or!numerical*based!findings!from!project!

information.!

•! Creative!and!practice*led!processes:!selection,!editing!and!storytelling!through!

documentation,!creative!development!etc.!

During!these!processes!there!were!a!number!of!considerations!that!I!took!into!account!in!the!

planning,!as!READ!facilitator.!These!included:!determining!who!needed!to!be!involved!in!the!

collation!process!to!ensure!adequate!representation;!confidentiality!issues!during!analysis;�defining!

the!audience!for!READ!outputs!(to!suit!permissions!and!preferences);!and!ongoing!monitoring!of!the!

impact!on!organisations!and!individuals!involved!in!the!process.!!

READ*output*and*outcomes*

For!the!Balaklava!Community!Arts!Project,!a!collection!of!digital!and!hard*copy!outputs!resulted!

from!using!the!READ!methodology,!including:!

•! Raw!data:!databases!of!information,!statistics,!film!content,!photos,!audio!recordings,!oral!

history,!archival!resources!and!so!on.!

•! Management!resources:!an!evaluation!plan!for!Nuclear!Futures!program,!templates/pro!

formats!for!evaluation!instruments!and!reflective!practice,!evaluation!findings,!residency!

report,!media!reports,!partner!summaries!and!communications,!financial!reports!and!

visual!summaries!such!as!infographics.!(RIPP:!Article!35,!p.126!has!residency!report.)!

•! Evaluation!and!project!reports:!such!as!the!Acquittal!Report!delivered!to!Arts!SA!and!

modified!reports!for!various!partners.!(RIPP:!Article!36,!p.140!has!Acquittal!Report.)!

•! Academic!Research:!research,!conference!presentations,!Research!Plan!for!publications.!

•! Creative!content/!artwork:!significant!photomedia!repository,!website!archive,!photo!story!

book,!promotional!materials.!!

•! Spin!offs/pending/future!outputs:!creative!works!(such!as!documentary!film!and!webisodes),!

program/promotion/marketing!assets.!(RIPP:!Article!31,!p.119!has!promotional!video.)!

The!access!and!shared!copyright!(authorship)!vary!across!partners!and!outputs.!Demonstrations!

of!the!outputs!are!included!in!the!READ%in%Practice%portfolio!(RIPP:!Section!F,!p.118).!

!

!

!


! 127!

In!the!process!of!the!READ!collaboration!there!were!also!various!outcomes.!For!Alphaville!(the!host!

organisation)!READ!resulted!in!the!generation!of!extremely!valuable!data,!for!understanding!and!

promoting!its!current!work!and!for!scoping!for!future!artwork!based!on!the!content!created.!Nuclear!

Futures!personnel!(artists,!art!workers,!administrators)!and!the!local!freelance!artist!participating!

directly!in!designing!and!implementing!READ!received!training!and!skills!development!(of!immediate!

and!future!benefit!for!their!practice).!Other!benefits!were!the!demonstrable!cultural!outcomes!

evidenced!for!the!Alphaville!organisation!from!the!reflective!practice,!such!as!the!development!of!a!

culture!of!learning!at!the!project,!program!and!organisational!level,!which!I!evaluated!to!be!an!

indication!of!capacity!building!for!the!community!arts!organisation!as!a!whole.!There!were!also!

many!material!benefits!of!the!READ!process!for!contributors!and!Alphaville,!such!as!templates,!plans!

and!resources!readily!applicable!for!other!Nuclear!Futures!READ!applications.!Relevant!for!funding!

partners,!Australia!Council!for!the!Arts!and!Arts!SA,!READ!produced!information!for!inclusion!in!

regular!reporting!(included!evaluation!findings!as!they!emerged)!and!acquittals.!

For!the!community!partners,!one!of!the!original!interests!was!to!‘make!the!story!go!far’!(Alphaville,!

2015).!For!this!objective,!READ!had!myriad!outcomes,!in!addition!to!the!community!arts!outcomes,!

and!these!will!continue!into!the!future.!For!the!Hudson!family!in!particular,!guidance!and!formalised!

arrangements!resulted!in!their!establishing!a!long*term!relationship!with!SLSA!in!relation!to!the!use!

of!their!significant!private!archive.!Also!large!volumes!of!documentation!(such!as!photos,!videos,!

transcripts!and!so!on)!created!valuable!resources!that!were!gifted!back!to!community!participants!

through!the!READ!process.!In!other!instances,!such!as!for!community!participants!involved!in!the!

Field%of%View!photography!exhibition,!the!digital!documentation!meant!their!works!were!

published!and!disseminated!to!wider!audiences!via!an!online!gallery!(RIPP:!Article!32,!p.120).!

Interestingly!through!READ!processes,!particularly!from!the!Academic!strategy,!the!mixed!cohort!

arising!from!the!creative!community!partnership!initiative!(involving!community!members,!artists,!

researchers,!art!workers!and!activists)!formed!a!community!of!practice.!The!benefits!of!communities!

of!practice,!within!which!knowledge!construction!is!made!explicit,!have!been!enunciated!by!several!

analysts!(Cox,!2005;!Hildreth!&!Kimble,!2004;!Wenger,!McDermott,!&!Snyder,!2002;!Lave!&!Wenger,!

1991).!For!Nuclear!Futures,!the!community!of!practice!evolved!around!the!common!interest!in!the!

domain!of!‘art!and!the!bomb’,!created!with!the!goal!of!making!explicit!the!knowledge!generated!

through!creative!practice,!and!publishing!original!knowledge!as!research.!This!was!clearly!

demonstrated!when!contributors!from!Nuclear!Futures,!and!beyond,!came!together!at!the!two*

day!academic!seminar!at!QUT!in!August!2015!(RIPP:!Article!34,!p.!122).!Further!discussion!on!the!

outputs!and!outcomes!of!READ!are!included!in!the!reflective!practice!chapter!next,!in!Chapter!6.!!

!

!


! 128!

Utilisation*and*dissemination*

READ!data!and!outputs!were!used!as!a!lens!to!help!artists,!administrators!(including!myself)!and!

community!participants!to!understand!and!review!the!community!arts,!which!Schon!would!refer!to!

as,!in*!and!on*!action!(see!Schon,!1983,!and!Chapter!2!for!elaboration!of!this!concept).!Scriven!(2012)!

describes!evaluation!in!terms!of!its!purpose!and!timing,!and!how!these!factors!influence!the!scope!

for!evaluation!utilisation.!For!READ,!the!evaluation!data!produced!was!generated!and!used!

throughout!the!project!for!the!following!purposes:!formative!(during!the!early!stages!of!

implementation);!developmental!(ongoing!or!for!uncertain!and/or!innovation*oriented!contexts);!

and!summative!(once!activity!is!completed).!Thus,!the!utilisation!of!the!data!collected!was!of!

immediate,!short*term,!intermediate!and!long*term!benefit.!!Figure!5.5!summarises!the!timeline!of!

project!and!READ!activities!(generalised)!and!the!formative!and!summative!READ!outputs!that!were!

generated!from!this!ongoing!utilisation!(refer!to!‘Key!for!Diagram’!box,!Figure!5.5,!for!explanation).!

As!illustrated!in!Figure!5.5,!the!arrows!illustrate!cycles!of!utilisation!arising!from!one!round!of!READ!

activity!and!fed!directly!into!the!planning!or!delivery!of!the!project.!For!example,!data!from!

reflective!practice!and!the!evaluation!instruments!used!during!the!March!‘Pop*up’!residency!was!

assimilated!and!disseminated!immediately!for!partner!updates,!program!management!purposes,!

and!to!inform!creative!development!and!continuous!improvement!for!the!subsequent!residencies.!

The!internal!cycles!of!utilising!READ!generate!data!also!extended!beyond!the!Balaklava!project,!to!be!

fed!into!the!broader!program!management!of!the!Nuclear!Futures!program.!Most!significantly!was!

the!value!created!by!the!READ!outputs!and!findings!for!the!planning!of!subsequent!Nuclear!Futures!

exhibitions/projects,!such!as!the!July*August!10%Minutes%to%Midnight%showcase!season!at!The!Block,%

QUT!and!the!later!Screengrab7!exhibition!of!10%Minutes%to%Midnight!at!the!Pinnacles!Gallery,!

Townsville!in!December!2015!–!February!2016.!

In!addition!to!the!cycles!of!utilisation!internally!for!the!program,!were!cycles!of!reporting!back!to!

communities!and!partners!(not!illustrated!in!Figure!5.5).!Presentation!and!reporting!formats!were!

designed!to!meet!stakeholder!needs!and!interests.!In!addition,!READ!generated!materials!were!also!

disseminated!to!industry,!academic!and!peer!forums!via!seminars,!presentations,!mainstream!

media,!online!platforms,!and!community!networks.!As!was!indicated!from!the!scoping!consultation,!

partner!interests!in!the!utilisation!of!READ!data!and!products!was!divergent.!While!the!examples!of!

utilisation!and!dissemination!resulting!from!the!READ!application!are!too!numerous!to!cover!in!full,!

below!are!a!number!of!examples.!!

!


! 129!

Figure!5.5:!!READ!timeline!of!activities!and!outputs!for!Balaklava!Community!Arts!Project!!

Community Photography Exhibition

Pre-project Planning and partnership development

Public showcase/events

2013 Dec

2014 Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

2015 Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

READ planning (Balaklava project)

Exhibition

Residency: Photography workshops

Residency: public events, creative

Fieldwork: RAD Tour

Residency: planning & partnerships

Reflection, Evaluation, Documentation

Academic Seminar, QUT

Nuclear Futures showcase/events, QUT

Reflection, Evaluation, Analysis Documentation

READ instruments developed

READ model devised 2012 2013

Reflection, Evaluation, Documentation

Reflection, Documentation


Reflection, Evaluation, Documentation Curation of community exhibition

Residency: Creative development workshop Reflection, Documentation

Reflection, Documentation, Analysis

Creative development Reflection, Analysis, Documentation


Reflection, Evaluation, Analysis, Documentation

Reflection, Evaluation, Analysis, Documentation Residency: Project closure meetings

READ instruments developed for showcase

Summative READ integration and outputs

Formative READ Outputs Draft Evaluation Plan Evaluation instruments Reflections Creative documentation Oral history READ data Partner updates Monthly reports Media/ blogs/ website Presentations (industry/academic) Research Evaluation Findings AES conference presentation



Future!utilisation!!

Cycles of utilisation

Planning workshop (delivery partners)

Formative READ integration (ongoing)

Summative READ Outputs Refined Evaluation Plan Project Acquittal Partner reports/ summaries Evaluation Findings Creative documentation Creative works Hardcopy photo-story book Research Media/ blogs/ websites Presentations (industry/academic)

Fieldwork: BNTVA, UK

Key for Diagram Abbreviations: READ (denotes the READ model and activities, integrating reflection, evaluation, analysis and documentation); RAD Tour (Radioactive Exposure Tour, organised by Friends of the Earth); BNTVA (British Nuclear Test Veteran Association); QUT (Queensland University of Technology). Explanation: On the left is the project timeline. Marked along the timeline are the community arts activities (black) and READ activities (orange). The data generated from READ activities usually had immediate uses, as well as a cycle of utilisation that fed into the next creative residency or activity (see arrows). During the project data also had long-term cycles of utilisation (not indicated in diagram). On the right side, are examples of the outputs produced in the formative and summative phases of the project’s READ activities.


! 130!

For!some!artists!and!community!partners,!the!material!was!used!for!social!justice!and!political!

campaigning,!for!others,!READ!materials!added!to!their!personal!archives.!For!example,!artist!Jessie!

Boylan!(of!the!Atomic!Photographers!Guild)!will!continue!to!exhibit!and!further!develop!the!creative!

documentation!in!line!with!her!anti*nuclear!activism,!and!present!the!material!in!the!future!with!

other!peace*related!and!political!organisations.!Likewise,!community!member!Avon!Hudson!has!

used!the!documentation!about!the!creative!works!as!a!story*telling!devise,!a!tool!for!communicating!

on!the!broader!issues!of!atomic!testing!in!his!networking!and!advocacy!work.!For!the!local!

government!partner,!Wakefield!Regional!Council,!evaluation!and!documentation!were!used!to!fulfil!

reporting!and!strategy!development!purposes.!Information!was!reported!back!to!the!Council!and!

staff,!and!filed!as!a!record!of!local!CACD!activities.!Venue!partners!in!Balaklava!utilised!

documentation!and!information!for!venue!promotions!and!as!a!means!to!gain!insight!into!how!well!

the!venues!worked!as!a!Fringe!Festival!venues!(in!terms!of!accessibility,!capacity,!functionality,!

audience!experience),!given!they!were!previously!unregistered!as!Fringe!venues.!For!funding!bodies,!

the!READ!outputs!were!used!for!accountability!purposes!and!for!final!project!acquittal.!

Alphaville!and!Nuclear!Futures!personnel!used!the!READ!outputs!for!continuous!improvement!and!

knowledge!generation!to!advance!their!community!arts!practice.!As!program!manager,!I!have!

utilised!the!data!and!outputs!of!the!READ!process!in!wide*ranging!and!ongoing!ways:!as!reference!

material,!for!communications!and!promotion,!and!for!managerial!decisions.!One!specific!example!

was!the!high!utility!value!of!the!READ!content!in!the!planning!and!curation!of!a!subsequent!

showcase!of!the!artwork!at!The!Block,!QUT.!In!this!case,!the!lessons!from!the!Balaklava!project!

(produced!through!READ)!strongly!informed!the!creative!team’s!approach!to!the!development!of!a!

second!projection!installation!Ngurini%(Searching),!as!well!as!underscored!decisions!on!the!

presentation,!production,!management,!and!resourcing.!Another!demonstration!of!utility!is!that!

Alphaville,!and!several!artists!independently,!have!used!documentation!and!evaluation!data!in!new!

funding!applications!for!spin!off!projects,!and!in!the!promotion!of!their!arts!practices!online.!!

Lastly,!READ!material!was!disseminated!in!the!public!domain!as!long*term!artefacts!for!broader!

learning!and!future!utilisation.!The!first!example!is!the!SLSA’s!collection!development!to!ensure!that!

nationally!significant!material!unearthed/generated!through!the!project!remains!publicly!accessible!

for!future!use!by!researchers!and!artists.!A!second!example!is!demonstrated!by!the!planned!book!

publication!based!on!the!outcomes!of!the!community!of!practice!(established!through!the!Analysis!

strategy!of!READ),!another!means!of!creating!longevity!and!relevance!beyond!the!timeframe!and!

bounds!of!the!original!arts!practice,!and!the!evaluation!practice.!


! 131!

5.6! Conclusion#

Characterised!by!a!diverse!program!of!creative!arts,!involving!multiple!community!and!institutional!

partnerships,!the!Balaklava!Community!Arts!Project!provided!a!dynamic!and!fascinating!test!ground!

to!explore!the!functionality!and!nuances!of!the!READ!approach!in!action,!within!the!community!arts!

context.!To!examine!the!application,!I!adopted!a!systems!approach!to!document!and!profile!the!six!

core!elements!of!the!READ!concept:!the!supporting!structures;!Reflection;!Evaluation;!Analysis;!

Documentation;!and!integration,!utilisation!and!dissemination.!My!role!as!researcher!was!to!lead!

the!tailored!design!and!implementation!of!the!READ!methodology!for!the!Balaklava!project!as!a!

collaborative!process!with!participating!partners!and!stakeholders.!The!methodology!developed!and!

applied,!as!described!in!this!chapter,!reflects!the!story!of!the!Nuclear!Futures!(Alphaville)!site!of!

practice!and!a!diverse!cohort!of!contributors!and!participants.!Arising,!is!an!extensive!body!of!work!

that!has!formulated!and!substantiated!many!of!the!technical!and!philosophical!aspects!of!the!READ!

approach.!

As!demonstrated!through!the!Balaklava!application,!the!READ!methodology!employed!a!series!of!

known!learning!and!appraisal!strategies!relevant!to!community!arts!and!management,!and!these!

utilised!an!innovative!suite!of!qualitative,!quantitative!and!performative!methods.!READ’s!integrated!

framework!supported!wide*ranging!evaluative!action!by!artists!and!community!participants,!and!

harnessed!the!strengths!and!resources!of!the!art*making,!management!structures!and!

partners/organisations!embedded!in!the!community!arts!practice.!Participants!and!artists!engaged!in!

reflective!journaling,!video!diaries,!oral!history,!interviews,!art*making,!creative!documentation,!

surveys,!media,!vox!pops,!questionnaires,!a!community!of!practice!and!research.!A!number!of!

original!and!constructive!processes!for!partnership!development,!creative!development,!engagement!

and!organisational!learning!were!actioned,!and!a!diverse!portfolio!of!summative!materials!(both!

evaluative!and!non*evaluative)!resulted,!tailored!to!suit!the!multiple!needs!of!the!partnerships.!!

The!theoretical!and!practical!demonstration!of!the!READ!approach!provided!in!this!chapter!lays!the!

foundation!for!the!critical!analysis!of!the!READ!model!of!practice!as!it!emerged!and!was!refined,!and!

this!is!the!central!concern!of!Chapters!6!and!7.!As!detailed!in!the!following!chapters,!I!employed!

forensic!reflective!practice!as!the!principal!research!strategy!to!review!READ’s!performance!as!a!

framework!for!learning!and!appraisal.!The!aim!of!the!reflective!analysis!was!to!draw!out!the!tacit!

learnings!arising!from!the!evaluation!practice!and!to!inform!the!further!development!and!refinement!

of!the!READ!model,!in!order!to!establish!a!consolidated!READ!product!for!future!use.!The!findings!of!

this!critical!review!of!the!activities!outlined!in!this!chapter,!are!included!in!Chapter!6.!

!


! 132!

!

! !

Blank!page!


! 133!

6! Findings##

Chapter!5!detailed!the!collective!design!and!implementation!for!the!industry!trial!of!the!integrated!

READ!evaluation!approach!for!the!Balaklava!Community!Arts!Project!in!South!Australia,!a!process!

which!I!led!as!researcher!for!the!Doctor!of!Creative!Industries!(DCI).!Developing!an!operational!

model!of!practice!for!the!READ!concept!involved!an!evaluation!needs!assessment!process,!ongoing!

consultation!with!stakeholders,!and!the!integration!of!data!collected!via!the!methodological!design!

of!READ’s!four!key!learning!and!appraisal!strategies!–!Reflection,!Evaluation,!Analysis!and!

Documentation!(READ).!Also!documented!in!Chapter!5!were!the!wide!range!of!dissemination!and!

utilisation!outcomes!linked!to!the!outputs!generated!from!the!READ!methodology!devised.!At!the!

site!of!practice,!the!international!creative!communities!partnership!initiative!‘Nuclear!Futures’!

(funded!by!the!Australia!Council!for!the!Arts!and!Arts!South!Australia),!the!integrated!READ!learning!

and!appraisal!processes!generated!many!immediate!and!medium*term!benefits.!

Concurrent!with!delivering!the!READ!application,!I!conducted!an!ongoing!process!of!forensic!

reflective!practice,!focused!on!my!evaluation!practice,!as!well!as!on!the!development!and!

performance!of!the!READ!model.!This!enquiry!was!driven!by!two!overarching!research!questions:!

a)! How!does!the!READ!model!contribute!to!program!learning!and!appraisal!for!creative!

community!partnerships!within!community!arts,!based!on!the!Nuclear!Futures!experience?!!

b)! What!innovation!does!READ!offer!evaluation!practice!and!program!evaluation?!

Chapter!6!establishes!the!research!findings!based!on!the!critical!enquiry!undertaken!during!the!DCI,!

via!forensic!reflective!practice,!and!is!the!basis!for!the!summative!conclusions!in!Chapter!7.!

This!chapter!is!written!from!my!perspective!as!evaluation!practitioner,!and!offers!insight!into!the!

collective!experience!at!the!site!of!practice!evidenced!while!trialling!the!READ!approach.!I!reflect!on!

the!READ!application!and!the!complex!interactions!between!the!stakeholders!and!the!READ!system,!

and!the!outcomes!generated!through!the!creative!and!participatory!processes!of!the!evaluation!

practice.!My!reflective!analysis!covers!the!major!themes!and!findings!that!arose,!which!I!then!used!

to!advance!the!READ!model!development.!Meta*evaluation!techniques!were!employed!to!analyse!

the!performance!of!the!READ!model!in!action,!and!to!identify!the!strengths,!constraints!and!

opportunities!of!the!READ!approach.!In!particular,!I!outline!the!relevance!of!positioning!the!READ!

facilitation!internally,!explore!the!versatility!of!the!model!as!a!modular!or!integrated!system,!and!

examine!several!of!the!READ!methods!that!demonstrated!strong!results,!such!as!collective!reflection!

and!creative!documentation.!The!chapter!culminates!in!the!presentation!of!the!developed!READ!

product,!which!includes!an!infographic!and!a!list!of!key!considerations!for!future!evaluation!practice.!


! 134!

6.1.1! Analysis#of#the#READ#application#

This!chapter!is!divided!into!two!parts.!First!is!the!critical!analysis!of!the!collaborative!design!and!

implementation!of!the!READ!methodology!for!the!Balaklava!Community!Arts!Project!(Section!6.2).!It!

details!the!challenges!and!successes!of!the!evaluation!practice!and!sheds!light!on!the!complexities!

associated!with!actioning!the!READ!approach.!The!analysis!identifies!what!aspects!of!READ!

performed!well!in!the!Balaklava!context,!details!examples!of!necessary!modifications!for!improving!

the!system’s!function,!and!highlights!some!of!the!unforeseen!and!surprising!consequences!of!the!

application.!The!purpose!of!the!analysis!is!to!produce!findings!to!inform!the!refinement!of!the!READ!

approach!so!that!a!general!model!of!practice!can!be!developed.!In!making!explicit!the!strengths!and!

weakness!of!the!model!(based!on!the!Balaklava!experience),!the!forensic!reflective!practice!aimed!to!

identify!aspects!of!the!model!that!need!further!development!beyond!this!research,!as!well!as!any!

areas!of!innovation!that!may!be!of!relevance!for!broader!evaluation!practice!in!community!arts.!!

The!themes!examined!are!the:!modular!and!integrated!approach!of!READ;!trialled!social!

constructivist!stance;!internal!facilitation;!emergent!evaluation!practice;!performance!(as!assessed!

via!meta*evaluation);!and!opportunities!for!enriching!future!evaluation!practice.!

During!my!reflective!analysis!across!these!themes,!I!approached!reflection!on!practice!through!the!

lens!of!my!own!internally!derived!goals!for!betterment!of!my!professional!practice!and!the!

continuous!improvement!of!evaluation!practice!at!the!Nuclear!Futures!site.!This!strategy!generated!

reflective!analysis!of!my!journey!as!the!READ!facilitator,!with!a!frame!of!reference!responsive!to!the!

information!and!insights!being!generated!through!the!collective!experience,!in!accord!with!my!own!

judgment!and!evaluation!experience.!As!outlined!in!Chapter!3,!data!for!my!meta*analysis!was!

collected!through!written!journaling,!video!diaries,!stakeholder!interviews!and!consultation,!

observation!and!use!of!structured!evaluative!tools!from!the!literature.!A!number!of!predominant!

themes!and!findings!emerged!and!these!are!covered!in!Section!6.2.!!

The!second!part!of!the!chapter,!Section!6.3,!consolidates!the!lessons!from!practice!and!presents!the!

substantiated!READ!product!that!has!been!designed!during!the!research,!that!offers!an!evaluation!

tool!for!future!practice!through!the!Nuclear!Futures!program.!The!infographic!in!Section!6.3.3!is!the!

outcome!of!this!product!development!and!is!supported!by!a!profile!summary!that!identifies!the!

characteristics,!assumptions,!constraints!and!variables!of!the!approach.!Section!6.3!articulates!the!

model!of!practice!developed!for!the!READ!approach!that!will!be!relevant!to!my!own!evaluation!

practice!in!the!future,!as!well!as!the!evaluation!practice!of!the!Nuclear!Futures!program!in!2016*7.!


! 135!

Before!presenting!the!findings!from!the!critical!analysis,!the!following!section!provides!an!overview!

of!the!reflective!questions!that!emerged,!and!that!guided!the!forensic!reflective!practice.!!

6.1.2! Questions#arising#from#forensic#reflective#practice#

The!Doctor!of!Creative!Industries!(DCI)!research!strategy!of!forensic!reflective!practice!invites!the!

opportunity!to!question!everything.!Reflective!practice!also!has!particular!relevance!as!an!

evaluator’s!tool,!and!for!critical!reflection!and!meta*evaluation!for!continuous!improvement!

(Davidson,!2005).!For!the!purpose!of!the!research!reflective!practice!became!the!lens!for!examining!

how!well!the!READ!model,!developed!during!the!research,!performed!as!a!learning!and!appraisal!

framework.!In!addition,!for!the!purpose!of!professional!development,!the!reflective!strategy!was!a!

means!to!scrutinise!and!advance!my!own!evaluation!practice.!!

Questions!about!why!elements!of!the!evaluation!practice!worked,!or!did!not!work,!generated!rich!

and!insightful!data.!Questions!also!focused!on!what!could!have!been!done!differently,!and!the!

consideration!of!how!my!answers!might!be!viewed!from!others’!perspectives.!To!my!surprise,!the!

forensic!reflective!practice!conducted!across!the!research!generated!a!large!volume!of!questions.!

The!process!of!questioning!was!as!valuable!as!the!process!of!generating!my!responses!and!analysis.!

Many!of!the!questions!arising!were!investigated!through!applied!reflection!on!the!DCI!process,!and!

others!served!as!technical!or!epistemological!signposts,!which!marked!important!conundrums!

beyond!the!scope!of!the!enquiry,!yet!were!relevant!for!future!investigation!and!contemplation.!

While!the!reflective!questions!produced!from!the!structured!and!reflexive!sessions!were!generated!

to!direct!analysis!towards!the!research!problem,!the!collection!of!questions!that!emerged!

constitutes!a!valuable!research!output!that!is!of!relevance!for!my!future!evaluation!practice.!!

Below,!in!Box!6.1,!is!a!summary!of!selected!reflective!questions!arising!during!the!research,!

generated!from!the!written!exercises!and!video!diary!sessions,!which!include!a!mix!of!strategic!and!

spontaneous,!open!and!closed,!and!evaluative!and!descriptive!(non*evaluative)!questions.!As!a!

collection,!the!questions!arising!from!the!forensic!reflective!practice!demonstrate!the!depth!and!

breadth!of!considerations!associated!with!the!pursuit!of!best!practice,!and!highlight!the!complexity!

involved!in!examining!evaluation!practice!to!check!validity,!utility,!ethical!conduct,!credibility!and!

cost!effectiveness!during!community!arts!program!appraisal.!Their!relevance!extends!beyond!the!

appraisal!of!the!READ!application,!and!provides!insight!into!the!plethora!of!issues!facing!

practitioners!engaged!in!continuous!improvement!of!evaluation!practice.!!

!

!


! 136!

Box!6.1:!Reflective!questions!arising!from!the!forensic!reflective!practice!sessions!for!the!DCI!

!

Questions about READ’s performance as a framework for appraisal of arts-based activities with multiple creative community partnerships:

• What worked well? What didn’t? • What were the challenges for implementation? How were potential limitations or inhibitors overcome? • What does a SWOT analysis on the application reveal? • Any failures? Why? • What outcomes are evident during and post application? • How well did READ perform as a guiding framework for program development and/or learning? • What were the results of the participatory elements? • Where those implementing READ familiar enough with the context? • Was there adequate evaluation expertise in evaluation and the subject of the evaluation? • How well were any conflicts of interest dealt with? • What does the experience of stakeholders say about READ? • Were outcomes and outputs satisfactory and useful for partner needs/expectations? How well? • Were findings and communications timely for decision-making processes? • Were the costs and inputs incurred by stakeholders and partners reasonable in relation to benefits? • What was the opportunity costs of the personnel devoted to READ application? • Were the conclusions arising from READ valid and justified? Clearly communicated? • Does it suit the context and partnership challenges? • How does READ perform against Scriven’s Key Evaluation Checklist for meta-evaluation? (Scriven,

2003) • Where did READ deliver most value? Were there omissions? If so, were they serious enough to

compromise validity or accuracy of findings? (Davidson, 2005) • How well was the READ application conducted in terms of ethical and professional standard

measures? (Australasian Evaluation Society, 2010) • Was the design and implementation of the tailored READ model culturally appropriate? On what

grounds? • What improvements/refinements of the model and support structures are necessary? • Are findings and lessons transferable and relevant for future practice? • What aspects of READ were particularly relevant to the needs for the creative community

partnerships of the Nuclear Futures Partnership Initiative experience? • What practical lessons arise for the use of READ at the Nuclear Futures site of practice? • How well does the model generalise to be of relevance to other practitioners? • What are the assumptions, variables and constraints of the application and site of practice? • In what ways and to what extent did the READ model advance evaluation practice? • What outstanding areas of innovation for community arts management and evaluation are

identifiable? What attributes were unique, effective, creative and/or outstanding? • What is the scope and significance of impact during the application? • What outcomes, impacts or dynamics were surprising or unexpected? • What can be improved for future READ applications? • How does the READ application at the site of practice relate to the community arts evaluation? • What are the original aspects of the READ model? • What does the READ product arising from the research look like? • What areas of READ need further research and development? What are areas of future research or

direction?


! 137!

6.2! Findings#and#discussion#

6.2.1! Modular#and#integrated#approach#

Through!the!development!of!the!READ!product,!I!examined!the!function!and!evaluative!capacity!of!

each!READ!strategy!to!build!a!theoretical!understanding!of!the!model!and!critically!review!READ’s!

performance!as!a!framework!for!appraisal.!For!the!purpose!of!analysis,!I!defined!appraisal!to!

encompass!the!broad!definition!of!evaluation!formulated!at!the!site!of!practice!(which!included!

merit*driven!processes!as!well!as!descriptive!accounts!without!judgement),!as!well!as!those!more!

unassuming!processes!of!sensemaking!and!reflective!assessment!that!were!inherent!in!the!creative!

arts!practice.!I!identified!that!the!focus!of!appraisal!at!the!site!of!practice!related!to:!the!creative!

processes;!artworks!produced;!community!engagement;!project!progress;!and!the!social!dynamics!

and!quality!of!the!partnerships.!

As!articulated!in!Chapter!5,!each!READ!strategy!(Reflection,!Analysis,!Evaluation,!Documentation)!

was!driven!by!different!impulses!and!approaches!to!learning!and!appraisal,!and!drew!on!different!

method!sets.!To!gain!a!holistic!picture!of!the!tailored!READ!framework!for!the!Balaklava!project,!I!

was!interested!in!understanding!what!appraisal!activities/methods!operated!across!all!of!the!four!

strategies,!as!well!as!those!undertaken!specifically!outside!the!Evaluation!strategy!(which!already!

resembled!conventional!evaluation).!My!intent!was!to!analyse!what!the!other!three!READ!strategies!

(and!the!integrated!system!as!a!whole)!contributed!to!the!appraisal!outcomes,!beyond!the!

Evaluation!strategy!alone.!In!one!reflective!practice!session!for!example,!I!considered:!What!is!the!

difference!between!READ!as!an!integrated!approach!and!a!conventional!evaluation*only!lens?!

The!systems!approach!was!useful!for!determining!and!exploring!the!activities!of,!and!relationships!

between,!the!strategies!(see!systems!approach!in!Section!5.1.2).!In!summary,!the!four!READ!

strategies!were:!reflection!for!establishing!a!culture!of!reflective!practice;!evaluation!driven!by!

agreed!key!evaluation!questions!in!order!to!make!evaluative!conclusions;!analysis!via!academic!

research!to!generate!broader!learning!and!dissemination!of!original!knowledge;!and!documentation!

to!deliver!creative,!program!and!legacy!artefacts!to!help!sensemaking!within!the!project!and!the!

broader!field!of!community!arts.!The!study!in!Chapter!5!showed!that!across!all!four!strategies,!a!

diverse!mix!of!appraisal!methods!were!employed,!including!conventional!evaluation!instruments,!

qualitative!methods!(such!as!interviews!and!group!feedback!sessions),!as!well!as!creative!and!

reflective!methods.!This!suite!of!methods!was!used!for!reviewing!the!art*making!(creative!

development,!content,!production!and!community!arts!and!cultural!development!(CACD)!practice),!

partnership!development,!and!the!artistic!vibrancy!of!the!creative!work!being!generated.!It!was!

evident!that!the!evaluative!processes!from!each!strategy!overlapped,!and!I!was!surprised!at!how!far*


! 138!

ranging!the!evaluative!activities!were!across!the!READ!system,!and!also!how!many!were!directly!

drawn!from!the!creative!practice!and!culture!of!reflective!practice.!!

The!Analysis!and!Documentation!strategies!produced!valuable!descriptive!content,!which!

predominantly!did!not!focus!on!evaluative!conclusions.!Rather,!these!strategies!illuminated!several!

of!the!broader!implications!of!the!community!arts!practice!in!its!wider!social!context.!Even!for!

outputs!produced!that!did!not!include!overt!evaluative!conclusions,!it!was!evident!in!many!instances!

that!the!processes!involved!in!generating!those!outputs!contained!embedded!evaluative!processes,!

such!as!the!framing!of!storytelling,!and!the!choices!made!during!any!form!of!visual!representation!or!

video!editing.!The!implication!was!that!READ!produced!a!mix!of!evaluative!and!non*evaluative!

outputs!that!were!all!considered!the!results!of!evaluation!practice!through!the!READ!approach!at!

the!site!of!practice!(see!Section!5.5!for!a!discussion!on!utility!and!READ%in%Practice%portfolio!

(RIPP):!Section!F,!p.117!for!examples!of!the!mix!of!READ!outputs).!

I!established!that!while!reflection,!analysis!and!documentation!are!certainly!all!well!known!and!are!

already!used!in!conventional!evaluation!as!methods,!the!difference!was!that!within!the!READ!

framework!they!operated!as!strategies!that!served!their!own!set!of!strategic!objectives.!Each!of!the!

READ!strategies!had!independent!yet!interrelated!plans!for!knowledge!generation,!related!to!the!

mix!of!evaluative!and!non*evaluative!aims!of!the!READ!system!as!a!whole.!Sometimes!the!methods!

used!for!fulfilling!each!strategy!overlapped.!One!significant!finding!was!how!the!benefits!from!one!

strategy!had!the!potential!to!enable!or!enhance!the!success!of!another.!For!example,!the!sustained!

culture!of!reflective!practice!meant!a!continuous!supply!of!data!relevant!for!Evaluation,!Analysis!and!

Documentation.!

I!concluded!that!when!combined,!the!four!strategies!generated!appraisal!and!learning!outcomes!

that!would!not!have!been!achieved!by!the!Evaluation!strategy!alone,!and!secondly!that!the!other!

three!READ!strategies!enhanced!the!effectiveness!of!the!Evaluation!strategy!in!addressing!the!

project’s!key!evaluation!questions.!Examples!of!the!outcomes!that!would!not!have!been!created!by!

the!Evaluation!strategy!alone!included!the!goal*free!creative!documentation,!the!community!of!

practice!(of!researchers!and!community!partners)!established,!community!archiving!and!public!

collection!development,!and!the!wider!dissemination!pathways!that!were!developed!for!the!

dissemination!of!findings!to!industry!and!academic!audiences.!The!results!from!the!READ!application!

demonstrated!that!bringing!together!data!generated!from!the!four!READ!strategies!enabled!a!wide!

summative!lens!to!be!created!for!recording,!reviewing!and!representing!community!arts!activities.!

!


! 139!

In!addition,!there!were!also!conceptual!benefits!arising!from!the!integrated!approach!of!the!four!

READ!strategies.!At!the!site!of!practice,!I!observed!that!the!READ!approach!shifted!the!focus!off!

‘evaluation’!and!onto!a!broader,!experimental!endeavour!of!sensemaking!about!the!arts!practice,!

and!this!was!engaging!for!stakeholders!(including!those!who!had!made!known!their!trepidation!and!

scepticism!towards!evaluation).!In!particular,!the!engagement!with!artists!was!secured!through!the!

creative!documentation!and!reflection!practices,!and!for!the!community!partners,!their!interest!in!

READ!was!ignited!by!the!legacy!and!broader!learning!opportunities!it!promised.!Once!engaged!and!

familiar!with!the!READ!approach,!I!observed!that!participation!then!spilled!over!into!Evaluation!and!

Analysis!activities,!and!that!there!was!renewed!receptiveness!to!the!more!standard!types!of!

evaluation!methods!within!the!READ!set.!Also,!once!trust!and!interest!had!been!established,!

participating!stakeholders!were!exposed!to!new!methods!(previously!unfamiliar!to!them),!and!many!

participated!in!data!collection!and!analysis!activities!that!were!beyond!their!original!sphere!of!

interest!or!which!may!have!lacked!credibility!initially!(for!them).!

One!example!is!as!follows.!In!a!planning!meeting!to!design!the!secondary!school!engagement!

sessions!(which!were!to!occur!after!students!had!viewed!the!10%Minutes%to%Midnight!projection!

installation!as!part!of!the!education!program!for!schools!(see!Chapter!4)),!I!asked!the!creative!team!

if!they!could!facilitate!the!student!evaluation!as!part!of!the!‘meet!the!artist’!session.!This!would!

involve!facilitating!the!process!of!students!filling!out!the!evaluative!feedback!forms!(included!in!

RIPP:!Article!19,!p.70).!The!artists!had!previously!expressed!interest!in!attaining!feedback!from!

students,!however!the!forms!were!a!much!more!structured!method!than!they!had!envisaged.!While!

this!was!agreed!to,!I!observed!some!reluctance!to!engage!with!the!delivery!of!the!standard!

evaluation!instruments.!I!got!the!impression!that!the!forms!were!perceived!as!a!bureaucratised!

means!of!engaging!with!the!audience.!!!

We!trialled!the!creative!team!delivering!the!evaluation!instrument!in!the!first!student!class!session,!

and!as!the!sessions!progressed,!the!artists!became!increasingly!engaged!in!the!evaluation!data!that!

was!being!generated!and!the!richness!of!the!resulting!discussions!with!students!(because!students!

had!already!formulated!ideas!via!the!form!to!input!into!the!collective!discussion).!The!data!collected!

became!immediately!relevant,!both!as!feedback!on!the!creative!work!and!as!a!guide!for!the!iterative!

improvement!of!how!the!sessions!with!students!worked!best.!By!the!third!student!class!through!the!

installation,!the!artists!were!gathering!together,!reviewing/scanning!the!collected!forms!and!

discussing!outstanding!comments!or!questions!at!the!end!of!each!session.!Over!the!two!days,!the!

end*of*session!review!of!the!forms!by!artists!continued!and!built!momentum.!The!creative!team!

delivering!the!forms!also!started!to!experiment!with!how!the!forms!were!used!in!order!to!maximise!

!


! 140!

student!engagement!and!input.!Increased!time!and!attention!was!dedicated!to!the!use!of!the!

evaluation!instrument,!and!this!evolved!to!be!coupled!with!reflection!in!pairs!to!further!draw!out!

student!ideas.!The!evaluation!activity!went!from!being!a!potential!imposition!to!a!valued!tool!for!

engagement!and!provided!data!on!the!creative!content!of!the!showcase!ahead!of!the!public!season,!

which!led!the!artists!to!make!refinements.!This!is!a!prime!example!of!real*time!utility!resulting!from!

the!participatory!approach!that!involved!stakeholders!in!the!design!and!implementation!of!READ!

processes,!and!shows!the!interactivity!between!evaluation!practice!(through!READ),!creative!

practice!and!program!management.!!!

The!reflective!analysis!also!highlighted!several!lessons!about!approaching!evaluation!as!an!

integrated!practice!and!revealed!that!each!READ!strategy!showed!evidence!of!operating!well!as!

stand*alone!elements!(modules).!Analysing!the!four!READ!strategies!as!distinct!operational!

components!of!the!model!of!practice!tested!the!notion!that!the!READ!strategies!have!the!potential!

to!function!independently!(as!modular!elements!in!the!system).!Part!of!this!analysis!was!dedicated!

to!examining!how!well!the!READ!strategies!functioned!for!their!own!unique!purpose.!!

From!the!outcomes!described!in!Chapter!5,!it!was!determined!that!each!READ!strategy!did!well!in!

achieving!this.!Take,!for!example,!the!Reflection!strategy,!aiming!to!generate!a!culture!of!reflective!

practice!for!learning!and!improvement!outcomes.!The!devised!approach!for!this!strategy!included!

primary,!secondary!and!tertiary!processes!of!reflection!(explained!in!Figure!5.4),!and!produced!

developmental!outcomes!for!individuals!and!the!organisation,!such!as!professional!development,!

skills!development!and!organisational!learning.!Another!example!is!the!Analysis!strategy,!and!how!it!

achieved!the!objective!of!generating!original!knowledge!through!academic!enquiry,!which!has!been!

demonstrated!through!the!research!outputs!of!the!community!of!practice!so!far.!I!concluded!that!

each!READ!strategy!represented!an!established!and!viable!learning!strategy!that!could!be!used!

independently!or!combined.!This!finding!indicates!the!potential!flexibility!of!the!model!for!delivering!

a!spectrum!of!appraisal!and!learning!needs!arising!from!diverse!contexts!and!partnerships.!

The!learning!for!my!own!evaluation!practice!as!facilitator!was!the!insight!the!study!of!the!READ!

application!offered!into!the!many!ways!that!we!had!gone!about!learning!and!appraisal!of!the!

activities!and!arts!practice.!Many!of!these!evaluative!activities!were!already!inherent!in!community!

arts!processes,!yet!they!were!tacit.!This!process!enabled!the!practical!wisdom!arising!from!the!

community!arts!to!be!made!explicit!through!the!deliberate!focus!and!methods!of!the!READ!

methodology.!I!also!observed!that!the!acronym!of!‘READ’!in!itself!offered!value!as!a!collective!

compass!for!our!shared!evaluation!practice.!The!easy!to!remember!acronym!was!useful!for!quickly!

assessing!a!situation!in!terms!of!its!potential!for!data!collection!by!scanning!the!four!READ!focus!


! 141!

areas!to!assess!relevance!and!opportunity!during!an!activity.!While!professional!evaluators!have!a!

level!of!coherency!about!how!to!undertake!evaluation!practice!(based!on!observation!from!previous!

experience),!this!is!not!the!case!for!many!creative!practitioners,!artists!and!technical!professionals!

who!may!benefit!from!having!a!conceptual!guide!such!as!the!READ!acronym!and!framework.!!

From!the!analysis,!I!determined!the!READ!approach!had!instrumental!value,!developmental!value!

and!conceptual!value!for!the!stakeholders.!The!following!summarises!the!emergent!findings!on!this!

theme!of!constructing!the!READ!model!to!have!both!modular!and!integrated!possibilities:!

•! Formal!evaluation!processes/methods!were!contextualised!and!strengthened!from!being!

presented!and!managed!as!part!of!the!broader!integrated!READ!framework.!

•! Information!from!different!program!areas,!art*making!processes!and!knowledge!bases!were!

connected!and!integrated!through!the!one!system,!and!also!linked!to!program!management.!

•! The!READ!approach!functioned!effectively!as!an!integrated!system!and!each!modular!

approach!employing!individual!strategies!was!effective!as!a!self*contained!subsystem.!

•! The!READ!strategies!draw!on!established!methods!that!are!not!new!to!evaluators!and!

creative!practitioners.!However,!the!innovation!of!the!system!lies!in!their!integration.!!

•! The!methodology!provides!a!means!to!make!explicit!some!of!the!often!under*valued!or!

disregarded!tacit!systems!of!knowledge!generation!inherent!in!community!arts!practice!and!

program!management,!providing!a!framework!inclusive!of!traditional!evaluation!

instruments!and!experimental,!arts*based!or!partner*specific!methods.!!

•! The!relevance!of!evaluation!instruments!and!traditional!methods!within!the!integrated!

framework!for!identifying,!deciphering!and!valuing!program!impact!was!affirmed!and!

offered!a!range!of!benefits!to!community!participants,!project!staff!(including!creative!

team),!volunteers,!program!partners!and!mentors.!

•! As!a!summative!lens,!READ!explored!the!efficacy!of!the!project!in!terms!of!meeting!its!stated!

objectives,!as!well!as!the!artistic,!theoretical,!academic!or!the!wider!social!implications!of!

community!arts!practice!(as!delivered!by!the!multi*strategy!approach).!!

•! There!were!benefits!from!the!extended!scope!of!evaluation!practice!via!READ.!

•! The!integrated!approach!delivered!a!wide*range!of!outputs!that!mixed!evaluative!and!non*

evaluative!products!that!were!relevant!for!gaining!an!understanding!of!community!arts!

practices.!As!a!collection,!these!were!beyond!what!would!have!been!delivered!with!an!

evaluation*only!approach.!


! 142!

6.2.2! Social#constructivist#stance#

Identifying*and*understanding*conceptualisations*of*evaluation*

Given!this!was!the!first!industry!application!of!READ,!and!that!the!methodology!and!epistemology!

were!being!developed!along!the!way,!the!level!of!scoping,!preparation!and!design!consultation!

required!a!large!amount!of!concerted!time!and!resources.!As!is!outlined!in!Section!5.3.3,!the!results!

from!the!scoping!research!I!conducted!revealed!that!stakeholders!used!many!different!constructs!of!

evaluation,!and!this!meant!that!stakeholders!had!mixed!world!views!and!interpretations!of!

evaluation!and!its!value.!Also!identified!was!a!lack!of!clarity!for!some!stakeholders!about!their!

evaluation!needs!and!definition!of!success.!!

The!constructs!of!evaluation!presented!by!stakeholders!were!influenced!by!their!context,!past!

experiences!of!evaluation,!and/or!the!values!of!the!organisation!stakeholders!represented.!One!

interesting!theme!arising!from!the!data!was!trepidation!expressed!by!multiple!stakeholders!towards!

evaluation!of!community!arts.!One!creative!practitioner,!for!instance,!questioned!how!formal!

evaluation!could!contribute!meaningfully!towards!professional!learning!and!future!practice,!while!

another!rejected!outright!the!legitimacy!of!evaluation!practice!for!community!arts!altogether:!‘I!

don't!like!evaluations,!I!don't!like!those!standard!evaluations!and!I!particularly!don't!like!arts!funding!

body!evaluations’!(see!Section!5.3.3).!Analysing!the!emergent!theme!of!evaluation!trepidation!

featured!heavily!in!my!forensic!reflective!practice!during!the!scoping!phase!and!was!of!personal!

interest,!as!it!resonated!with!my!observations!and!experience!from!previous!management!and!

evaluation!work.!!

I!interpreted!the!evident!mistrust!and/or!evaluation!fatigue!of!some!stakeholders!as!a!potential!

barrier!to!the!effective!delivery!of!the!READ!application,!and!wanted!to!examine!this!finding!in!

detail.!I!recognised!that!these!views!would!also!have!a!bearing!on!partnership!brokerage!and!

engagement.!While!the!trepidation!theme!was!not!representative!of!all!stakeholder!responses,!I!

considered!it!important!for!me!to!develop!a!sophisticated!understanding!of!this!dimension,!in!order!

to!progress!the!READ!model!development!and!application.!

My!interpretation!of!the!data!from!interviews!and!consultation!was!that!for!some!stakeholders,!the!

pursuit!of!merit*based!and!formal!evaluation!were!perceived!as!unwanted!acts!of!institutionalisation!

or!instrumentalisation!of!their!creative!practice!to!fit!a!social!benefits/community!development!lens,!

or!that!the!process!was!a!waste!of!time/effort!where!no!apparent!end*utility!or!meaningfulness!

eventuated.!One!tension!identified!was!that!orthodox!and/or!formalised!evaluation!models!and!

instruments!were!perceived!to!undervalue!or!dismiss!the!relevance!of!the!reflective!and!evaluative!


! 143!

processes!inherent!in!creative!arts!practice.!

The!evaluation!trepidation!sub*theme!also!resonated!with!my!own!professional!experience!and!

observation.!In!order!to!identify!and!assess!my!espoused!values,!forensic!reflective!practice!was!

employed!to!examine!my!conceptualisations!of!evaluation!and!my!possible!biases.!Reflective!analysis!

revealed!and!clarified!that!my!trepidation!of!evaluation!was!of!strictly!positivist!approaches,!and!I!

also!identified!some!scepticism!(based!on!my!observation!and!professional!experience)!that!too!

often!evaluation!outputs!(traditionally!word*heavy!reports)!are!under*utilised,!not!adequately!

disseminated!or!worst,!sit!unread!or!redundant!on!a!funding!agency’s!shelf.!I!also!reflected!on!

previous!discussions!with!industry!peers!and!colleagues,!which!identified!an!observed!tendency!

within!the!community!arts!context!for!practitioners!and!organisations!to!talk!up!the!impact!and!

outcomes!of!activities!(or!present!positive*lens!only!findings),!in!case!favourable!evaluation!and!

acquittal!reporting!relates!to!future!funding.!This!trend!within!the!field!reduces!the!meaning!that!

some!practitioners!place!on!evaluations!and!discourages!learning!about!the!failures!and!responses!

that!may!have!been!part!of!a!project.!This!means!that!a!representative!picture!of!creative!practice!

and!its!consequences!is!difficult!to!establish.!

As!covered!in!Chapter!2,!trepidation!of!community!arts!evaluation!was!evident!in!the!literature!and!

industry!forums!I!cited!for!the!contextual!scan!(Belfiore,!2014;!Jackson,!2009;!Jennings!&!Baldwin,!

2010;!Newman,!et!al.,!2003;!Donovan,!2012;!Selwood,!2002).!Most!striking!was!the!quote!included!in!

Chapter!2!by!Goldbard!(2008)!arguing!the!detrimental!effect!evaluation!and!the!instrumentalisation!

of!the!arts!has!had!on!the!field.!As!part!of!her!argument,!Goldbard!asserts!that!damage!is!done!when!

terminology!and!assumptions!are!borrowed!from!other!fields!and!misappropriated!for!use!in!

community!arts.!Such!an!argument!supports!Alphaville’s!initial!impulse!to!draw!on!the!methods!and!

philosophy!of!community!arts!(see!Goldbard,!2015,!2008).!

The!reflective!analysis!was!driven!by!questions!such!as:!how!can!we!make!READ!inclusive!and!how!

can!such!divergent!views!of!evaluation!co*exist,!or!be!represented,!explored!and!responded!to!

collectively!during!the!READ!application?!The!implication!of!acknowledging!the!diversity!of!values!

about!evaluation!was!that!across!the!READ!application,!there!were!many!open!and!frank!discussions!

amongst!participating!stakeholders!that!questioned!and!challenged!what!READ!should!deliver!and!

what!the!boundaries!of!the!evaluation!practice!could!and!should!be.!For!my!own!evaluation!practice,!

I!gained!skills!in!identifying!and!articulating!different!stakeholder!constructs!of!evaluation!(as!READ!

facilitator!and!manager)!and!negotiated!these!with!multiple!partners,!which!offered!deeper!insight!

into!the!role!that!partnership!brokerage!plays!in!facilitating!a!shared!dialogue!and!establishing!the!

terms!of!reference!for!evaluation!practice.!!


! 144!

Negotiating*evaluation*constructs*

One!way!in!which!I!addressed!the!findings!of!the!scoping!phase!with!stakeholders!was!by!

negotiating!and!establishing!a!wide!frame!of!reference!for!our!internal!evaluation!practice,!during!

our!application!of!the!multi*strategy!READ!approach.!As!explained!above,!this!involved!open!

dialogue!with!all!partners.!Of!interest,!was!one!negotiation!process!with!the!Australia!Council!for!the!

Arts,!as!funding!body,!which!had!significant!bearing!on!the!READ!application!that!eventuated.!An!

issue!had!arisen!from!Alphaville’s!proposal!to!conduct!the!READ!application!as!an!internal!process,!

which!was!contrary!to!the!specification!for!independent!evaluation,!as!stated!within!the!Australia!

Council!funding!contract.!READ!also!included!activities!beyond!the!conventional!scope!of!evaluation.!

Thus,!the!prospective!READ!application!required!discussion!and!negotiation.!The!outcome!of!this!

process!was!important,!in!light!of!another!of!the!contract!specifications,!that!required!10!per!cent!of!

the!secured!funding!value!(a!significant!amount!of!resources)!to!be!dedicated!to!evaluation.!!

Despite!many!of!the!READ!practices!not!being!strictly!evaluative!in!outcome,!it!was!agreed!that!wide!

representations!and!accounts!of!the!program!would!provide!insight!into!the!wider!social!

implications!of!the!arts!practice!and!for!an!experimental!program!such!as!Nuclear!Futures!this!was!

desirable.!The!scoping!process!with!stakeholders!resulted!in!license!being!granted!to!myself!and!

Alphaville!to!experiment!with!a!broad!mix!of!learning!and!appraisal!strategies!(beyond!‘evaluation’).!!

While!the!READ!application!represented!a!departure!from!the!positivist!foundation!of!program!

evaluation,!it!was!by!no!means!a!rejection!of!evaluation!as!a!systematic!practice!for!learning!and!

appraisal.55!As!one!of!the!four!core!READ!strategies,!evaluation!commands!a!central!position!in!the!

READ!practice.!At!the!same!time!however,!READ’s!integration!responds!to!the!limitations!of!current!

practice!(see!problem!statement!in!Chapter!1)!and!reasons!that!resources!allocated!to!‘evaluation’!

in!the!community!arts!context!might!be!best!spent!across!a!spectrum!of!critical!review,!learning!and!

appraisal!strategies!(using!qualitative!and!quantitative!methods).!In!deviating!from!the!dominant!

empirical!epistemologies!of!evaluation!practice!towards!more!constructivist!approaches!–!where!no!

objective!or!unified!judgment!of!program!worth!may!result!from!the!‘evaluation’!process!–!advocacy!

and!negotiation!on!a!project*to*project!basis!is!needed!to!secure!the!agreement!of!stakeholders.!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

55!It!is!worth!reiterating!that!general!discussion!about!‘evaluation’!is!fraught!given!the!term!encompasses!such!a!broad!

range!of!definitions,!underpinned!by!divergent!value!paradigms,!and!that!there!are!myriad!approaches!to!how!it!is!

undertaken.!In!the!literature,!the!diversity!for!what!constitutes!evaluation!is!much!more!clearly!acknowledged!than!is!

evident!in!contemporary!practice!where!a!somewhat!limited!scope!for!practice!is!demonstrated,!conforming!to!the!

definition!of!evaluation!focused!on!research!for!the!purpose!of!making!judgments,!usually!about!the!effectiveness,!

efficiency!and/or!appropriateness!of!an!activity!(such!as!those!definitions!by!Australasian!Evaluation!Society,!2010;!Scriven,!

2012;!and!Davidson,!2005).!!


! 145!

For!Nuclear!Futures,!the!conditions!were!favourable!for!such!negotiation,!and!renegotiation,!of!how!

widely!the!evaluation!resources!could!legitimately!be!allocated.!I!attribute!this!outcome!to!the!

experimental!context!of!the!Creative!Communities!Partnership!Initiative!(CCPI)!funding!scheme!

(which!was!established!to!encourage!recipient!organisations!to!be!reflexive!as!a!means!of!promoting!

innovation)!and!the!fact!that!the!partners!acknowledged!the!differing!knowledge!paradigms,!

languages,!ideologies!and!cultural!systems!at!the!table!(which!included!Aboriginal!and!non*

Aboriginal!organisations!and!community!partners,!academic!institutions,!international!communities,!

government,!non*for*profit!and!community!organisations).!

Social*construction*

The!READ!approach!for!Nuclear!Futures!also!challenged!the!‘independent’!requisite!of!the!Australia!

Council.!As!explained!in!Chapter!5,!it!was!determined!that!a!constructivist!stance!for!the!READ!

application!in!Balaklava,!and!for!Nuclear!Futures!generally,!would!be!favourable!for!accommodating!

the!differing!partner!values!identified.!The!interpretative!constructivist!stance!recognises!that!

individuals!and!groups!construct!their!perceived!reality!(Glaser!&!Strauss,!2009;!Charmaz,!2000;!

Cresswell,!1994),!and!was!seen!as!a!credible!stance!by!the!stakeholders!because!it!had!the!capacity!

to!deal!with!a!register!of!voices!and!knowledge!systems.!The!implication!was!that!the!evaluation!

practice!was!not!conducted!to!conform!to!notions!of!seeking!objective!summative!judgments.!

The!data!reviewed!for!the!analysis!evidenced!that!the!constructivist!approach!for!the!READ!

application!proved!effective!for!accommodating!and!being!inclusive!of!the!divergent!world!views!

and!evaluation!needs!arising!from!the!community!partnerships.!Stakeholder!input!from!informal!

feedback,!interviews!and!reflective!discussions!on!the!evaluation!processes!and!outcomes!(and!the!

overall!experience!of!READ)!indicated!consistently!that!READ!deliverables!in!the!domains!of!

appraisal!and!engagement!exceeded!expectations!and!requirements.56!!

While!the!constructivist!approach!was!effectively!applied!in!this!context,!it!is!conceivable!that!future!

READ!applications!could!have!success!using!alternative!interpretative!stances!(if!deemed!

appropriate!for!the!context!and!partners)!because!of!the!methodological!design!flexibility!READ!

offers.!For!example,!in!any!of!the!READ!strategies,!a!different!interpretative!stance!would!lead!to!a!

different!methodological!orientation!and!method!selection,!as!long!as!the!chosen!stance!

accommodated!the!needs!and!values!at!the!site!of!practice.!This!claim!is!speculative!and!is!a!

potential!area!for!future!research.!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

56!Data!from!observation!and!communications!corroborated!this!finding.!


! 146!

6.2.3! Internal#facilitation#

Of!paramount!significance!for!the!outcomes!of!the!READ!application!for!the!Balaklava!project!was!

the!internal!position!of!the!READ!delivery,!where!design!and!implementation!of!the!tailored!READ!

framework!was!undertaken!by!stakeholders!situated!within!the!project.!Likewise,!the!leadership!

governing!the!READ!application!(my!role!as!facilitator)!was!embedded!within!the!project!and!arts!

practice,!and!closely!linked!with!my!role!as!Alphaville’s!program!manager!for!Nuclear!Futures.!

For!the!Balaklava!project,!it!was!evidenced!that!the!internal!position!of!the!READ!facilitation!was!

effective!and!generated!particular!benefits!as!a!result!of!this!positioning.!In!my!role!as!facilitator,!my!

existing!knowledge!of!the!politics,!organisational!culture,!project!structure,!personnel,!partners!and!

community!participants!(including!my!experience!of!working!with!some!community!partners!

previously)!gained!from!my!manager!role!created!immediate!and!sustained!advantages!for!

progressing!the!READ!delivery.!This!was!also!true!for!other!participating!stakeholders!to!varying!

degrees,!who!were!able!to!bring!their!local!knowledge!or!familiarity!with!the!project!into!the!

appropriate!tailoring!of!the!design!and!implementation!of!READ!for!the!Balaklava!project.!

Beyond!the!general!benefits!attributed!to!the!internal!position!of!the!READ!delivery,!the!analysis!

showed!that!positive!outcomes!were!enhanced!by!my!dual!role!as!READ!facilitator/program!

manager.!For!example,!relationship!building!and!establishing!trust!with!participants!was!already!at!

an!advanced!level!from!the!commencement!of!the!READ!application,!and!in!this!dual!role!I!had!

ready!access!to!project!resources!and!communication!channels!with!participants.!The!dual!

facilitator/manager!role!also!meant!I!was!able!to!feed!raw!and!processed!READ!data!into!the!

decision*making!cycles!of!the!creative!process,!program!management!and!strategically!link!delivery,!

activities,!resource!and!outcomes!between!the!arts!practice!and!READ.!One!outcome!was!increased!

reflexivity!of!program!planning!and!management,!and!real*time!impact!of!evaluation!data!collected.!

A!specific!demonstration!of!how!my!capacity!as!READ!facilitator!was!aided!by!the!enriched!

understanding!of!the!political!and!social!dimensions!relevant!for!the!evaluation!conduct!–!a!direct!

result!of!the!internal!position!–!was!in!regard!to!known!sensitivities!and!partnership!brokerage!

needs!for!this!specific!project!and!community!of!interest.!For!the!Balaklava!project,!there!were!

unique!personal!and!political!sensitivities!related!to!the!subject!matter!of!atomic!testing!for!nuclear*

affected!communities!and!general!audiences!being!exposed!to!this!content!for!the!first!time.!The!

testimonies!and!information!at!the!centre!of!the!community!arts!for!this!project!can!be!distressing!

and!uncomfortable!for!participants,!artists!and!audiences.!For!community!participants!especially,!

having!familiar!insiders!(myself!and!the!Nuclear!Futures!team!delivering!READ)!maintained!a!level!of!

continuity!and!confidence!across!the!art*making,!management!and!review!processes.!Inviting!and!


! 147!

collecting!READ!input!also!became!a!normalised!and!integrated!process!across!project!operations!

(and!evolving!sub*culture),!in!simple,!efficient!and!opportunistic!ways!with!our!community!partners.!!

Figure!6.1!is!a!diagram!sourced!from!my!reflective!practice!documentation!(originally!hand!drawn)!

where!I!had!graphically!explored!the!implications!of!the!internal!position!in!relationship!to!the!

manager!role.!The!top!diagram!in!Figure!6.1!represents!the!scenario!when!the!two!roles!are!

distinct/separate,!and!the!reflective!extract!explores!the!implications!of!this!approach.!The!lower!

diagram!represents!the!dual!position!of!evaluator/manager!(where!evaluator!terminology!is!

equivalent!to!READ!facilitator)!and!discusses!the!observed!efficiencies!achieved,!while!also!warning!

that!validity!checks!are!needed.!The!reflection!was!based!on!observation!from!the!Balaklava!project!

as!well!as!on!my!previous!experience!of!both!scenarios.!It!is!included!as!example!of!data!collected!in!

my!primary!reflective!sessions!during!the!forensic!reflective!practice.!

!

!

!

Figure!6.1:!Extracts!from!reflective!practice:!exploring!the!internal!positioning!of!READ!facilitation!

!

Original notes: a significant relationship exists between the two roles of manager and lead evaluator if conducted by different individuals, each is important to the other. Open communication and shared information systems are needed to make optimal. Program manager is an important champion for the evaluator, the eyes of the project. Evaluator has the tools and expertise for capture and assessment, and is a program management resource for reviewing operations, via an evaluative- analytical lens. Timeliness is important for evaluator to feed in information, as evaluation data impacting creative practice or program management may need real-time action. ‘Objectiveness’ vs opportunity for use?

**Tim

e*access*communicatio

n**!

**validity

!checks!im

portant**!

Original notes: utility and relevance of both processes may be maximised by manager as internal evaluator, if issue of conflicts of interest, bias and validity is addressed. There are so many times, ongoing, that I am making management decisions based on READ data. Time was saved in terms of setting up the lines of communication and accounting between roles in preparation phase. Promotes data collection via program documentation and incidental feedback from informal sources arising from management communications. If validity can be managed, the exchange between roles is rich and rewarding.

!


! 148!

In!the!READ!application,!one!of!the!overriding!benefits!of!the!internal!position!and!dual!role!was!the!

real*time!access!created!and!resultant!utility!of!the!information!being!generated!throughout!the!

READ!and!project!cycles.!Internal!management!also!meant!I!was!able!to!respond!to!gaps!in!data!

collection!and!instigate!opportunities!to!collect!further!data!where!necessary.!It!is!important!to!

acknowledge!however!that!there!are!limitations!and!compromises!for!the!internal!position!of!the!

evaluation!practice,!in!regard!to!implications!for!validity!and!credibility!(I!address!these!in!Section!

6.2.5).!More!broadly,!the!benefits!of!the!internal!approach!may!have!been!undermined!if!tensions!or!

conflicts!had!arisen!between!partners,!which!was!not!the!case!for!the!Balaklava!partners,!or!if!the!

facilitation!of!the!READ!was!conducted!via!unethical!means,!or!if!bias!privileged!some!partners!over!

others!in!terms!of!participation,!inclusion!of!input,!representation!and!access!to!information.!!

For!the!Balaklava!experience!and!READ!application,!I!established!that!positive!outcomes!resulted!

from!the!internal!facilitation,!and!these!were!well!supported!from!the!data!collected.!Overall,!the!

assessment!led!me!to!conclude!that!a!number!of!the!utility,!engagement!and!empowerment!

outcomes!were!evident!for!the!Balaklava!project!as!a!result!of!the!internal!position!and!dual!role.!

While!context*specific,!I!interpret!this!to!be!a!relevant!finding!for!the!wider!community!arts!context,!

as!the!example!shows!that!creative!practitioners!and!community!partners!at!the!heart!of!the!arts!

were!practice!are!well*positioned!to!undertake!the!evaluative!practice!(assuming!an!adequate!level!

of!evaluation!experience!exists!or!can!be!accessed),!and!their!proximity!for!collecting!data,!and!the!

relevance!of!their!collective!insight!for!making!sense!of!the!data,!has!the!potential!to!enhance!the!

overall!outcomes!of!evaluation!practice.!The!finding!also!raises!a!question!about!the!value!and!

appropriateness!of!the!routine!prescription!by!funding!bodies!for!independent!evaluation!for!the!

community!arts!context.57!Assuming!that!issues!such!as!validity!and!credibility!can!be!adequately!

managed!and!ensured!through!methodological!means,!the!internal!position!of!evaluation!offers!

potential!partnership*oriented!benefits!for!this!context.!

! !

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

57!While!independent!evaluation!does!not!exclude!internal!facilitation,!it!does!mean!that!significant!resources!need!to!be!

invested!in!verification!of!the!internal!process!by!an!external!specialist!(at!the!very!least).!


! 149!

6.2.4! Emergent#evaluation#practice#

Theoretical*orientation*

The!outcome!of!the!internally!driven!and!constructivist!approach!were!defining!theoretical!

underpinnings!of!the!evaluation!practice!that!developed!for!the!Balaklava!project.!To!gain!insight!

into!the!emergent!evaluation!approach!for!the!Balaklava!application,!and!to!analyse!the!theoretical!

underpinnings!of!the!evaluation!practice,!I!examined!the!approach!in!relation!to!contemporary!

theories!within!the!evaluation!field.!Via!a!structured!reflective!exercise,!I!examined!where!the!READ!

application!for!Balaklava!would!be!positioned!within!the!field.!The!findings!from!this!exercise!focus!

only!on!the!approach!of!the!Balaklava!tailored!methodology,!not!for!the!general!READ!model.!!

First,!I!established!that!along!the!spectrum!of!participatory*oriented!approaches!described!in!

Chapter!2,!the!evaluation!approach!for!the!Balaklava!project!reflected!a!stakeholder*oriented!and!

utilitarian!approach,!rather!than!an!empowerment!example!(although!capacity!building!and!

empowerment!outcomes!did!result).!This!was!based!on!the!premise!that!while!the!activities!were!

participatory!and!interested!stakeholders!did!participate,!overall!the!hands*on!learning!and!capacity!

building!dimensions!of!evaluation!practice!were!embraced!to!varying!levels!by!stakeholders!(less!

than!what!I!had!initially!envisaged).!In!many!cases,!input!was!facilitated!through!consultation!rather!

than!active/direct!participation,!which!meant!the!process!remained!centralised!through!Alphaville,!

and!me!as!facilitator/lead!evaluator.!What!was!demonstrated!by!the!approach!I!adopted!was!a!

prioritisation!of!stakeholder!relationships,!brokerage!and!shared!learning.!This!dimension!of!the!

emergent!evaluation!practice!aligned!with!some!of!feminist!evaluation!theory,!such!as!by!Belenky,!

Clinchy,!Goldberger,!and!Tarule!(1986),!in!Women’s%Ways%of%Knowing,!where!meaningful!

connections!with!program!stakeholders!are!developed!in!order!to!better!understand!the!reality!of!

their!program!experiences,!as!well!as!the!impacts!of!evaluation!on!stakeholders,!delivery!partners!

and!community!participants!(Sielbeck*Bowen,!2002).!

Of!importance!for!defining!the!philosophy!of!practice!of!the!READ!approach!for!the!Balaklava!project!

was!the!constructivist!stance!outlined!in!Section!6.2.2.!In!attempting!to!understand!how!the!

application!related!to!the!field!of!evaluation,!I!referred!to!the!evaluation!tree!by!Christie!and!Alkin,!a!

diagrammatic!schema!that!maps!the!‘roots’!and!‘branches’!of!evaluation!theory!(Christie!&!Alkin,!

2008).!The!evaluation!tree!proposed!by!Christie!and!Alkin!is!included!as!Figure!6.2.!!

!

!

!


! 150!

!

!

Figure!6.2:!Evaluation!theory!tree!by!Christie!and!Alkin!(version!3)!!

(Source:!Christie!&!Alkin,!2008,!p.133)!

!

In!the!illustration,!Christie!and!Alkin!have!plotted!seminal!evaluation!theorists!along!three!respective!

branches!according!to!the!primary!emphasis!of!the!models!included.!While!all!evaluation!theorists!

are!interested!in!the!process!of!valuing,!the!methods!used!and!their!utility,!approaches!do!differ!in!

terms!of!the!particular!emphasis!placed!on!one!or!another!of!these!dimensions.!The!diagram!also!

defines!the!two!main!theoretical!roots!of!contemporary!evaluation.!The!first!is!social!accountability!

and!fiscal!control,!and!the!second!is!social!inquiry.!

My!analysis!during!this!reflective!practice!exercise!revealed!that!the!emergent!READ!approach!for!

Balaklava!represented!a!theoretical!orientation!from!the!values!branch!of!Christie!and!Alkin's!(2008)!

theory!tree,!aligned!with!theorists!such!as!Jennifer!Greene,!who!advocates!a!value*engaged!

approach!for!framing!evaluation,!with!an!emphasis!on!stakeholder!values!(Greene,!2005).!While!the!

values!branch!originated!in!response!to!theorists!such!as!Scriven,!who!argue!that!the!process!of!

attributing!value!is!an!essential!component!of!an!evaluator’s!work,!this!branch!also!includes!

constructivist!approaches.!Christie!and!Alkin!(2008,!p.134)!states:!‘this!branch!also!includes!the!work!

of!those!interested!in!social!justice!in!evaluation,!as!well!as!those!who!espouse!the!philosophy!of!

subjectivity,!that!is,!the!claim!that!there!is!no!one!objective!reality.!Here,!theorists!are!also!

concerned!with!valuing,!but!the!emphasis!is!on!the!extent!to!which!values!shape!the!evaluation’.!!

Use branch revisions

In its most recent incarnation, the Use branch underwent onlyminor revision and still closely resembles the Tree presented inChapter 26 of the Evaluation Roots book. The slight modificationsinclude repositioning Joseph Wholey’s work (2004) to reflect anemphasis towards the Methods branch. Also, we deleted MalcolmProvus from the branch because much of his work has hadrelatively little influence on other perspectives on the branch.

Methods branch revisions

The Methods branch received more substantial changes. RalphTyler was repositioned to a sub-branch to reflect that while histheoretic point of view was in fact heavily methods related(objectives based evaluation), he was not a theoretical predecessorof those further up on the branch. His original positioning wasintended to reflect his influence on the field of educationalevaluation (which was very significant), but upon furtherreflection we concluded that his overall influence on the methodsbranch specifically was less than his original position implied.Thus, we placed him on a small sub-branch near the base of thebranch.

Donald Campbell continues to be the heart and soul of thisbranch both in the way that he directly influenced other theoristswith his work on experiments, quasi-experiments, and validity, aswell as the way in which his views provided a basis for counterpoint, particularly with respect to Lee Cronbach’s views.

We have also deleted Edward Suchman from the branch. Werecognize the important influence of Suchman’s writing onevaluation—particularly in positioning Campbell’s work promi-nently in the evaluation discourse. Nonetheless, despite hishistorical significance, it did not seem appropriate to continueto include him on the branch because he himself did not offer thefield a specific evaluation model. Rather, he promoted Campbell’swork as the most effective approach for conducting evaluationstudies to measure program impact. Peter Rossi continues to be

shown on the main portion of the branch as an influence on bothCarol Weiss and, more particularly, on Huey Chen.

A significant change on this branch was moving Tom Cook whopreviously had been positioned in a sub-branch flowing fromCampbell. He is now positioned on the main branch, and his sub-branch was deleted. A new sub-branch was created for Bob Boruch.Boruch’s sub-branch comes directly out of Campbell and Cook. Thisplacement reflects the steadfast influence of Campbell’s rando-mized control trial (RCT) work on Boruch. The branch is offset on asub-branch to emphasize his departure from both Cook andCampbell in his lack of regard for quasi-experiments. This positionalso offsets Boruch from the others on the methods branch, all ofwhom have embraced quasi-experiments as an acceptable methodfor studying causality, although everyone on the branch wouldagree that the experiment is ideal if context and conditions permit.

Another major change on this branch is the addition of GaryHenry, Mel Mark and George Julnes. In the Roots book we said:

(T)heories included were able to be classified onto a singlebranch of the tree. . . Whether some theories were not included dueto their comprehensiveness or our conceptual inability is unclear.A particular example comes to mind: the work of Mark, Henry, andJulnes (2000). These authors view social betterment as theultimate objective of evaluation and present a point of viewgrounded in what they refer to as a ‘common sense realistphilosophy.’. . . The very diversified nature of this perspective,while a great strength in presenting an understanding ofevaluation, precludes its inclusion on the tree (pp. 58–59).

We admit that in this instance our conceptual analysis wasincomplete. Our views in determining exclusion were basedheavily on the Mark et al. (2000) book. However, in reflectingfurther on the writings of these authors, we were further struck bythe views presented in the ‘‘realist evaluation’’ monograph in NewDirections for Evaluation (Henry, Julnes, & Mark, 1998). Based upona more in-depth reading of this text, we have placed Henry, Mark,and Julnes on the methods section branch, leaning toward thepost-positivist valuing branch. The leaning highlights Henry andMark’s collaboration with George Julnes to create ‘‘a new theory

Fig. 2.

C.A. Christie, M.C. Alkin / Studies in Educational Evaluation 34 (2008) 131–135 133


! 151!

Specifically,!it!can!be!argued!that!the!Balaklava*tailored!approach!sits!in!the!valuing!branch!as!a!

post*positivist!and!constructivist!paradigm!framework,!towards!the!right!fork!where!the!valuing!

theorists!are!located.!While!the!findings!from!reflective!practice!also!identified!the!READ!application!

had!a!secondary!relationship!to!the!‘use’!branch!of!the!tree,!in!that!the!application!was!concerned!

with!utility,!legacy!and!the!end*use!purposing!of!evaluation!practice!for!the!benefit!of!stakeholders,!

overall,!it!was!the!social!enquiry!orientation!of!the!values!branch!that!best!matched!the!practice.!

The*model*of*practice*arising*from*the*READ*methodology*

An!alternative!review!technique!I!used!to!understand!and!analyse!the!evaluation!practice!was!to!

consider!the!different!domains!of!activity!resulting!from!the!READ!methodology!in!a!generalised!

way.!This!section!presents!the!simplified!summary!table!I!created!to!capture!the!transferable!

elements!of!the!model!of!practice!(see!Table!6.1).!While!it!is!based!on!the!activities!and!observations!

of!the!Balaklava!trial!(Chapter!5),!I!attempted!to!step!back,!and!create!a!snapshot!of!the!sources!and!

flow!of!information,!and!link!these!to!the!outcomes!and!relationships!across!the!four!READ!

strategies.!The!purpose!of!generating!the!summary!table!was!to!present!a!consolidated!record!of!

the!READ!model!and!evaluation!practice!arising!from!the!Balaklava!trial,!that!could!be!easily!

communicated!back!to!partners.!Combined!with!the!analysis!of!the!theoretical!orientation!of!the!

READ!application!in!Balaklava!(discussed!above),!the!tabular!summary!provides!insight!into!the!

emergent!evaluation!model!that!evolved!from!the!original!READ!concept.!It!was!also!a!formative!

process!for!the!READ!product!development!presented!later!in!the!Chapter!in!Section!6.3.!

(Please!note:!in!Table!6.1,!I!have!used!categorisations!for!data!collection,!analysis!techniques,!and!

outputs.!However,!in!practice,!several!of!these!processes!and/or!methods!overlapped,!and!the!

system!as!a!whole!was!iterative!and!not!sequential!as!is!implied!by!the!table.)

AN#INTEGRATED#APPROACH#TO#EVALUATION# ELLISE#BARKLEY#

# 152#

#Table#6.1:#Summary#of#READ#model#processes#and#outputs#

STRATEGIES APPROACH DATA SOURCES ANALYSIS & INTEGRATION

OUTPUTS

REFLECTION Individual & collective reflective practice makes tacit knowledge explicit

•! Voluntary, inclusive, safe •! Culture of reflective practice •! Individual and collective,

regular and reflexive •! Self-guided, semi-structured

or facilitated •! Suit methods to participants

Journals Video diaries Interviews Oral history Digital stories Field notes Documentation

Blogs, websites Communications Critical buddies Observation News media Social media Art methods

Lead by partner needs, desired outputs and level of participation Mix of evaluative, reflective, creative and critical analysis processes

Examples include: Thematic analysis Textual analysis Comparative analysis Peer review Reflective analysis Evaluative reasoning Rubrics/performance criteria Statistical/quantitative analysis Creative methods Practice-led methods Story-based methods (Most Significant Change technique)

Formative and summative outputs generated across READ implementation

Mix of evaluative and non-evaluative content

Examples include:

Raw data Management resources Evaluation reports Publications Research Creative content Infographics Artworks Promotional assets Archives Project stories Pending/future outputs (outputs not yet completed or known) (Access and shared: copyright may vary across partners)

•! Participatory and stakeholder engagement approaches

•! Formative and summative •! Unique project goals •! Skills development •! Key evaluation questions •! Access to all READ data

Scoping Questionnaires Feedback forms Interviews Vox pops Reflections Statistics

Documentation Research Field notes Observation Communications Media Focus groups

ANALYSIS Scholarship & research enquiry to generate original knowledge for broader learning

•! Network of engaged researchers and partners: Community of practice

•! Priority on publication and dissemination

•! Practice-led research •! Promotes broader learning

Scholarship Forums Seminars Media Research Critical reviews

Practice-led methods Commentary Presentations Contextual scans Publications

DOCUMENTATION Collection of written, visual and audio records, artefacts, & stories

•! Multi-platform •! Mixed sources •! Links with creative practice •! Legacy and archival elements •! Digital content archiving for

long-term storage •! Rich ephemeral data

Photography Film/video Publications Artworks Reports Websites Social media Webisodes

•! Databases/ repositories Sound recordings Communications Archival artefacts Writing Digital stories

Real-time and short-term outputs to support project decision-making, improvement (creative practice and organisational), accountability, partner updates/communications

Medium to long-term outputs to support accountability, continual improvement, broader learning, legacy

EVALUATION Key evaluation questions addressed using evaluative instruments and reasoning

#


# 153#

6.2.5! Meta)evaluation0

While#findings#from#the#reflective#practice#using#the#metaKevaluation#and#SWOT#tools#(see#Chapter#

3)#were#applied#across#many#of#the#themes#presented#in#this#chapter,#this#section#provides#a#focused#

metaKevaluation#discussion#aligned#to#Davidson’s#criteria#(2005)#of#validity,#utility,#conduct,#

credibility#and#cost.#Informed#by#the#findings,#I#was#able#to#identify#and#assess#the#constraints#and#

opportunities#of#the#evaluation#practice,#for#continuous#improvement#for#future#applications.#

Validity(of(READ(findings((

The#internal#position#of#the#READ#delivery#and#my#dual#role#of#READ#facilitator/manager#were#

identified#as#potential#conflicts#of#interest#that#could#have#implications#for#the#validity#of#the#READ#

findings.#It#could#be#argued#that#READ#results#may#be#subject#to#my#personal#and#professional#bias#

given#that#‘the#project#manager#is#seen#as#ultimately#responsible#for#the#project’s#success#in#

achieving#objectives’#(Davidson#and#Rowe,#2009,#p.573).#To#ensure#validity#of#the#findings#measures#

were#incorporated#into#the#READ#methodology#so#that#evaluation#practice#would#be#rigorous#and#

credible,#such#as#using#triangulation#techniques#to#corroborate#data#and#conclusions.#

During#the#design#and#implementation#of#the#tailored#READ#framework,#quality#and#rigour#were#

addressed#by#ensuring#the#authenticity#of#the#data#collected#and#the#trustworthiness#of#the#data#

analysis#processes#used.#Within#each#of#the#READ#strategies,#established#research#and#evaluation#

methods#were#selected#based#on#their#appropriateness#and#reliability.#To#ensure#adequate#expertise#

existed#amongst#the#collective,#participating#stakeholders#received#skills#training#where#necessary#by#

a#member#of#the#team#with#adequate#expertise#in#that#method.#For#example,#artists#involved#in#

documentation#were#responsible#for#conducting#the#semiKguided#reflections,#vox#pop#recordings#and#

interviews#and#this#relied#on#them#employing#question#design#and#delivery#techniques#that#would#

ensure#reliable#and#authentic#responses#from#participants.#Before#documentation#of#key#events,#the#

documentation#approach#and#questions#were#critically#reviewed#with#another#team#member#and#

refined.#For#several#artists,#there#was#already#a#high#level#of#expertise#in#documentation#and#

interview#methods,#given#their#experience#in#investigative#journalism#and#documentary#artKmaking.#

This#example#also#demonstrated#the#way#in#which#the#READ#methodology#was#designed#to#draw#on#

the#existing#strengths#of#the#participating#stakeholders,#while#delivering#appropriate#methods#for#the#

data#collection#needs#of#the#project.#

Given#that#the#READ#application#largely#relied#on#interpretative#processes,#collective#review#of#data#

during#analysis#with#other#participants#was#undertaken#so#conclusions#were#not#solely#my#own.#

Different#analysis#techniques#were#experimented#with,#including#evaluative#reasoning#based#on#a#


# 154#

performance#rubric#to#guide#criteria#for#claims#and#findings#on#the#artistic#merit#of#the#Balaklava#

creative#works#(see#Chapter#5,#Section#5.4.3).#In#complement#to#the#collective#reflective#practice#

being#undertaken,#one#measure#I#implemented#to#regulate#my#own#evaluation#practice#was#to#utilise#

the#forensic#reflective#practice#as#an#evaluator’s#tool,#and#this#involved#video#reflections#and#keeping#

a#log#of#practice#where#reflection#within#and#on#practice#was#used#to#explore#issues#arising,#identify#

conflicts#of#interests#and#examine#assumptions#during#the#READ#application.#I#also#coordinated#the#

triangulation#of#data#involving#multiple#participant#reviews,#to#verify#the#validity#of#data#collected#

from#different#sources.##

For#the#scale#of#the#Balaklava#project,#no#external#review#of#the#READ#process#was#deemed#

necessary.#This#was#a#combination#of#the#scale#of#the#project#and#the#identified#evaluation#outcomes#

needed#by#partners#for#the#Balaklava#project.#However,#for#the#summative#program#evaluation,#the#

strategy#to#engage#an#evaluation#specialist#external#to#the#project#to#review#the#methodology#and#

findings#is#being#considered#as#a#means#to#overcome#the#vulnerability#that#the#internal#positioning#of#

the#READ#delivery#places#on#validity.#On#reflection,#the#evaluative#conclusions#were#relatively#

straightforward#to#establish#for#the#Balaklava#project,#as#the#multiple#perspectives#represented#in#

the#data#collected#led#to#convergent#conclusions.#A#different#level#of#rigour#and#process#to#ensure#

validity#of#findings#would#have#been#needed#if#divergent#conclusions#had#arisen,#or#if#data#required#

synthesis#from#a#more#complex#range#of#methods#covering#diverse#program#areas#and#goals#(such#as#

will#be#the#case#in#the#summative#evaluation#needs#of#the#Nuclear#Futures#program#in#2017).#

For#example,#one#area#for#improvement#I#identified#from#the#metaKevaluation#was#the#further#

development#of#techniques#for#data#analysis,#to#make#the#collective#processes#of#review#more#robust#

in#future#during#the#Nuclear#Futures#summative#evaluation.#For#the#endKofKprogram#evaluation,#the#

level#of#sophistication#for#these#processes#will#need#to#increase#compared#with#what#was#

implemented#at#the#Balaklava#project#level.#The#summative#evaluation#may#also#need#to#revisit#the#

findings#established#at#the#end#of#individual#projects,#to#take#into#account#any#later#successes#or#

impacts#generated#from#the#community#arts#(such#as#subsequent#awards,#reviews#and#exhibitions).#I#

anticipate#that#reKevaluating#the#arts#practice#across#a#longer#time#period#will#be#important#for#

contributing#to#a#richer#assessment#of#the#merit#and#vibrancy#attributed#to#artwork#generated#in#

Balaklava#beyond#time#constraints#of#this#research.##

In#approaching#future#applications#of#the#READ#model,#there#are#a#number#of#‘next#steps’#that#I#

identified#in#the#reflective#practice#that#will#be#of#relevance.#For#example,#there#may#need#to#be#an#

increased#use#of#rubrics#and#establishment#of#thresholds#for#practice#and#outcomes#to#identify#

minimum#standards.#Indices#may#also#need#to#be#developed#that#speak#to#impact#categories#across#


# 155#

the#project#domains#within#the#program#evaluation,#to#enable#cohesive#and#summative#collective#

analysis.#This#may#also#involve#investigating#and#assessing#the#applicability#of#the#latest#international#

standards#or#cultural#indicators#that#may#be#developed#in#the#future,#and#examining#the#role#of#

technology#in#future#applications,#as#well#as#the#latest#techniques#for#presenting#impact#case#studies.#

It#is#worth#noting#that#analysis#of#the#validity#of#READ#findings#was#not#straightforward,#and#required#

different#credentials#be#used#given#the#mix#of#learning#and#appraisal#methods#employed#across#the#

four#READ#strategies.#While#it#is#beyond#the#scope#of#the#research#to#cover#this#point#in#detail,#it#is#

acknowledged#that#the#READ#methodology#was#a#combination#of#evaluative#(the#process#of#valuing),#

researchKoriented#(the#process#of#enquiry)#and#creative#(the#process#of#artKmaking)#methods,#and#

there#is#a#difference#between#what#will#count#as#valid#for#each#of#these#methods.#For#example,#the#

parts#of#the#READ#methodology#dedicated#to#determining#the#quality,#value#or#significance#of#

community#arts#practice#demand#quite#specific#methodologies#to#ensure#validity#(that#extend#

beyond#social#enquiry#methods#to#include#evaluative#reasoning),#as#per#definitions#of#evaluation#by#

Davidson#(2005)#and#Scriven#(2011).#In#the#READ#application,#drawing#valid#evaluative#conclusions#

relied#on:#ensuring#credible#data#was#collected;#inclusion#of#the#multiple#perspectives#of#partners;#

the#utilisation#of#multiple#data#sources;#and#applying#criteria#for#merit#that#reflected#partner#values.##

Utility(

As#detailed#in#Chapter#5,#the#READ#methodology#generated#written#and#video#reflections,#creative#

documentation,#raw#content,#reports,#research#outputs,#artworks,#management#resources,#archives#

and#evaluation#findings#(see#Table#6.1).#These#READ#outputs#(both#formative#and#summative)#

delivered#shortK#and#longKterm#value#to#partners,#and#led#to#improvements#in#arts#practice,#program#

management#and#partnership#brokerage,#plus#broader#dissemination#of#original#knowledge#arising#

from#practice#via#academic#and#industry#forums.#The#outputs#have#already#been#used#in#diverse#

ways,#beyond#the#project#scope,#such#as#in#the#case#of#several#original#documentary#videos#and#

artworks#that#evolved#from#the#data#collection,#which#have#since#been#distributed#to#wider#

audiences.#

While#some#theorists#such#as#Scriven#(1991)#argue#the#limitations#of#using#endKutility#as#a#measure#

for#metaKevaluation#(because#the#actual#use#of#findings#or#outputs#is#beyond#the#control#of#the#

evaluation#team),#the#analysis#of#utility#outcomes#offered#valuable#insight#into#where#the#READ#

deliverables#were#shared#(in#terms#of#which#outputs#were#shared#with#whom#and#what#organisation,#

and#why,#and#how#outputs/findings#were#used,#if#at#all).#The#findings#of#this#analysis#evidenced#that#

the#mix#of#learning#and#evaluation#outputs#delivered#by#the#tailored#framework#were#sufficient#for#


# 156#

meeting#the#identified#needs#of#the#stakeholders,#and#exceeded#expectations#and#requirements.#I#

attribute#the#effectiveness#of#the#READ#approach#in#this#domain#to#be#the#result#of#the#tailored#

methodology#designed#and#also#the#open#and#ongoing#communication#with#stakeholders#about#their#

endKuse#preferences#throughout#the#duration#of#the#project.#SielbeckKBowen#et#al.#(2002)#argue#that#

the#construction#and#generation#of#evaluation#outputs#is#an#interactive#process#that#occurs#within#

relationships,#and#from#my#perspective#as#READ#facilitator,#this#rang#true#for#the#partnerKoriented#

community#arts#initiative#in#Balaklava.##

One#of#the#surprising#results#arising#was#the#diverse#and#longKterm#utility#value#that#was#delivered#

through#creative#documentation.#The#creative#documentation#outputs#included#raw#data#and#

creative#content,#photographs,#video,#produced#assets#(such#as#online#material,#promotional#video,#

photoKstory#book)#and#creative#works.#Using#multiKart#methods,#the#creative#documentation#was#

effectively#used#for#engagement,#data#collection,#analysis,#communications#of#findings,#promotions,#

artKmaking#and#for#spinKoff#purposes#(beyond#the#Balaklava#project).#This#is#discussed#further#in#

Section#6.2.6.#

At#the#organisational#level,#one#outstanding#example#was#the#demonstrated#utility#of#the#READ#

generated#findings#by#Alphaville#and#the#Nuclear#Futures#team,#and#this#use#of#outputs#significantly#

impacted#creative#practice.#READ#data,#findings#and#outputs#played#a#seminal#role#in#decisionK

making#and#creative#development#for#future#showcases#in#a#number#of#ways.#One#outcome#specific#

to#the#work#generated#during#the#Balaklava#project#was#that#minor#modifications#were#made#to#the#

projection#installation#10#Minutes#to#Midnight#in#response#to#audience#feedback.#The#findings#also#

influenced#the#development#of#new#artworks#created#for#future#showcases,#such#as#the#creative#

approach#to#the#companion#projection#installation#Ngurini#(Searching)#that#was#developed#for#a#later#

exhibition.#For#Alphaville#as#host#organisation,#READ#findings#(including#the#resources#generated)#

were#used#for#organisational#learning,#partnership#development,#program#management,#funding#

proposals#and#strategic#planning.#Examples#of#utility#outcomes#for#community#partners#and#other#

stakeholders#are#included#in#Chapter#5.#

Conduct(

The#conduct#for#the#evaluation#practice#aimed#to#be#appropriate#and#inclusive,#to#ensure#relevance#

for#all#partners#and#so#that#ownership#of#the#process#was#shared#amongst#interested,#participating#

partners.#For#all#activities,#consideration#was#given#to#the#ethical#and#legal#aspects#of#practice,#and#all#

participating#stakeholders#adhered#to#ethical#conduct#guidelines#for#evaluation#practice#that#was#

overseen#by#myself#as#facilitator#(see#Australasian#Evaluation#Society,#2010,#for#code#of#conduct).#


# 157#

Implementation#of#ethical#conduct#was#supported#by#industry#and#research#protocols,#such#as#

those#established#for#the#data#collection#and#documentation#with#persons#under#18#years#of#age#

(RIPP:#Article#7,#p.15#is#an#example#of#a#permission#form#that#required#parental#consent).##

For#some#cohorts#participating#in#the#READ#application,#their#activities#required#adherence#to#

specific#disciplineKbased#codes#of#conduct,#such#as#in#the#case#of#the#community#of#practice#

established#through#the#Analysis#strategy,#where#researchers#conformed#to#academic#policies#for#the#

ethical#conduct#of#research.#The#organisational#culture#of#Alphaville#also#guided#conduct#aligned#with#

principles#for#community#arts#and#cultural#development.#Overall#this#was#a#relatively#easy#process#to#

manage#as#facilitator,#as#the#creative#team#personnel#and#the#majority#of#partners#were#accustomed#

to#working#with#codes#of#conduct#and#adherence#to#these#was#understood#to#be#a#professional#duty.##

In#the#leadership#role#as#READ#facilitator,#my#conduct#was#important#as#a#I#was#a#role#model#for#

ethical#behaviour#and#responsible#for#facilitating#engagement#with#our#multiple#partners#in#a#

culturally#sensitive#and#just#manner.#Clements#comments:#

In#reality,#the#evaluator#is#awkwardly#placed#between#funder,#manager,#animateur#and#participant#as#well#as#methodologically#ensnared#by#conflicting#demands.#This#requires#a#keen#political#awareness#and#reflexivity#in#order#to#prioritise,#encourage#and#facilitate#transparency#as#well#as#help#enfranchise#participants#by#supporting#their#cultural#and#educational#rights,#thereby#enabling#greater#democracy.#(Clements,#2007,#p.334)#

As#facilitator,#one#of#the#potential#issues#I#identified#was#the#power#imbalance#between#myself#and#

other#stakeholders,#given#my#authority#in#governing#the#process#as#facilitator.#This#was#true#as#

facilitator#and#as#researcher,#as#I#had#final#authorship#of#the#account#of#the#fieldwork,#for#the#final#

design#decisions#for#the#application,#and#was#responsible#for#how#the#evaluation#practice#and#final#

READ#product#were#represented.#Fonow#and#Cook#(1991,#p.9)#point#out#that#the#‘emphasis#on#

collaboration#between#researcher#and#researched#masks#the#real#power#of#the#researcher,#who#has#

much#greater#control#over#the#research#process#and#product’.#Reflective#practice#as#an#evaluator’s#

tool#was#used#to#explore#my#process#of#integrating#partner#input,#to#assess#the#authenticity#and#

priority#of#partners’#input,#and#to#gain#insight#into#the#levels#of#agency#I#was#assuming#at#different#

stages#of#the#READ#application#(in#the#field#and#in#writing#up#the#research).##

Credibility((

In#assessing#the#credibility#of#the#READ#application,#I#examined#the#three#elements#stated#by#

Davidson#(2005,#p.211)#to#be#most#important#for#ensuring#credibility:#‘(a)#familiarity#with#the#

context;#(b)#independence,#impartiality,#and/or#lack#of#conflict#of#interest;#and#(c)#expertise#in#

evaluation#and#in#the#subject#matter#under#investigation’.#For#the#Nuclear#Futures#CCPI#context,#I#

interpreted#credibility#to#mean#gaining#acceptance#by#and#maintaining#relevance#to#community#

#


# 158#

partners#during#the#exchanges#and#outputs#produced#by#the#READ#approach.#Further,#we#aimed#for#

the#approach#and#its#outcomes#to#be#deemed#acceptable#and#sound#by#peers#and#other#community#

arts#practitioners.#I#determined#that#there#were#a#number#of#factors#that#supported#the#credibility#of#

the#READ#methodology#and#implementation.#These#included:#

•! The#participatory#approach#was#inclusive#of#various#cultural#and#knowledge#paradigms#to#

reflect#partner#values,#capacity#and#needs#(important#for#the#community#partnerships#

context).#

•! READ#processes#and#findings#were#based#on#the#different#and#combined#expertise#of#a#range#

of#individuals#across#professional#and#community#roles#participating#at#the#site#of#practice.#

•! The#guiding#principles#and#priorities#for#both#internal#and#broader#learning#during#the#READ#

application#were#partnerKoriented,#based#on#stakeholder#input,#inclusive#of#all#partners.#

•! Stakeholder#expertise,#input#and#participation#was#overtly#valued,#as#well#as#critically#

reviewed.#

There#was#also#evidence#that#the#READ#application#being#undertaken#as#academic#research#added#

credibility#for#the#trial#and#ultimately#the#findings,#from#the#perspective#of#several#stakeholders#

(based#on#feedback#and#interviews).#While#the#READ#research#was#not#funded#by#the#Australia#

Council#for#the#Arts,#the#close#association#with#Nuclear#Futures#partners#and#Alphaville#through#the#

program,#also#provided#credibility#for#the#READ#application#for#the#Balaklava#project.#Further,#my#

known#previous#experience#in#evaluation#and#program#management,#combined#with#my#familiarity#

with#the#project#context#and#subject#matter,#afforded#credibility#for#me#as#READ#facilitator.#The#

known#expertise#of#Alphaville#personnel#(who#were#handsKon)#also#added#significant#credibility.#

Cost(and(time(

The#metaKevaluation#criteria#of#cost#and#time#were#the#most#difficult#to#analyse,#as#the#READ#

facilitation#had#been#undertaken#as#academic#research#and#did#not#reflect#the#resourcing#or#terms#of#

reference#of#a#standard#industry#application.#For#example,#the#time#demands#for#conducting#the#

READ#application#as#academic#research#were#much#greater#than#if#the#application#had#been#

administered#by#Alphaville#as#an#industryKbased#evaluation#practice.#The#academic#conventions#also#

influenced#my#management#style#and#the#level#of#focus#directed#to#the#theoretical#aspects#of#

practice.#While#this#enhanced#the#evaluation#practice,#this#was#beyond#the#scope#of#my#usual#

practitioner#approach.#In#order#to#maximise#innovation#and#encourage#experimentation,#there#was#a#

deliberate#flexibility#for#the#scope#and#scale#of#the#READ#activities,#which#had#time#and#cost#

implications,#and#these#managerial#aspects#of#the#READ#application#were#not#addressed,#as#they#

would#have#been#outside#an#academic#context.#


# 159#

The#process#of#developing#and#testing#the#tailored#READ#framework#beyond#the#conceptual#

representation#of#READ#formulated#for#Alphaville’s#funding#applications#took#concerted#effort.#The#

Balaklava#project#was#the#first#of#the#Nuclear#Futures#projects#to#come#under#the#READ#lens,#so#it#was#

not#a#straightforward#delivery#of#an#established#approach.#As#this#dissertation#documents,#my#

research#has#involved#devising#both#a#tailored#methodological#framework#for#the#Balaklava#project#

and#a#more#generic#model#of#practice,#and#these#each#involved#extensive#consultation#and#iterative#

development.#Such#processes#were#often#concurrent#with#the#implementation#itself.#Lastly,#following#

the#Balaklava#project,#it#is#evident#from#my#experience#of#facilitating#the#READ#approach#in#other#

Nuclear#Futures#projects#that#the#evaluation#approach#at#this#site#of#practice#is#so#much#more#

advanced#as#a#result#of#the#knowledge#gained#and#systems#created#through#the#Balaklava#application.#

This#will#reduce#the#time#and#effort#of#applying#the#READ#model#of#practice#for#subsequent#projects.#

While#no#conclusion#was#possible#on#the#considerations#of#cost#and#time,#some#general#observations#

were#made:#

•! The#main#financial#outlay#for#the#application#was#for#the#salaries#of#the#creative#team,#

involved#in#the#delivery#of#technical#aspects#of#documentation#(covered#by#host#organisation#

Alphaville).#This#paid#work#overlapped#significantly#with#their#creative#residency#activities#

and#was#often#absorbed#within#the#creative#development#artist#fee#and#project#budget.#My#

role#was#not#remunerated#by#the#Balaklava#project#budget,#as#the#research#was#resourced#

separately.#Some#mentoring,#training#and#resources#for#team#members#was#paid#for#also.#

•! My#role#as#facilitator#demanded#the#most#time#out#of#all#of#the#READ#team#roles.#

•! In#most#cases,#involvement#by#participating#stakeholders#was#unpaid,#inKkind#input.#The#

voluntary#contributions#from#stakeholders#were#usually#absorbed#by#their#paid#role#with#

partner#organisations.#In#some#instances,#community#partners#were#paid#fees#or#had#their#

expenses#covered#for#participation#as#their#input#was#not#covered#by#a#funded#organisation.#

However,#offers#to#do#so#were#often#not#accepted#by#the#community#partners.#For#

community#partners,#READ#was#seen#as#an#extension#of#existing#community#arts#activities,#

rather#than#being#viewed#as#completely#extraneous.#From#informal#feedback#it#appears#

partners#were#not#focused#on#remuneration#as#a#motivation#to#participate.#

•! An#important#aspect#of#managing#partner#contributions#was#ensuring#equity#amongst#

partners#in#respect#who#was#participating#via#paid#versus#inKkind#support.#Given#the#high#

level#of#inKkind#input,#the#READ#activities#were#managed#to#generate#mutually#beneficial#

outcomes#for#the#inKkind#stakeholders.#

•! No#equipment#needed#to#be#hired#or#purchased#for#data#collection#or#analysis,#as#video#and#


# 160#

photography#equipment#was#already#available#for#the#community#arts#projects#already#

underway.#InKkind#contributions#by#institutional#partners#covered#some#nonKsalary#costs#

such#as#printing#of#evaluation#instruments.#

•! NonKsalary#costs#and#equipment#needs#were#mainly#associated#with#the#storage#and#

management#of#the#data#collected,#which#included#the#need#for#at#least#10#large#hard#drives#

for#storage#and#to#create#a#robust#backup#system.#An#enormous#volume#of#data#was#

generated#from#the#documentation#process.#

•! Creative#artists,#arts#workers,#community#partners#and#institutional#partners#considered#the#

READ#processes#and#outputs#to#be#useful,#yet#it#was#also#considered#that#the#data#collection#

and#processing#was#timeKconsuming.##

•! Feedback#from#the#artists#indicated#that#is#was#often#challenging#for#them#to#undertake#the#

dual#roles#of#artist#in#residence#and#READ#documenter.#In#some#cases,#the#dual#briefs#were#

very#demanding.#The#role#of#documenter#was#shared#by#several#people#to#help#appease#this.#

•! There#were#many#timeKconsuming#aspects#of#the#READ#application,#such#as#data#

management#and#storage,#plus#the#review,#synthesis#and#editing#of#the#data#collected.#

•! The#strategy#to#work#with#academics#through#the#community#of#practice#relied#on#longKterm#

processes#and#relationships,#which#extended#well#beyond#the#life#of#the#Balaklava#project.#

This#arrangement#has#ongoing#time#and#resource#implications#for#the#project#partners#and#

for#host#organisation#Alphaville.#Community#partners#and#stakeholders#working#with#the#

academics#or#participating#in#the#community#of#practice#are#doing#so#in#a#voluntary#capacity.#

Thus,#this#may#not#be#a#suitable#arrangement#for#shortKterm#community#arts#activities.##

•! LongKterm#strategies#and#methods#require#ongoing#partnership#brokerage,#a#resource#

demand#for#the#host#organisation,#and#will#require#sustained#community#engagement#and#

feedback#cycles#to#ensure#information#is#disseminated#back#to#communities.#

•! Another#longKterm#task#is#overseeing#and#resolving#coKauthorship#issues,#so#that#all#those#

contributing#to#the#READ#outputs#–#whether#it#be#directly#through#artKmaking#or#the#analysis#

process#–#are#appropriately#acknowledged.##

•! For#public#dissemination#of#READ#outputs,#it#is#not#known#what#level#of#vigilance#will#be#

required#by#Alphaville#to#periodically#the#check#the#materials#in#institutions#or#

collections/repositories.##

•! Even#after#the#project#had#finished#(and#partner#closure#for#the#project#was#undertaken),#

evaluation#and#READ#data#that#required#processing#and#storage#continued#to#emerge.#


# 161#

Constraints(and(strengths(of(the(READ(model(and(evaluation(practice(

The#above#discussion#of#time#and#cost#implications#illuminates#some#of#the#constraints#of#the#

methodology.#One#consequence#of#the#wide#use#of#data#collection#methods#was#the#implications#for#

storing#and#managing#data,#not#merely#for#READ#analysis#processes#but#for#longerKterm#accessibility.#

The#four#READ#strategies#produced#very#large#quantities#of#data,#which#meant#a#system#for#

organising#(cataloguing)#and#safeguarding#storage#was#required.#There#was#also#systematic#

documentation#of#authorship#required#for#the#creative#and#research#content.#These#types#of#

associated#tasks#were#challenging#aspects#of#the#facilitation#role,#and#certainly#had#time#and#cost#

implications.#On#the#flipside,#the#archive#of#information#and#material#is#extremely#valuable#for#

communities#and#participating#organisations#as#they#are#assured#ongoing#access.#Copies#of#the#

digital#content#were#shared#with#community#partners#(based#on#community#selection/requests)#to#

ensure#they#retain#ownership#and/or#access#to#the#material.#

One#of#the#broader#constraints#of#the#READ#model#of#practice#identified#is#that#READ#will#not#offer#an#

approach#suitable#for#all#contexts#and#evaluation#needs,#and#will#have#limitations#for#application#

depending#on#the#capacity#of#the#host#organisation#to#drive#the#concurrent#READ#strategies,#partner#

interests,#and#the#resources#and#timeframes#allocated#to#the#READ#process.#It#is#worth#reiterating#

that#the#READ#model,#as#with#any#guiding#evaluation#framework,#will#not#guarantee#positive#

appraisal#outcomes,#participation,#inclusion#or#the#delivery#of#socially#and#politically#just#means#for#

data#collection,#analysis#and#management.#Likewise,#given#the#flexibility#of#the#model#and#the#extent#

of#methodological#variation#that#may#result,#the#READ#model#cannot#guarantee#validity#or#credibility#

of#the#findings#generated#in#future#applications.#These#are#matters#determined#by#the#organisation,#

management#and#methodological#design#of#READ#systems#in#the#future.#

I#interpret#this#latter#finding#to#be#both#a#weakness#and#a#strength#of#the#READ#model#developed#to#

date.#On#the#one#hand,#as#mentioned#above,#it#means#that#the#responsibility#is#transferred#to#the#

practitioners#implementing#the#READ#model#in#the#future,#to#ensure#that#their#tailored#methodology#

is#rigorous#and#robust,#based#on#the#needs#of#the#application.#As#a#strength#however,#it#ensures#

methodological#design#of#the#READ#approach#can#be#reflexive#and#contextKspecific,#and#have#the#

capacity#to#respond#to#variation.#During#the#READ#application,#there#were#two#instances#in#

particular,#that#demonstrated#the#benefit#of#the#reflexivity#of#the#evaluation#practice#afforded#by#the#

READ#approach#in#responding#to#variation.#

The#first#was#related#to#a#variation#in#project#delivery#from#the#originally#conceived#community#arts#

project,#and#also#a#time#schedule#extension.#During#the#project,#the#original#concept#for#the#


# 162#

community#showcase#evolved,#shifting#from#a#performanceKbased#theatre#showcase#to#a#diversified#

multiKarts#approach#to#suit#the#emergent#community#interest#in#photography#and#visual#arts.#The#

personal#health#circumstances#of#one#of#the#creative#team’s#key#artists#were#also#a#consideration#in#

this#variation.#This#variation#represented#a#major#shift#in#our#original#creative#brief#and#had#

significant#ramifications#for#the#creative#practice#and#community#partnerships,#and#also#for#the#READ#

methodologies#that#were#already#being#designed#and#implemented.#

The#READ#methodology#proved#flexible#enough#to#respond#to#the#resulting#project#variations#and#

interestingly#was#effective#at#capturing#and#recording#the#changes#in#ways#that#were#revealing#of#the#

arts#practice#and#processes.#The#main#strategy#impacted#was#Evaluation,#and#this#needed#revising#in#

order#to#ensure#the#instruments#for#data#collection#were#suited#to#the#participants#and#artKmaking#

that#eventuated.#For#Analysis,#this#strategy#wasn’t#affected#as#the#community#of#practice#was#

working#along#longer#timeframes#than#the#Balaklava#project#alone.#It#did#shift#the#research#interests#

to#some#degree,#and#if#anything#the#shift#in#arts#practice#was#viewed#as#an#interesting#development#

in#the#project#from#a#research#perspective.##

Interestingly,#the#strategies#of#Reflection#and#Documentation#played#a#role#in#supporting#and#

documenting#the#shift#in#creative#practice,#which#was#an#unforeseen#value#of#these#strategies.#For#

example,#the#Reflection#methods#and#approach#did#not#need#to#change#in#response#to#the#project#

variation#and#was#a#means#to#capture#learning#and#feedback#from#the#creative#team#and#community#

partners#about#the#process,#and#sharing#these#insights#advanced#the#creative#development#in#a#new#

direction.#Reflection#was#a#lens#to#analyse#the#challenges#and#choices#at#hand#and#elicited#rich#

findings#in#this#respect.#Documentation#also#played#a#valuable#role#in#capturing#the#shift,#and#the#

implications#for#the#creative#process#and#the#unfolding#drama#of#the#partners#to#respond.#At#the#

same#time,#the#reflection#and#documentation#methods#kept#the#continuity#of#data#collection#

happening#while#we#were#reorienting#the#evaluation#questions#and#agreed#approach.#

The#second#variation,#instigated#by#external#circumstances#beyond#our#control,#was#the#

announcement#by#then#Federal#Arts#Minister,#George#Brandis,#of#proposed#budget#cuts#of#

significance#for#the#Australia#Council#for#the#Arts,#in#May#2014.#The#reforms#were#a#direct#threat#on#

the#arm’sKlength,#peerKreviewed#grants#model#of#the#Australia#Council#and#generated#a#massive#

reaction#from#the#arts#sector.#One#development#of#direct#relevance#for#the#Nuclear#Futures#program#

was#that#the#Australia#Council#terminated#its#CCPI#fund.#While#the#funding#commitment#for#the#

existing#CCPI#projects#(such#as#Nuclear#Futures)#was#honoured#and#there#was#no#budget#loss#for#

current#projects,#the#context#of#the#arts#practice,#and#consequently#the#relevance#of#evaluation#

changed#significantly.#In#responding#swiftly#to#the#proposed#reforms,#internal#structural#changes#


# 163#

ensued#for#the#Australia#Council,#as#did#a#serious#level#of#uncertainty#within#the#organisation.#

These#events#unfolded#during#the#months#at#the#close#of#the#Balaklava#project,#when#the#evaluation#

findings#were#scheduled#to#be#delivered#to#partners.#One#implication#of#this#timing#was#that#the#

Australia#Council#had#reduced#capacity#for#engaging#with#the#program#and#with#the#formative#

evaluation#findings.#When#I#asked#my#colleague#what#the#changes#might#mean#for#our#reporting#to#

the#Australia#Council,#he#jokingly#replied#‘where#we#used#to#do#a#5Kpage#report#we’ll#be#doing#a#3K

pager’.#While#this#comment#was#in#jest,#it#stuck#with#me#in#a#reflective#practice#session,#and#in#

examining#the#impact#of#the#external#events,#it#led#me#to#reaffirm#the#merit#and#significance#of#our#

decision#to#direct#our#evaluation#efforts#towards#the#interests#of#all#partners,#and#not#only#the#needs#

of#the#major#funding#body.#I#contemplated#the#potentially#different#scenario#of#our#situation#if#the#

driving#motivation#for#the#project’s#evaluation#had#been#solely#directed#towards#an#evaluation#report#

for#the#Australia#Council#criteria,#and#how#redundant#the#efforts#may#have#been#compared#to#what#

developed#through#the#READ#approach,#where#learning#and#evaluation#outcomes#were#delivered#for#

many#partners.58#While#the#upshot#of#the#reforms#remain#unknown#(at#the#time#of#writing),#it#is#

possible#that#by#2017#–#when#the#summative#findings#from#the#READ#approach#are#presented#to#the#

Australia#Council#about#Nuclear#Futures#and#its#broader#social#implications#as#a#CCPI#–#that#the#

findings#could#have#increased#relevance#for#the#Australia#Council,#potentially#offering#policyKrelated#

information#on#a#wellKdocumented#and#rigorously#evaluated#case#example#that#demonstrates#the#

value#of#experimental#and#partnershipKoriented#community#arts#funding.#

Recommendations(

The#first#recommendation#for#future#evaluation#practice#is#that#the#methodology#for#the#collective#

process#of#review#and#interpretation#of#the#READ#data,#including#collective#processes#of#evaluative#

reasoning,#can#be#improved#and#further#developed.#From#my#analysis#of#the#READ#application,#it#was#

an#area#of#evaluation#practice#that#could#be#strengthened#at#the#site#of#practice.#One#observation#

from#the#Balaklava#project#was#that#participation#from#multiple#stakeholders#was#easier#to#facilitate#

during#the#data#collection#phase,#as#was#the#analysis#of#data#collected#from#the#early#project#

activities.#This#was#because#these#collection#processes#could#occur#faceKtoKface#and#concurrently#

with#the#creative#residencies#when#visiting#artists#were#on#the#ground#and#the#project’s#momentum#

was#high#and#visible#for#partners.#For#the#reflective#and#evaluative#assessment#processes#in#the#later#

stages#of#the#project,#as#in#the#case#of#the#public#showcase,#while#some#progress#was#made#at#the#

#############################################################

58#This#is#not#to#suggest#that#Australia#Council’s#reduced#capacity#for#serious#evaluation#affects#the#benefits#that#individuals#or#organisations#generate#from#wellKconstructed#evaluations#of#previous#projects,#when#applying#for#new#funding.#


# 164#

close#of#the#season#and#during#bump#out#(where#several#facilitated#debriefs,#and#reflective#and#

evaluative#assessment#meetings#were#conducted),#the#process#extended#over#several#months#and#

required#virtual#communication#with#participants#given#our#geographic#dispersion.#For#the#

summative#evaluation#of#Nuclear#Futures,#this#will#be#one#of#the#challenges#of#facilitation,#and#will#

require#planning#and#processes#for#enabling#adequate#and#robust#systems#for#collective#input.#

The#second#key#recommendation#is#that#the#scope#and#scale#of#the#READ#application#needs#to#be#

clearly#defined,#and#this#proved#an#aspect#difficult#to#gauge#for#the#Balaklava#application#because#of#

its#academic#context#(as#outlined#in#previous#discussion).#For#the#application,#this#aspect#of#READ#

planning#was#not#managed#as#it#would#have#been#if#READ#was#undertaken#solely#in#the#industry#

context#as#a#practitioner.#Rather,#the#scope#and#scale#of#the#READ#application#were#deliberately#kept#

open#and#flexible#to#encourage#experimentation#and#innovation.#Defining#the#scope#and#scale#will#

need#to#be#adequately#addressed#by#the#READ#facilitator#and/or#program#manager#to#ensure#

viability#of#the#application.##

6.2.6! Opportunities0for0enriching0evaluation0practice0

This#section#reports#on#the#opportunities#identified#during#the#reflective#analysis.#The#opportunities#

represent#the#areas#of#the#READ#approach#that#made#a#significant#contribution#to#the#efficacy#and/or#

enrichment#of#evaluation#practice#in#the#community#arts#context.#

Reflective(practice(

Reflection#was#used#as#a#data#collection#method#across#the#READ#methodology.#Significant#outcomes#

were#achieved#through#the#approach#to#reflection#as#a#core#READ#strategy,#arising#from#the#culture#

of#reflective#practice#that#was#established.#Reflective#practice#represents#a#significant#opportunity#

for#future#evaluation#practice#in#several#ways.#First,#the#collective#program#of#reflective#practice#

enabled#artists#and#community#members#to#become#critical#analysts#of#the#practice#in#new#ways,#

and#they#were#encouraged#to#share#their#insights.#This#resulted#in#individual#benefits#and#

constructive#exchanges#between#peers.#One#artist,#for#example,#explained#that#she#would#not#have#

normally#conducted#reflective#practice,#if#not#for#the#systematic#and#collective#process#implemented#

through#the#READ#approach,#and#that#the#reflections#had#added#value#to#the#iterative#cycles#of#her#

creative#practice.#Numerous#critical#buddy#exchanges#and#small#group#reflections#were#undertaken#

also,#which#helped#to#progress#creative#development#(evidenced#by#feedback#and#observation).#

Second,#there#was#continuous#improvement#through#organisational#learning#and#the#sharing#of#

information#generated#from#the#reflections.#For#Balaklava#this#related#to#the#improvement#of#the#

program#management#and#community#arts#practice.#Based#on#my#experience#across#disciplines#in#


# 165#

evaluation#and#program#management#previously,#systematic#reflective#practice#for#organisational#

learning#is#not#widely#used#and#this#presents#an#opportunity#for#community#arts#organisations.#

Davidson#and#Rowe#(2009)#explain#how#tacit#knowledge#within#organisational#culture#is#often#

untapped#because#procedural#systems#for#capturing#knowledge#fail#to#draw#out#practical#wisdom:#

Closely#linked#to#the#organizational#culture,#this#difficultKtoKdefine#knowledge#adds#a#perspective#that#comes#only#from#experience,#is#rarely#acknowledged#or#written#down,#and#may#be#intuitive,#yet#contributes#to#judgment#and#wisdom.#This#can#produce#the#creation#of#serendipitous#knowledge#creation#which#more#procedural#systems#inhibit#through#their#emphasis#on#capturing#explicit#knowledge#alone.#(Davidson#and#Rowe,#2009,#p.565)#

The#role#of#reflective#practice#in#capturing#and#translating#tacit#knowledge#for#the#benefits#of#the#

creative#team,#community#partners,#and#Alphaville#organisation#more#broadly,#was#a#significant#

lesson#for#the#management#of#the#Nuclear#Futures#creative#practice,#which#will#have#implications#for#

evaluation#practice#in#the#future.#

Third,#as#well#as#its#value#for#improving#creative#practice#and#program#management,#reflective#

practice#was#relevant#during#the#READ#application#as#an#evaluator’s#tool,#where#individual#and#

collective#reflection#focused#on#the#READ#processes#and#results,#and#this#improved#the#conduct#of#

evaluation#practice.#This#finding#is#consistent#with#theorist#such#as#Bolton#(2010),#who#argue#the#

relevance#of#reflective#practice#for#evaluators:#‘Reflection#on#the#part#of#the#professional#evaluators#

is#also#crucial,#given#the#inherently#politicised#and#valueKbased#nature#of#evaluation,#and#the#need#

for#critical#monitoring#of#bias’#(Bolton,#2010,#p.5).#Thorpe#asserts#that#‘only#through#contemplating#

what#one#is#doing#and#why#can#one#demonstrate#that#intelligent#and#competent#practice#ensues’#

(Thorpe,#2004,#p.330).#For#the#Balaklava#project,#I#concluded#that#the#Reflection#strategy#of#the#READ#

approach#contributed#to#the#continuous#improvement,#robustness#and#logic#of#the#overall#evaluation#

practice.#

Unlocking(the(academies(

Another#opportunity#for#enriching#evaluation#practice#is#demonstrated#by#the#READ#methods#aiming#

to#unlock#the#academies.#As#explained#in#Chapter#5,#Section#5.4.4,#this#concept#relates#to#arts#

organisations#partnering#with#academic#institutions#to#promote#knowledge#generation#and#exchange#

between#academic,#industry#and#community#partners#through#social#enquiry#to#promote#broader#

learning.#While#this#approach#is#not#unique#and#follows#several#other#successful#examples#in#the#field#

(such#as#by#Somebody’s#Daughter#Theatre#Company#and#Milk#Crate#Theatre),#it#is#relatively#

underutilised#as#a#knowledgeKgeneration#strategy#for#community#arts#and#cultural#development.#For#

Nuclear#Futures,#the#community#of#practice#that#resulted#augmented#partnership#development#and#


# 166#

is#producing#research#outputs#that#engage#with#the#wider#implications#of#the#community#arts#

practice#in#ways#that#would#not#have#been#delivered#through#conventional#evaluation.#

Legacy(goals(

The#innovative#use#of#longKterm#artefacts#and#processes#to#keep#the#stories#circulating#and#

accessible#to#future#generations#of#artists,#communities#and#researchers#was#concluded#to#be#an#

original#and#socially#driven#outcome#of#the#READ#methodology.#The#impulse#of#the#legacy#approach#

was#motivated#by#the#objective#of#the#creative#practice#to#‘make#the#story#go#far’,#to#create#

intergenerational#messaging#and#promote#conversations#between#communities,#artists,#researchers#

and#audiences.#The#objective#aligns#with#ideas#by#Mark#(2011)#that#meaningful#evaluation#should#

have#consequences,#such#as#community#development#outcomes#or#social#justice.#The#legacy#goals#

that#were#included#for#READ#extended#the#purpose#of#the#evaluation#practice#beyond#those#more#

orthodox#purposes#such#as#accountability#and#improvement.#

Early#findings#reveal#that#the#iteration#of#the#READ#model#for#Balaklava#has#already#contributed#to#

the#initial#realisation#of#this#objective#by#producing#multimedia#accounts#of#the#broader#range#of#

artworks#and#commentary#generated#during#the#program#that#are#now#secured#in#cultural#

repositories#and#in#the#public#domain.#For#example,#for#the#Balaklava#project,#instigating#legacy#was#

explored#through#a#partnership#with#the#State#Library#of#South#Australia#that#established#longKterm#

collection#development#for#oral#history#and#artefacts#generated#or#unearthed#during#the#project.#

Arrangements#with#families#for#future#donations#have#also#resulted.#The#significant#historical#items#

arising#from#the#creative#research#have#been#transferred#between#the#private#and#public#domains,#

and#will#have#the#benefit#of#being#organised#and#resourced#by#a#cultural#institution#with#a#stake#in#

longKterm#management.#While#it#can#only#be#speculative#as#to#the#benefits#the#approach#will#achieve#

(given#the#short#timeframe#of#assessment),#this#aspect#of#the#READ#application#offers#insight#into#

how#and#why#legacy#objectives#can#be#incorporated#into#the#evaluation#practice.#

The#second#example#from#the#READ#application,#highlighting#the#opportunity#for#evaluation#practice#

to#incorporate#legacy#goals,#was#through#the#utility#of#the#documentation.#It#is#argued#in#the#

literature#that#documentation#of#process#and#artefacts#enables#a#story#to#be#taken#further#afield,#and#

to#be#moved#and#used#beyond#the#story’s#place#of#origin#and#context#(UNESCO#&#ICCROM,#2015).#

This#is#true#for#the#documentation#about#the#creative#works,#where#the#message#and#impact#of#the#

creative#works#can#be#represented#through#the#documentation,#and#in#some#instances#this#has#the#

potential#to#reach#wider#audiences#than#the#artwork#itself.#Documentation#offered#a#means#to#

maintain#and#extend#the#significance#of#the#work#produced.#Future#plans#for#a#series#of#webisodes#


# 167#

and#a#featureKlength#documentary#film,#both#spinKoff#projects#from#the#original#Balaklava#project,#

promise#to#extend#the#utility#arising#from#the#creative#documentation#beyond#the#Nuclear#Futures#

CCPI#funding#and#become#resources#for#public#dissemination.#

Linking(creative(and(evaluation(practice(

The#fourth#opportunity#identified#to#enrich#evaluation#practice#in#the#community#arts#context#relates#

to#the#compelling#examples#demonstrating#the#benefits#of#the#creative#documentation#methods#

used,#which#extended#data#collection#and#diversified#the#endKuse#possibilities#for#READ#outputs#

(RIPP:#Article#26K27#p.98K104#illustrates#some#examples#of#creative#documentation).#The#impulse#

for#including#creative#documentation#in#the#READ#methodology#was#to#draw#on#the#strengths#of#the#

creative#practice,#which#in#the#case#of#the#Balaklava#project#was#inspired#by#documentary#art#

traditions.#From#the#outset,#this#meant#looking#for#opportunities#that#utilised#documentary#artK

making#practices#as#methods#for#achieving#the#goals#and/or#enquiry#of#the#evaluation#practice.#One#

way#this#was#realised#was#via#multiKplatform#creative#documentation,#using#collaborative#CACD#

processes.#Implemented#by#filmmakers,#multiKmedia#artists,#community#members,#program#

administrators#and#partners,#the#creative#documentation#methods#generated#extensive#raw#data#

and#produced#many#evaluationKrelevant#outputs.#This#included#a#vast#collection#of#photomedia#

and#video#footage,#the#development#of#promotional#assets,#storyKbased#artworks/#

documentation,#video#interviews#and#website#content#(see#RIPP:#Article#27,#p.100).##

In#the#READ#application,#creative#documentation#added#to#the#richness#of#the#results#when#used#

alongside#conventional#evaluation#instruments#such#as#surveys#and#questionnaires.#Creative#

documentation#methods#served#to#capture#nuanced#and#ephemeral#data#that#was#not#revealed#

by#the#quantitative#methods#(RIPP:#Article#15,#p.36#has#an#image#that#illustrates#this).#One#of#the#

strengths#of#the#data#collected#through#creative#documentation#was#that#it#provided#insights#

about#the#engagement#and#personal#reactions#of#participants#and#audiences.#Thomas#(2015),#in#

research#on#a#gallery#curation#case#study#at#Queensland#University#of#Technology,#makes#this#

observation#on#the#limitations#of#traditional#reporting#for#representing#engagement:#

What#is#lost#in#the#traditional#reporting#system#is#the#recording#of#information#pertaining#to#engagement.#Engagement#quality#is#a#less#tangible#KPI#[key#performance#indicators]#than#numbers#but#neverKtheKless#was#identified#as#an#intended#and#highly#relevant#deliverable#outcome#of#the#program.#Beyond#the#attendance#figures#the#richness#and#diversity#of#the#engagement#remained#hidden.#(Thomas,#2015,#p.50)#

Based#on#the#DCI#findings,#the#suite#of#creative#methods#used#in#the#READ#application#were#effective#

for#revealing#the#subtle#artistic,#intuitive#and#emotive#expressions#of#engagement#and#experience,#

#

#

#


# 168#

making#a#valuable#contribution#to#the#data#collection,#as#well#as#serving#as#an#engaging#means#to#

review#and#present#data#(on#both#the#processes#and#products#of#the#community#arts#activities).#

This#finding#aligns#with#Patton’s#(1981)#assertion#that#creative#methods#(such#as#photomedia)#are#

useful#for#data#collection#as#well#as#making#meaning#and#communicating#results#(even#when#findings#

are#derived#from#other#more#conventional#and/or#nonKcreative#methods).#In#the#READ#application,#

analysis#revealed#that#the#process#of#making#and#sharing#documentary#art#was#an#entry#point#for#

engagement#with#evaluation#practice.#Overall,#the#central#activity#of#creative#documentation#

enhanced,#legitimated#and#contextualised#the#evaluation#practice#for#the#partners#and#artists#

because#of#the#links#it#made#with#the#arts#practice#and#program#goals.#This#was#true#for#several#

constituencies#within#the#partnerships,#and#of#note,#it#was#particularly#true#for#the#creative#team.##

In#a#context#where#trepidation#of#evaluation#was#overt,#another#related#finding#relevant#for#future#

practice#was#that#the#diversified#READ#approach#shifted#emphasis#off#‘evaluation’,#and#contradicted#

the#assumption#that#evaluation#and#the#creative#practice#under#review#are#mutually#exclusive.#As#a#

result,#it#transformed#group#perceptions#of#what#and#who#could#be#involved#in#the#practice#of#

evaluation#and#in#the#documentary#artKmaking#(see#Section#6.2.2#for#further#background#discussion).#

In#many#instances#the#lines#between#these#practices#blurred,#and#artists#and#community#partners#

used#the#art#methods#for#simultaneous#purposes.#Importantly,#the#creative#documentation#approach#

raised#the#question:#what#is#evaluation#practice#and#what#is#documentary#artKmaking?##

In#challenging#the#assumption#at#the#site#of#practice#that#evaluation#practice#is#extraneous#to#arts#

practice,#traditional#binaries#of#community#arts#and#evaluation#of#community#arts#were#discarded.#

Produced#were#artworks#that#were#evaluation#products,#and#evaluation#products#that#were#

artworks.#Highlighted#were#the#many#linkages#able#to#be#created#and#leveraged#between#artKmaking,#

evaluation,#documentation#and#storytelling.#In#turn,#this#led#to#fluidity#and#resource#efficiencies#

across#the#domains#of#the#READ#implementation,#creative#arts#practice#and#program#management.#

The#incorporation#of#creative#documentation#into#the#READ#practice#supported#the#notion#that#there#

are#many#ways#of#knowing,#and#many#valid#sources#of#knowledge#relevant#for#the#creative#

community#partnership#initiative.#More#broadly,#Hawkesworth#(1989,#p.551)#states#that#‘perception,#

intuition,#conceptualisation,#inference,#representation,#reflection,#imagination,#remembrance’#are#all#

forms#of#knowing.#This#highlights#the#opportunity#practitioners#and#partners#have#for#evaluating#

community#arts#using#creative#methods#as#an#alternative#way#of#learning,#knowing#and#representing#

the#arts#practice#and#partnerships,#to#complement#conventional#methods.#For#Nuclear#Futures,#

creative#documentation#broadened#the#emergent#language#of#READ#processes#and#findings#and#


# 169#

offered#an#aesthetic#description#of#the#community#arts,#which#is#so#often#lacking#in#evaluation#

outputs#in#this#context.59#Doloughan#(2002)#describes#such#a#discourse,#in#the#context#of#art#and#

design,#as#one#that#integrates#‘the#discursive,#the#pictorial,#the#persuasive,#and#the#instrumental’#

(Doloughan,#2002,#p.62)#and#challenges#the#privilege#traditionally#given#to#written#language.#In#the#

community#arts#evaluation#context,#this#is#relevant#given#the#current#tendency#in#the#field#to#

attempt#to#represent#the#vibrancy#and#impact#of#community#arts#through#written#reports,#

bureaucratised#language,#and#static#communication#formats.##

Finally,#reflection#on#the#role#of#creative#methods#for#learning#about#and#appraising#arts#practice#led#

me#to#revisit#the#discourse#canvassed#in#the#contextual#scan,#such#as#Goldbard’s#(2015)#argument#for#

qualitative#storyKbased#methods#as#a#means#to#demonstrate#impact#without#the#narrow#prescription#

of#cultural#measurement#and#impact#quantification;#or#Badham’s#(2015)#notion#of#approaching#the#

notion#of#cultural#value#from#the#bottom#up#so#as#to#best#reflect#and#encapsulate#stakeholder#values#

and#context.#With#the#increased#affordability#and#access#to#good#quality#audio#and#video#equipment#

and#software,#as#well#as#evolving#online#media#platforms#and#creative#art#methods,#practitioners#are#

offered#new#scope#and#sophistication#for#documentation,#analysis,#learning#and#appraisal#in#this#

arena.#In#conclusion,#significant#opportunities#are#being#missed#for#linking#the#domains#of#creative#

practice,#evaluation#practice#and#program#management#of#community#arts,#as#was#experimented#

with#in#the#READ#application.#Greater#exploration#of#these#linkages#would#be#of#interest#for#the#field.#

6.3! The0final0READ0product0

6.3.1! Product0development0

The#findings#and#practical#lessons#arising#from#the#research#enabled#a#workable#model#of#practice#to#

be#established,#and#has#resulted#in#original#evaluation#theory.#In#response#to#the#shared#problem#of#

community#arts#evaluation#at#my#site#of#practice,#I#determined#that#one#useful#output#of#the#DCI#

would#be#a#simplified#READ#product#able#to#guide#future#evaluation#practice.#During#the#DCI#research#

I#engaged#in#the#product#development#of#such#a#tool,#to#consolidate#the#READ#methodology#and#

model#of#practice#developed#to#date.60#The#READ#product#presented#in#this#section#is#the#culmination#

of#the#READ#application#that#I#facilitated#at#the#site#of#practice#(see#Chapter#5)#and#the#critical#review#

I#conducted#using#forensic#reflective#practice,#including#the#metaKevaluation.#It#was#inspired#by#the#

need#to#communicate#the#emergent#READ#model#in#a#simple#and#engaging#way.##

#############################################################

59#One#of#the#best#examples#of#creative#documentation/evaluation#output#reviewed#was#the#memory#basket#by#Big#hArt#(2010)#for#Ngapartji#Ngapartji#(a#treasure#box#of#stories#and#symbolic#objects).#This#however#is#rare#in#the#field.#60#A#product#is#defined#as#‘a#good,#idea,#method,#information,#object#or#service#created#as#a#result#of#a#process#and#serves#a#need#or#satisfies#a#want.#It#has#a#combination#of#tangible#and#intangible#attributes’#(WebFinances,#2016).#


# 170#

For#this#product#development#I#prioritised#my#commitment#to#reflexivity#and#creative#

experimentation#in#order#to#maximise#innovation,#as#well#as#the#use#of#the#systems#approach#

introduced#in#Chapter#5#(Section#5.1.2).#This#has#involved#generating#new#knowledge#about#each#of#

the#READ#strategies#and#methodologies,#and#the#system#as#a#whole,#in#order#to#substantiate,#reK

evaluate#or#redesign#ideas#from#the#conceptual#the#model.#Understanding#has#been#gained#into#

previously#unknown#aspects#of#the#model,#such#as#its#methodological#domains#and#flexibility,#the#

interrelationships#between#strategies,#how#the#integration#processes#achieve#a#wide#range#of#

evaluative#and#nonKevaluative#outputs,#and#the#nuances#of#the#operational#system#once#developed.#

Figure#6.3#shows#the#stages#of#the#structured#READ#product#development#through#the#DCI.##

#

############

#

###

##

Figure#6.3:#Structured#design#and#development#for#the#READ#product##

#(Diagram#is#based#on#‘Structured#Ideas#Development’#(author#unknown)#retrieved#May#9,#2014#from#http://www.processfacilitation.info/processKfacilitationKcreativeKcollaborationKstructuredKideaK

development/.#Link#no#longer#active.)##

#####

Early iteration READ concept

Tailored Balaklava design & initial

model development

Applying and testing at site

of practice

Model theorisation

refinement, & articulation

Final READ product

(consolidated findings)

2012 2016 (not to scale)


# 171#

6.3.2! Profile0for0the0READ0product0

Based#on#the#reflective#analysis,#I#developed#the#following#profile#for#the#READ#product.#The#profile#

lists#the#characteristics,#advantages#and#disadvantages#evident#during#model#development#and#

testing.##

Characteristics((

•! ParticipatoryKoriented#

•! Functions#as#an#integrated#or#modular#approach#to#learning#and#appraisal#

•! Organised#around#four#READ#strategies#–#Reflection,#Evaluation,#Analysis#and#Documentation##

•! Suitable#for#formative#or#summative#learning#and#appraisal#

•! Acknowledges#the#importance#of#the#processes#and#outcomes#of#community#arts#

•! Inclusive#of#culturally#diverse#partners#and#participants#

•! Incorporates#creative#and#artKbased#methods#

•! Participant#focus#and#program/arts#organisation#ownership#

•! Flexible,#able#to#respond#to#variation#

•! KnowledgeKgenerating#

Advantages(

•! Driven#by#locally#relevant#questions,#stakeholderKoriented#

•! Flexible#to#address#a#wide#range#of#stakeholder#needs#for#learning#and#appraisal#

•! Produces#multiKplatform#outputs#(evaluative#and#nonKevaluative)##

•! For#accountability,#improvement#(individual#and#organisational),#broader#learning#and#legacy##

•! Instrumental#and#developmental#benefits#

•! Adaptable#for#capacityKbuilding#and#participant#empowerment##

•! Participatory,#representative,#inclusive#

•! Sustains#organisational#learning#and#growth#

Disadvantages((

•! Time#and#commitment#of#all#involved#

•! ResourceKintensive,#depending#on#the#scope#of#READ#application#

•! Conflict#resolution/facilitation#may#be#needed#for#partnership#brokerage#

•! Not#necessarily#independent#findings#if#undertaken#internally#

•! LongKterm#management#and#maintenance#will#be#required#for#legacy#objectives#

#

#


# 172#

6.3.3! Infographic0

Figure#6.4#presents#the#infographic#developed#to#summarise#the#transferable#elements#of#the#READ#

model#based#on#the#Balaklava#application#findings,#to#demonstrate#the#general#processes#involved#in#

the#READ#approach.#It#aims#to#simplify#and#describe#the#READ#product#as#a#visual#communication#

tool#and#as#a#means#for#branding#the#approach.#The#need#for#an#infographic#was#identified#during#

the#Balaklava#project,#as#I#became#aware#that#a#oneKstop#diagram#that#encapsulated#the#model#(in#

an#engaging#way)#would#be#useful#in#communications#with#stakeholders#and#project#participants.#

The#infographic#presented#in#Figure#6.4#aims#to#fulfil#this#purpose#and#will#be#used#for#future#Nuclear#

Futures#evaluation#practice,#and#to#present#the#model#of#practice#publicly#to#other#practitioners#and#

scholars.#It#has#not#yet#been#tested#at#the#site#of#practice,#and#it#is#anticipated#that#there#will#be#

further#iterations#of#the#infographic#developed#during#2016#based#on#stakeholder#feedback#and#my#

experience#of#using#it#in#practice.#

Design(rationale(

During#the#design#process,#I#experimented#with#several#approaches#for#visualising#the#READ#product.#

In#one#early#attempt,#I#explored#using#the#systems#approach#(based#on#Table#6.1,#and#further#

developing#Figure#5.1),#however#the#result#was#too#complex.#I#wanted#a#clear#and#visually#dynamic#

means#to#demonstrate#the#model.#In#this#section,#I#provide#an#explanation#of#the#emergent#design.#

The#circular#design#developed#as#a#way#to#represent#READ#as#a#summative#lens#on#community#arts#

and#cultural#development#practice,#which#has#been#included#at#the#centre#of#the#infographic.#In#the#

top#semiKcircle#are#the#four#READ#strategies,#clearly#distinguished,#with#their#respective#descriptions.#

The#impulse#to#use#the#pieKshaped#colour#pieces#for#the#READ#strategies#responded#to#the#need#to#

communicate#their#modular#and/or#integrated#potential.#The#READ#strategies#were#given#

prominence#in#the#design#and#visually#command#the#viewer#to#start#interpreting#the#infographic#

from#the#top.#

The#lower#half#of#the#infographic#represents#the#processes#involved#in#designing#and#implementing#

the#READ#approach.#The#inner#sphere#(of#the#lower#semiKcircle)#includes#colourKcoded#data#sources,#

to#provide#examples#of#the#data#collection#options#for#each#of#the#strategies.#It#aims#to#communicate#

the#scope#for#choice#for#practitioners.#Examples#include#gathering#data#from:#publications,#surveys,#

interviews,#field#notes,#sound#recordings,#documentation#and#so#on.#The#instructive#text#‘mix#+#

integrate’#signals#that#practitioners#tailor#the#selection#to#meet#their#needs#and#resources.##

# #


# 173#

#

#

Figure#6.4:#The#READ#model#infographic#

##

(Version#updates,#READ#model#summaries,#and#alternative#print#formats#will#be#made#available#in#the#future#via#www.nuclearfutures.org.)#

#


# 174#

It#is#important#to#note#that#the#data#sources#included#in#the#infographic#represent#a#guide#only,#and#

are#not#prescriptive.#Also,#while#each#is#colourKcoded#to#offer#extra#insight#into#the#READ#strategies,#

some#methods#can#be#utilised#in#multiple#strategies.#For#example,#journaling#is#a#method#that#could#

be#used#in#all#four#of#the#READ#strategies,#but#for#the#purpose#of#the#infographic#this#has#been#colour#

labelled#within#Reflection.#This#is#one#potential#constraint#of#the#infographic#in#its#current#form.#

In#the#next#sphere,#in#the#lower#semiKcircle,#is#the#‘create#+#share’#band.#Included#in#this#section#are#

examples#of#the#resources#and#artefacts#able#to#be#produced#from#the#integrated#(synthesised)#data.#

This#band#communicates#the#products#potentially#generated#from#the#READ#model,#and#includes#

outputs#such#as#raw#data,#evaluation#reports,#publications,#artwork,#management#resources,#and#

project#stories.#To#keep#the#design#simplified,#I#elected#not#to#represent#the#types#of#analysis#

processes#that#may#be#relevant#for#the#integration#(see#Table#6.1).#While#the#infographic#provides#no#

instruction#for#the#integration#process,#the#act#of#integration#is#still#conveyed#and#I#considered#this#

sufficient#for#the#purpose#of#communicating#the#model’s#key#processes.#

The#outer#band,#in#the#lower#semiKcircle#communicates#the#purpose#for#which#READ#may#be#applied:#

for#accountability,#improvement,#broader#learning#and#instigating#legacy.#These#concepts#have#been#

described#in#Chapter#5#and#6,#so#are#not#covered#again#in#this#descriptive#section#on#the#infographic.#

This#band#aims#to#tell#the#viewer#what#can#be#expected#from#applying#the#model,#and#that#READ#can#

be#directed#towards#a#range#of#purposes.#Overall,#the#lower#half#of#the#infographic#conveys#that#the#

choice#of#data#sources#can#be#tailored#to#suit#the#ultimate#purpose#of#applying#the#model,#which#will#

vary#between#projects#and#contexts.#

I#envisage#that#when#used#in#practice#to#

communicate#the#READ#model#of#practice,#ideally#

the#infographic#will#be#verbally#explained#or#

accompanied#by#a#succinct#description#of#the#key#

concepts#via#a#basic#comprehension#guide#(see#

Chapter#7#for#future#research#and#dissemination#

plans).#Version#updates,#READ#model#summaries,#

and#alternative#print#formats#will#be#made#available#

in#the#future#via#www.nuclearfutures.org#(for#

example,#see#image#on#right#for#greyscale#version).# #

#

Greyscale#version#of#the#READ#infographic.#


# 175#

6.3.4! Assumptions,0constraints0and0variables0

The#emergent#READ#product#presented#in#this#section#is#closely#linked#to#the#context#from#which#it#

has#been#developed.#In#the#literature,#it#is#well#documented#that#evaluation#approaches#are#

significantly#contextualised#and#politicised#based#on#their#underpinning#theoretical#concepts#and#

contextual#influences#(Edenloff,#2011;#Davidson,#2005).#Some#scholars#such#as#Boud#and#Walker#

(1998,#p.196)#argue#that#context#is#‘so#allKpervasive#that#it#is#difficult#to#recognise#its#influence’.#With#

this#in#mind#I#used#reflective#practice#to#critically#examine#the#assumptions#and#constraints#of#the#

READ#model,#and#identified#a#list#of#variables#influencing#future#practice.##

One#of#the#significant#assumptions#made#at#the#site#of#practice#when#deciding#to#experiment#with#

the#READ#concept#was#that#engaging#communities#and#partners#in#Reflection,#Evaluation,#Analysis#

and#Documentation#was#going#to#be#beneficial#for#participants/organisations,#and#produce#positive#

experiences#and#outcomes.#However,#as#SielbeckKBowen#et#al.#(2002,#p.4)#points#out,#‘the#evaluation#

or#research#process#can#lead#to#significant#negative#or#positive#effects#on#the#people#involved#in#the#

evaluation/research’.#In#reality,#evaluation#practice#has#the#potential#to#produce#harm#if#conducted#

unethically#or#cause#distress#to#participants#for#reasons#unknown#to#the#evaluators#(MacDowall,#et#

al.,#2013).#Related#is#another#assumption#of#the#model#of#practice:#that#encouraging#participation,#

collaboration#and#learning#as#part#of#the#integrated#READ#approach#will#result#in#positive#exchanges#

for#partners#and#participants.#In#recognising#these#assumptions,#it#is#recommended#that#

practitioners#using#the#READ#approach#in#the#future#constantly#challenge#and#scrutinise#the#

assumptions#for#their#application#and#context#(during#all#phases#of#the#project).#In#doing#so,#

measures#can#be#taken#to#avoid,#minimise#or#respond#to#potential#negative#harm#that#may#occur.##

A#number#of#variables#were#identified#that#will#have#a#bearing#on#the#performance#and#efficacy#of#

future#READ#applications.#These#variables#revolve#around#biases#produced#by#the#selection#of#the#

READ#facilitator#and#team,#the#context#in#which#it#is#implemented,#and#the#methodological#solutions#

designed#for#each#READ#strategy.#In#summary,#the#variables#identified#for#READ#practice#are#as#

follows:#

•! Interpretation:#Although#the#model#attempts#to#distinguish#Reflection,#Evaluation,#Analysis#

and#Documentation#as#independent#yet#interrelated#READ#strategies,#the#methods#

applicable#for#each#can#overlap#significantly,#and#the#strategies#and#methods#will#be#subject#

to#interpretation.##

•! Purpose:#This#includes#why#the#implementation#is#being#undertaken#and#what#stakeholder#

needs#are#to#be#served.#

•! Subject/Focus:#The#CACD#processes#or#activities#at#the#focus#of#the#READ#application.##


# 176#

•! Context:#This#includes#consideration#of#the#impact#of#different#institutional#settings#(public#

and#private#arts#organisations,#government#agencies,#independent#artist,#education#arts#

settings);#and#the#social#context#(individual#or#group/team,#internal#versus#external#

position);#as#well#as#the#cultural#(including#ideologies#and#protocols)#and#geographic#context#

(access,#safety,#resources).#

•! Partnerships#and#level#of#participation:#Questions#arise#about#which#stakeholder#interests#

are#reflected#and#represented#during#the#scoping,#design,#conduct,#implementation#and#

dissemination#phases,#and#how#participatory#the#application#needs#to#be.#(Note:#

Participation#does#not#guarantee#ethical#or#representative#input#or#validity#or#relevance#of#

findings#(SielbeckKBowen#et#al.,#2002))#

•! Stakeholder#values:#Davidson#(2005)#states#that#the#question#is#not#whose#values#should#be#

incorporated#but#what#values,#based#on#the#context#and#terms#of#reference.#

•! Scale,#scope#and#site#of#practice:#For#smallKscale#READ#applications#the#available#resources,#

time#and#expertise#may#be#limited#compared#to#large#organisations#that#may#have#resources#

to#engage#in#more#comprehensive#or#longitudinal#studies#for#national#or#international#

programs.#

•! EndKuses#and#formats:#This#includes#questions#of#immediate#versus#longKterm#utility,#the#

varying#audiences#involved#and#format#(or#language)#for#delivered#outputs#such#as#digital,#

hardcopy,#written,#audio#visual#and#so#on.#

•! Conduct:#There#will#be#different#and#conflicting#approaches#on#how#to#implement#the#READ#

framework#and#what#key#players#hope#to#get#out#of#it.#

•! Design#and#implementation:#This#includes#the#methodological#and#epistemological#aspects,#

and#methods#selection#(qualitative,#quantitative#and/or#rich#ephemeral#data).#

From#these#variables,#and#based#on#the#findings#from#the#contextual#scan,#I#then#developed#a#

preliminary#list#of#guiding#questions#for#future#evaluation#practice#(addressed#to#the#practitioner).#I#

include#the#question#set#as#a#demonstration#of#the#analysis#of#future#variables,#to#illustrate#the#

considerations#arising#from#the#reflective#exercise.#The#questions#for#future#practitioners#are#

presented#on#the#following#page.#

# #


# 177#

Preparation#questions#for#future#READ#practitioners:#

•! What#is#your#site#of#practice#or#context?#

•! Why#is#READ#being#undertaken?#

•! Why#is#improved#learning#and/or#appraisal#of#value?#

•! How#do#you#currently#learn/reflect/evaluate/analyse/share?#

•! What#is#the#scale#and#scope#of#the#activities#or#review?#

•! What#do#you#hope#to#achieve?#

•! What#barriers#exist#(such#as#for#evaluation,#improving#learning#and#sharing)?#

•! What#information#is#useful#and#for#whom?##

•! What#would#you/partners#like#to#know?#Are#you#prepared#to#be#surprised?#

•! Which#‘language’#will#best#represent#your#story#(text,#images,#film,#reports,#statistics,#

infographics)?#

•! How#rigorous#do#you#need#to#be?##

•! What#will#count#as#‘evidence’?#

•! Is#it#a#solo#or#shared#process?##

•! Who#will,#or#needs#to#be,#involved,#and#in#what#capacity?#

•! Will#you/participants#need#support?#Where#will#you#find#it?#

•! How#and#who#will#facilitate#and#broker#relationships?#

•! What#philosophies#and#priorities#will#frame#the#approach?#

•! Is#organisational#learning#and#a#culture#of#reflection#relevant?#

•! Would#you#like#to#advance#professional#practice?#

•! Are#the#evaluative#questions#set,#or#do#they#need#developing?#

•! What#evaluation#values#need#to#be#represented?#

•! What#methods#best#match#your#learning#and#appraisal#needs?#

•! Is#creative#documentation#of#interest?#

•! What#would#a#strengthKbased#approach#look#like?#

•! Where#are#the#current#gaps#in#information#and#communication?#

•! How#will#meaning#be#made#and#shared?#

•! What#validity#checks#will#be#included?#Is#external#review#relevant?#

•! How#will#you#engage,#empower#and/or#inspire#others#along#the#journey?#

•! What#are#the#dissemination#goals#and#platforms#for#information#sharing#most#relevant#to#

your#setting?#


# 178#

6.4! Chapter0conclusion0

The#development#and#application#of#the#READ#model#offered#a#contextKspecific,#inKdepth#exploration#

of#the#theory#and#practice#of#integrating#Reflection,#Evaluation,#Analysis#and#Documentation#as#a#

summative#lens#for#learning#and#appraisal#within#a#dynamic#community#arts#setting.#The#READ#

application#conducted#for#the#Balaklava#Community#Arts#Project#was#guided#by#an#interpretative#

approach#that#allowed#for#a#detailed#understanding#about#the#model#in#action#to#be#developed.#

From#the#findings,#I#constructed#a#general#model#of#practice#and#positioned#this#within#the#field#

alongside#contemporary#evaluation#theories.#This#informed#the#development#of#a#consolidated#READ#

product#for#future#practice,#and#the#reflective#analysis#confirmed#a#number#of#key#findings.##

The#findings#demonstrate#that#the#emergent#READ#model#offers#flexibility#as#an#integrated#or#

modular#evaluation#approach#that#can#be#tailored#for#varying#needs.#It#is#argued#that#while#

reflection,#evaluation,#analysis#and#documentation#are#routine#methods#used#in#contemporary#

evaluation#practice,#the#innovation#of#the#READ#model#lies#in#the#integration#of#these#as#strategies.#

For#evaluation#practice,#this#means#that#the#model#can#be#compartmentalised#in#order#to#maximise#

the#desired#learning#and#appraisal#outcomes,#and#that#multiKplatform#outputs#can#be#produced#for#a#

range#of#purposes#including#accountability,#improvement,#broader#learning#and#creating#legacy.#The#

process#is#also#inclusive#of#creative#methods#that#offer#rich#descriptive#representations,#and#that#are#

not#necessarily#driven#by#valuation,#merit#attribution#or#judgement#as#the#central#concern.##

The#findings#also#indicate#that#READ’s#capacity#to#deliver#benefits#for#partners#was#enhanced#by#the#

internal#conduct#of#the#evaluation#practice.#My#dual#role#of#READ#facilitator/manager#created#

numerous#advantages,#such#as#realKtime#information#exchange#and#utilisation#across#READ#activities,#

management#and#creative#practice.#In#keeping#with#the#community#arts#ethos,#the#internal#position#

kept#the#power#to#govern#learning#and#appraisal#processes#and#outcomes#within#the#creative#

partnerships.#On#examination#of#the#constructivist#stance#adopted#for#the#READ#application,#the#

interpretive#approach#demonstrated#good#credentials#for#offering#an#inclusive#and#partnerKfriendly#

means#of#accommodating#the#divergent#world#views#and#evaluation#constructs#at#the#site#of#practice.##

In#the#chapter#I#also#presented#a#broader,#holistic#assessment#of#the#model,#based#on#results#using#

analytical#and#metaKevaluation#tools#(for#evaluating#the#evaluation)#where#efficacy#was#analysed#

against#the#criteria#of#validity,#utility,#conduct,#credibility#and#cost.#This#highlighted#the#strengths#and#

weaknesses#of#the#READ#model#during#the#Balaklava#trial,#and#the#opportunities#and#constraints#of#

the#READ#approach.#The#most#significant#constraint#identified#of#the#READ#methodology#was#the#

time#and#resources#required#to#manage#the#four#concurrent#READ#strategies,#and#the#subsequent#


# 179#

information#management#demands#arising#from#the#volume#of#data#collected.#Further,#while#the#

model#provides#a#guiding#framework#for#learning#and#appraisal#through#the#four#READ#strategies,#

there#is#limited#prescription#for#the#design#of#the#methodology#or#participatory#processes,#given#that#

evaluations#are#best#designed#and#implemented#to#be#contextKspecific.#Practitioners#applying#the#

READ#model#will#thus#need#to#ensure#the#appropriateness,#rigour#and#credibility#of#their#tailored#

framework#in#response#to#partner#needs#and#context,#and#consider#strategies#that#may#be#required#

to#satisfactorily#manage#the#vulnerabilities#of#the#model#when#internally#delivered.#

Of#broader#relevance#are#the#opportunities#revealed#by#the#READ#approach,#and#the#insight#these#

offer#for#extending#and#enriching#evaluation#practice.#First,#significant#value#was#added#from#the#

collective#program#of#reflection,#which#delivered#benefits#as#an#evaluator’s#tool,#for#the#

improvement#of#creative#arts#practice,#and#as#a#strategy#for#organisational#learning.#Second,#the#

methods#implemented#for#achieving#the#broader#learning#and#legacy#objectives#were#shown#to#

extend#the#purpose#and#impact#of#the#evaluation#practice,#in#order#to#disseminate#and#safeguard#

READ#outputs#in#the#public#domain.#This#included#methods#to#unlock#the#academies,#and#the#

potential#to#involve#cultural#institutions#as#partners#in#the#evaluation#practice,#so#that#the#wider#

implications#of#the#community#arts#practice#become#known#and#shared.#Fitting#with#the#community#

partnerships#approach#defining#contemporary#community#arts,#these#methods#widened#the#project’s#

scope#for#creating#mutually#beneficial#partnerships#between#arts#and#nonKarts#organisations#(one#of#

the#goals#for#Creative#Communities#Partnership#Initiative#funding).#Last,#the#findings#point#to#the#

opportunities#offered#by#creative#documentation#–#for#engagement,#participation,#linking#project#

domains#and#drawing#on#documentary#art#traditions#–#to#the#point#where#artKmaking#and#evaluation#

practice#were#performed#simultaneously#on#many#occasions.#

The#findings#provide#a#scaffold#for#learning#and#appraisal#as#well#as#a#practical#tool#for#

communicating#READ’s#integrated#approach.#In#summing#up,#the#READ#application#demonstrated#

that#at#the#site#of#practice#the#original#knowledge#generated#through#the#READ#lens#impacted#and#

improved#evaluation#practice#and#contributed#to#advancing#the#community#partnerships.#Of#

significance#was#the#value#delivered#from#the#mix#of#evaluative#and#nonKevaluative#outputs#

produced,#a#consequence#of#READ’s#wide#focus#and#multiKstrategy#approach.#The#findings#in#Chapter#

6#establish#the#grounds#for#the#general#model#of#READ#practice#proposed#through#the#DCI#and#is#the#

foundation#for#the#original#knowledge#claims#argued#in#Chapter#7,#the#final#chapter#of#the#

dissertation.#In#Chapter#7,#I#reflect#on#the#two#overarching#research#questions#and#present#my#

summative#conclusions.#In#addition,#I#discuss#the#research#implications#on#my#own#evaluation#

practice,#at#my#site#of#practice#and#the#field,#and#my#plans#for#future#dissemination#and#research.#


# 180#

#

Blank#page#


# 181#

70 Conclusion0

This#research#aimed#to#develop#and#test#a#new,#integrated#approach#to#evaluation#for#the#

community#arts#context#–#the#READ#model.#The#READ#model#combines#Reflection,#Evaluation,#

Analysis#and#Documentation#(READ)#to#create#a#wide#summative#lens#for#accountability,#

improvement,#broader#learning,#and#legacy#purposes.#Prior#to#this#research,#the#idea#for#READ#

existed#in#concept#form#only.#It#was#coined#in#2012#by#SydneyKbased#arts#company#Alphaville#to#

meet#the#evaluation#needs#of#the#Nuclear#Futures#Partnership#Initiative#(Nuclear#Futures),#an#

international#arts#and#cultural#program,#and#the#site#of#my#professional#practice.#The#Doctor#of#

Creative#Industries#(DCI)#offered#a#means#to#advance#the#early#READ#concept#into#a#working#model#

of#practice,#and#trial#the#collectively#developed#methodology#in#an#industry#context.#To#study#the#

design,#efficacy#and#potential#of#the#READ#model,#I#utilised#forensic#reflective#practice#as#my#

principal#research#strategy.#This#involved#the#analysis#and#theorisation#of#the#emergent#READ#

methodology,#to#create#original#knowledge#on#the#READ#model#and#a#product#for#future#practice.##

Chapter#7#consolidates#the#research#and#presents#the#findings#from#the#investigation.#In#focus#are#

my#summative#reflections#on#the#Nuclear#Futures#application#of#the#READ#approach,#elucidating#the#

key#practitioner#insights#and#original#knowledge#arising,#to#address#the#two#DCI#research#questions:#

a)! How#does#the#READ#model#contribute#to#program#learning#and#appraisal#for#creative#community#partnerships#within#community#arts,#based#on#the#Nuclear#Futures#Partnership#Initiative#experience?##

b)! What#innovation#does#READ#offer#evaluation#practice#and#program#evaluation?#

The#chapter#begins#with#a#summation#of#the#shared#problem#of#community#arts#evaluation#tackled#in#

the#DCI,#and#next,#draws#out#the#key#elements#of#the#READ#model#of#practice#resulting#from#the#

research.#I#review#the#DCI#outputs#and#the#areas#of#innovation#identified,#as#well#as#the#assumptions#

and#constraints#of#the#findings.#The#relevance#and#impact#of#the#research#are#examined#for#my#own#

professional#evaluation#practice,#at#the#site#of#practice,#and#for#the#field#of#community#arts#and#

beyond.#Following#this,#I#provide#my#intent#and#direction#for#dissemination#and#future#research.#

The#summative#conclusions#corroborate#the#efficacy#and#impact#of#the#READ#model#as#a#learning#and#

appraisal#framework#at#the#site#of#practice,#and#it#is#argued#that#the#integrated#mixed#method#

approach#offers#both#rigour#and#reflexivity.#As#well#as#equitably#dealing#with#divergent#stakeholder#

needs#and#values#(including#accommodating#trepidation#of#evaluation),#READ#embraces#methods#for#

engagement#and#knowledgeKgeneration#embedded#in#community#arts#and#cultural#development.#In#

widening#the#scope#and#creativity#of#evaluation#practice,#the#research#and#the#developed#READ#

model#advance#evaluation#practice#and#deliver#an#innovative#new#approach#for#community#arts.##


# 182#

7.1! The0DCI0research0

My(approach(to(a(shared(problem(

The#DCI#research#was#inspired#by#the#shared#evaluation#problem#arising#at#my#site#of#practice,#during#

the#planning#and#establishment#of#the#Nuclear#Futures#program#during#2012#and#2013,#by#arts#

company#Alphaville.#Amongst#our#cohort,#we#considered#how#we#could#best#approach#the#requisite#

evaluation#of#this#large#and#multiKstakeholder#community#arts#program#(funded#by#the#Australia#

Council#for#the#Arts),#and#in#doing#so,#many#questions#were#raised.#As#program#manager,#I#was#a#key#

player#in#these#discussions#and#deliberations.#

As#discussed#in#Chapter#2,#evaluation#in#community#arts#is#contentious#and#widely#disputed.#The#

advent#of#increasingly#structured#evaluation#as#a#routine#requirement#for#external#funding#sees#

practitioners#face#mounting#pressure#to#demonstrate#the#value#of#their#work#in#competitive#

environments.#Coupled#with#this#is#the#broader#trend#towards#cultural#measurement,#and#the#

pursuit#to#quantify#the#value#and#impact#of#the#arts.#For#practitioners#undertaking#evaluation#of#

community#arts,#there#are#both#philosophical#and#technical#challenges#to#navigate,#and#no#agreed#

framework#or#criteria#for#best#practice#exists.#While#significant#developments#have#occurred#in#the#

field#over#the#past#30#years,#effective#evaluation#practice#for#community#arts#remains#elusive,#

unresolved#and#disputed.#At#one#extreme,#opponents#argue#evaluation#in#its#current#usage#and#

orientation#is#detrimental#for#the#field#(Goldbard,#2006,#2008,#2015;#Belfiore#&#Bennett,#2008).##

Inspired#by#the#needs#of#the#Nuclear#Futures#Partnership#Initiative,#with#its#diverse#community#

partnerships#and#multiKarts#creative#practice,#host#organisation#Alphaville#sought#an#original#solution#

to#the#dilemma#of#community#arts#evaluation.#For#the#Nuclear#Futures#program,#the#required#

evaluation#framework#would#need#to#assess#the#processes#and#outcomes#of#community#arts#with#a#

variety#of#stakeholders,#from#artists#to#atomic#survivor#communities#in#Australia,#Japan,#Kazakhstan#

and#the#Marshall#Islands,#as#well#as#to#evaluate#the#multiKplatform#creative#works#produced,#and#the#

impact#generated#for#partners#and#participants.#To#address#the#key#evaluation#questions#arising,#

Alphaville#devised#the#READ#concept#to#combine#Reflection,#Evaluation,#Analysis#and#Documentation#

as#concurrent#strategies#for#learning#and#appraisal.##

Prior#to#the#research,#only#a#basic#brief#for#the#conceived#READ#approach#existed#(see#Chapter#5,#

Section#5.1.1#for#the#280#word#READ#proposal#establishing#the#extent#of#previous#concept#

development).#To#create#a#working#READ#model,#the#DCI#research#has#centred#on#my#evaluation#

practice#as#lead#designer#and#facilitator#for#the#collective#development#process#of#a#READ#

methodology#for#the#Balaklava#Community#Arts#Project,#delivered#in#regional#South#Australia#in#


# 183#

2014K5#as#the#first#of#the#Nuclear#Futures’#projects.#My#role#has#involved#operationalising#a#model#of#

practice#for#the#READ#approach,#and#consolidating#the#READ#epistemology#and#methodology#

through#a#process#of#critical#analysis#of#the#model’s#performance#in#action.#In#order#to#critically#

analyse#my#evaluation#practice,#and#the#emergent#READ#model#of#practice,#I#employed#forensic#

reflective#practice#as#the#principal#research#strategy#for#the#DCI,#and#this#utilised#research#methods#

such#as#written#and#video#reflection,#digital#storytelling,#interviews#and#observation#(see#Chapter#3).##

As#outlined#in#Chapter#5,#at#the#site#of#practice,#there#were#numerous#evaluation#needs#identified#in#

the#scoping#with#stakeholders#that#were#taken#into#account#during#the#collective#evaluation#practice,#

such#as#the#need#for:#formative#and#summative#outputs#(including#project#and#evaluation#reporting);#

storyKbased#and#creative#documentation;#community#archiving;#broader#learning#and#dissemination#

outcomes;#information#generation#for#continuous#improvement,#for#creative#practice,#evaluation#

and#management;#and#alignment#with#the#ethos#and#practice#of#the#community#arts.#The#focus#of#

the#scoping#was#both#the#Balaklava#Community#Arts#Project#and#the#Nuclear#Futures#program#

generally.#Based#on#the#findings#of#the#stakeholder#scoping,#I#determined#that#a#broad#definition#of#

evaluation#practice#would#need#to#be#adopted#in#order#to#cultivate#an#inclusive#and#engaged#culture#

of#learning#and#appraisal#at#the#site#of#practice,#and#to#produce#authentic,#representative#and#useful#

outputs#and#findings.#

To#ensure#relevance#and#credibility,#the#evaluation#solution#pursued#also#needed#to#suit#the#nuanced#

setting#of#the#Nuclear#Futures#creative#communities#partnership#initiative,#and#this#included#the#

need#to#embody#the#ethos#of#the#creative#practice#which#was#participationKoriented#and#strongly#

influenced#by#documentary#art#traditions#and#community#arts#and#cultural#development#(CACD)#

principles.#For#validity,#the#READ#model#of#practice#needed#to#rely#on#a#rigorous#and#substantiated#

methodology,#and#include#measures#for#managing#the#implications#of#the#internally#positioned#

evaluation#delivery.#In#addition,#overarching#the#siteKspecific#evaluation#needs#and#agendas#of#the#

participating#communities,#artists#and#institutions,#was#the#more#challenging#question#at#the#centre#

of#the#Nuclear#Futures#creative#practice:#What#creative#art#works#and#processes#can#endure#across#

the#deepKtime#of#the#nuclear#future?#This#question,#of#interest#to#the#Nuclear#Futures#cohort,#related#

to#the#objective#to#enable#the#program#stories#‘to#go#far’#into#the#deep#nuclear#future,#to#generate#

legacy#impacts#from#the#program#and#the#evaluation#practice#(Alphaville,#2015).#

During#the#DCI#I#facilitated#collective#input#and#participation#in#the#READ#model#development#prior,#

during#and#following#the#Balaklava#project,#and#this#was#combined#with#data#generated#from#my#

systematic#process#of#forensic#reflective#practice#to#formulate#the#articulated#READ#model#and#

product#presented#in#Chapters#6#and#7,#and#overall#findings.#While#the#READ#application#for#the#


# 184#

Balaklava#Community#Arts#Project#was#the#central#research#focus,#it#was#intended#that#if#the#READ#

concept#was#deemed#to#be#viable#and#effective#as#a#potential#model#of#practice,#based#on#the#DCI#

findings,#that#READ#could#be#applied#across#all#projects#of#Nuclear#Futures#and#for#the#program’s#

summative#evaluation.#This#meant#that#the#DCI#research#promised#immediate#benefits#for#extending#

my#own#evaluation#practice,#as#well#as#a#prospective#means#to#develop#an#evaluation#model#for#the#

program#partners#and#host#arts#organisation#Alphaville.#Given#the#research#problem#also#resonated#

with#identified#research#and#knowledge#gaps#in#the#field#(see#Chapter#2),#the#academic#context#of#

the#research#presented#the#opportunity#to#share#any#innovations#in#evaluation#practice#with#other#

scholars#and#practitioners,#and#contribute#to#the#discourse#on#effective#community#arts#evaluation.#

DCI(outputs(

Overall,#the#DCI#research#has#advanced#my#professional#evaluation#practice,#and#the#evaluation#

practice#for#Nuclear#Futures,#and#makes#a#valuable#and#original#knowledge#contribution#to#the#

contested#field#of#community#arts#evaluation.#The#principal#DCI#research#outputs#are#the:#

1.! Dissertation#(this#document):#A#comprehensive#document#outlining#the#research#problem,#site#of#practice#and#research#findings.#Included#are#original#research#outputs#such#as#the#digital#story,#the#reflective#questions#set,#and#the#READ#product#(including#infographic).#

2.! READ#in#Practice#portfolio:#A#compilation#of#materials#and#resources#generated#through#the#evaluation#practice,#facilitating#the#READ#application.#Included#are#original#outputs#such#as#the#Nuclear#Futures#evaluation#plan,#designed#evaluation#instruments,#program#documentation#and#reports,#and#conference#presentations#(see#companion#document#submitted#for#examination#–#164#pages).#The#portfolio#is#intended#as#a#companion#document#to#the#dissertation#and#has#been#structured#as#a#standKalone#research#output.#

Assumptions(and(constraints(

In#presenting#this#research,#there#are#several#assumptions#and#constraints#to#acknowledge.#First,#the#

research#is#not#a#complete,#summative#analysis#of#the#READ#model#in#action.#It#focuses#on#the#early#

developmental#stage#of#the#conceptualised#model,#limited#to#the#study#of#READ’s#design#and#trial#for#

one#Nuclear#Futures#project.#For#some#aspects#of#READ,#such#as#the#Analysis#strand#and#academic#

outputs,#while#solid#progress#has#been#made,#several#of#the#anticipated#READ#outcomes#will#extend#

beyond#the#research.#Future#READ#applications#will#also#lead#to#the#further#refinement#of#the#model,#

such#as#addressing#the#weaknesses#identified#in#Chapter#6#(where#possible)#in#preparation#for#the#

summative#evaluation.#Second,#it#is#noted#that#the#considerations#and#scope#of#the#READ#application#

for#the#DCI#study#extends#well#beyond#the#usual#capacity#of#community#arts#evaluation#(where#

generally#evaluations#are#underKresourced).#For#this#reason,#subsequent#READ#applications#will#need#

to#be#adapted#to#suit#available#resources#and#expertise#using#the#infographic#and#guide#in#Chapter#6.##


# 185#

Third,#it#is#assumed#that#for#the#DCI#research#conducted,#that#the#validity#and#credibility#of#the#

research#have#been#adequately#addressed#in#the#research#design#(see#Chapter#3).#The#practiceK

informed#approach#and#qualitative#methodology#was#based#on#systematic#forensic#reflective#

practice,#and#utilised#methods#such#as#ongoing#reflection,#interviews,#digital#storytelling,#and#

observation.#I#consider#the#methodology#implemented#to#have#been#effective#for#accommodating#

the#practitionerKasKresearcher#approach#of#the#DCI#and#that#it#has#produced#reliable#and#valid#

results.#One#noted#constraint#of#the#DCI#methodology#is#that#there#are#limitations#for#generalising#

the#findings,#given#my#central#position#as#researcher#(as#evaluator#and#forensic#reflective#

practitioner)#and#the#strong#influence#of#the#context#on#the#evaluation#practice#(which#is#the#case#for#

all#evaluation#theory).#Other#methodKbased#constraints#limiting#the#potential#to#generalise#the#

findings#include#the#data#collected#from#the#interviews,#which#while#producing#rich#and#detailed#

information#of#relevance#for#the#context,#relied#on#a#very#small#sample#size#and#purposeful#selection,#

and#thus#is#not#representative#of#the#community#arts#field#in#entirety.##

Fourth,#the#DCI#findings#in#Chapter#6#are#contextKspecific,#inextricably#linked#to#the#Nuclear#Futures#

site#of#practice.#The#influence#of#context#has#had#an#impact#on#the#model’s#development#and#on#my#

own#reflective#practice.#For#example,#the#analysis#and#metaKevaluation#conducted#on#the#READ#

application#was#underpinned#by#criteria#for#good#practice#grounded#in#the#Balaklava#project#context.#

The#measures#and#constructs#of#good#evaluation#applied#in#the#reflective#practice#were#informed#by#

the#metaKevaluation#tools#selected,#my#values#and#interpretations#of#the#tools,#and#the#values#and#

priorities#identified#by#project#stakeholders.#As#a#result,#the#research#is#not#replicable,#as#no#two#

READ#applications#will#be#the#same.#Thus,#the#claims#made#about#the#outcomes#and#impact#of#the#

READ#model#in#this#dissertation#relate#specifically#to#the#Balaklava#designed#application.#

Despite#such#constraints,#the#findings#do#contain#significant#and#relevant#practical#instruction#about#

the#READ#model#of#practice#and#integrated#approach#beyond#the#site#of#practice,#and#is#certainly#

relatable#for#and#transferable#by#other#practitioners.#While#the#Balaklava#project#(and#Nuclear#

Futures#site#of#practice)#is#not#characteristic#of#all#creative#community#partnerships,#it#does#

exemplify#many#of#the#common#attributes#of#such#initiatives#(see#Chapter#2,#Section#2.1.2).#Thus,#it#is#

assumed#as#reasonable#to#present#the#READ#model#as#an#emergent#evaluation#theory#of#relevance#to#

other#sites#of#practice,#and#offer#it#as#an#original#and#credible#approach#for#community#arts#

evaluation,#based#on#its#success#in#the#Balaklava#context.#Fitzpatrick,#et#al.,#(2012,#p.112)#argue#‘the#

various#approaches,#or#theories,#proposed#by#Evaluators#make#up#the#content#of#the#field#of#

evaluation’.#In#this#way,#the#READ#product#becomes#a#generalised#model,#offering#a#guide#framework#

for#learning#and#appraisal#of#potential#interest#to#practitioners#beyond#the#site#of#practice.#


# 186#

7.2! Summative0conclusions0

In#this#section#I#present#my#summative#conclusions#based#on#the#findings#from#Chapter#6.#I#argue#that#

the#READ#model#achieved#effective#and#appropriate#evaluation#outcomes#for#the#Balaklava#

Community#Arts#Project,#and#that#the#model#demonstrated#innovation#by#applying#an#integrated#

approach#to#learning#and#appraisal#that#utilised#the#creative#strengths#of#the#partnerships#and#arts#

practice.#In#doing#so,#I#argue#that#the#READ#approach#makes#an#original#and#relevant#contribution#as#a#

model#of#evaluation#practice#for#this#field.#While#contextKspecific,#the#conclusions#demonstrate#a#

number#of#innovative#developments#in#evaluation#practice,#and#provide#practical#insight#into#the#

epistemological#and#technical#aspects#of#the#READ#model#as#an#emergent#approach#for#community#

arts#evaluation,#which#is#of#potential#interest#to#other#scholars#and#practitioners.##

How(does(the(READ(model(contribute(to(program(learning(and(appraisal(for(creative(community(

partnerships(within(community(arts,(based(on(the(Nuclear(Futures(Partnership(Initiative(experience?((

The#READ#model#makes#a#strong#contribution#as#a#learning#framework.#It#produces#a#variety#of#

outcomes#from#utilising#the#evaluation#practice#as#a#knowledgeKdriven#lens#for#gaining#understanding#

into#the#community#arts#practice.#This#included#the#fulfilment#of#contextKspecific#and#broader#

learning#goals.#The#application#demonstrated#that#the#developed#READ#model#had#the#capacity#to#

incorporate#learning#processes#across#all#four#READ#strategies,#and#this#resulted#in#a#range#of#diverse#

learning#outcomes,#evidenced#across#three#planes:#the#individual#participant#level,#at#the#small#

groups/teams#level,#and#at#the#program#and#organisational#level.##

Examples#of#the#learning#outcomes#achieved#during#the#collective#design#and#application#process#

taken#from#Chapter#6#included:#

•! Professional#development#(via#mentoring,#training,#conference#attendance);#

•! Skills#development#(via#mentoring,#training,#workshops,#information#dissemination,#exposure#

to#new#information/partner#approaches#and#collaboration);#

•! Advancement#of#creative#practice#(via#individual#and#collective#reflective#practice#findings,#

application#of#evaluation#and#research#findings#during#creative#development);#

•! Advancement#of#evaluation#practice#(via#individual#and#collective#capacity#building#during#

design#and#implementation,#technical#training,#experimentation#and#empowerment);#

•! Development#of#a#sustained#culture#of#reflection#(particularly#for#Alphaville#and#the#core#

Nuclear#Futures#personnel);#

•! Establishment#of#a#community#of#practice#(practitioners#and#researchers#working#towards#

academic#analysis#goals#and#publication#of#research);#


# 187#

•! Organisational#learning#(for#Alphaville#through#reflection,#evaluation,#analysis#and#

documentation#processes#and#outputs);#

•! Original#knowledge#generation#(via#artKmaking,#reflective#practice,#evaluation,#research);#

•! Continuous#improvement#(use#of#READ#data#to#improve#program#and#creative#practice,#for#

the#Balaklava#project#and#future#Nuclear#Futures#projects);#

•! Education#and#broader#learning#(academic#and#industry#presentations,#research#

dissemination;#contribution#to#collection#development#and#dissemination#of#educationK

related#resources);##

•! Partnership#development#(via#learning#exchange#and#information#sharing#between#partners).#

Importantly#for#the#various#future#contexts#to#which#the#READ#model#may#be#applicable,#the#learning#

outcomes#produced#benefited#a#mix#of#stakeholders#including#community#members,#artists,#

participants,#management#personnel,#staff,#and#representatives#from#partner#organisations.#Through#

the#allied#learning#strategies#of#READ,#knowledge#from#a#range#of#sources#was#explored,#valued#and#

incorporated,#including#tacit,#local,#indigenous#and#scientific#data.#The#information#sharing#occurred#

in#structured#and#spontaneous#ways,#enhanced#by#my#dual#role#of#READ#facilitator/manager,#and#

this#in#turn#had#positive#impacts#for#the#creative#practice,#partnership#development,#program#

management#and#overall#learning#outcomes.#In#terms#of#maximising#the#utility#and#dissemination#of#

the#data#generated#through#READ,#the#internal#position#and#dual#role#of#the#READ#facilitator/#

program#manager#provided#an#enormous#advantage.#

For#the#evaluation#and#analysis#of#READ’s#performance#as#a#framework#for#appraisal,#I#defined#this#to#

encompass#the#broad#definition#of#evaluation#formulated#at#the#site#of#practice,#which#included#

more#general#sensemaking#processes.#At#the#site#of#practice,#appraisal#focused#on:#the#creative#

process,#artworks#produced,#community#engagement,#project#progress,#and#partnership#quality,#

dynamics#and#needs.#Based#on#the#data#generated,#findings#confirmed#that#the#mix#of#knowledgeK

driven#appraisal#methods#employed#across#the#four#READ#strategies#sufficiently#met#the#identified#

evaluation#needs#of#the#stakeholders.#Stakeholder#feedback#on#the#evaluation#processes#and#

outcomes#(and#the#overall#experience#of#READ)#indicated#consistently#that#READ#deliverables#in#the#

domain#of#appraisal/evaluation#exceeded#expectations#and#requirements.#The#Balaklava#READ#

application#also#performed#well#against#Davidson’s#metaKevaluation#criteria#(2005):#validity,#utility,#

conduct#and#credibility#(see#Chapter#6).#On#assessing#cost,#the#final#criterion#from#Davidson’s#set,#no#

conclusion#was#possible#as#the#READ#application#did#not#follow#the#usual#scope#of#practice#(in#terms#

of#industry/practitioner#delivery)#and#conformed#to#academic#research#conventions.##


# 188#

Interview/consultation#data#and#the#forensic#reflective#practice#findings#revealed#that#in#addition#to#

the#instrumental#value#arising#from#the#READ#model#(in#terms#of#its#direct#use),#the#integrated#model#

had#conceptual#value,#which#changed#the#way#evaluation#was#practiced#and#generated#new#

understandings#of#the#interrelationships#between#evaluation,#creative#practice#and#program#

management.#One#interesting#observation#as#facilitator,#was#that#once#the#model#was#known#to#

stakeholders,#many#appeared#to#be#more#receptive#and#engaged#with#the#terminology#and#concept#

of#READ,#as#opposed#to#‘evaluation’.#In#taking#the#emphasis#off#‘evaluation’#as#terminology,#which#

came#with#previously#constructed#notions#of#the#practice,#the#diversified#READ#methodology#

developed#in#the#research#(and#the#acronym#itself)#changed#the#perception#and#nature#of#the#

evaluation#practice.#As#the#READ#facilitator#for#the#pending#summative#evaluation#of#the#Nuclear#

Futures#program#(across#many#other#projects),#this#is#an#important#finding,#which#will#be#influential#

for#future#evaluation#practice#at#the#site.#In#the#community#arts#context#more#broadly,#where#there#is#

evident#trepidation#of#evaluation#for#some#stakeholders,#this#finding#on#the#impact#of#READ’s#

alternative#rhetoric#for#appraisal#and#evaluation#is#also#of#significance.#

For#program#appraisal,#the#READ#application#involved#numerous#approaches#to#evaluative#thinking#

and#analysis,#and#this#increased#the#scope#of#what#was#included#as#evaluation#practice#and#the#

outcomes/outputs#this#produced.#The#use#of#creative#documentation#methods#for#example,#

supported#evaluative#processes#driven#by#an#accountability#agenda#while#utilising#moreKsubtle#

appraisal#processes#for#exploring#creative,#aesthetic,#cultural#and#social#questions.#Both#the#

Reflection#and#Documentation#strategies#of#the#READ#approach#played#a#particularly#important#role#

in#engaging#with#the#often#underKvalued#or#disregarded#tacit#systems#of#knowledge#generation#

inherent#in#community#arts#practice#and#program#management,#allowing#experimental,#artsKbased#or#

partnerKspecific#suggested#methods#to#be#explored.#While#these#strategies#tapped#into#tacit#

processes#that#were#already#being#employed#by#the#creative#team,#the#activities#appeared#to#be#

extended,#made#explicit,#and#shared#because#of#the#collective#READ#practice.#The#findings#revealed#

that#the#use#of#creative#documentation#positively#influenced#participation#and#engagement#in#

evaluation#practice,#and#legitimated#evaluative#activities,#particularly#for#the#creative#team.#Securing#

the#interest#and#participation#of#the#creative#team#in#the#evaluation#practice#was#important,#and#

meant#data#collection#became#a#component#of#creative#practice,#and#built#on#the#relationships#the#

artists#had#established#with#community#partners#through#the#artKmaking.##

The#findings#in#Chapter#6#also#elucidate#some#of#the#potential#issues#and#challenges#of#adopting#an#

integrated#approach#to#evaluation#practice,#and#identified#several#areas#of#the#formulated#READ#

approach#that#will#require#further#consideration.#Specifically,#the#potential#cost,#time#commitment#


# 189#

and#the#issue#of#managing#the#large#volume#of#data#generated#to#ensure#accessibility#and#utility#

present#real#challenges#for#using#the#fourKstrand#READ#model.#In#order#to#manage#the#concerns#

identified,#and#minimise#the#associated#risks#during#future#applications,#it#is#essential#that#the#scope#

and#goals#of#future#READ#applications#are#clearly#defined.#It#is#acknowledged#that#for#more#

sophisticated#and#demanding#applications,#such#as#the#pending#Nuclear#Futures#summative#

evaluation,#new#domains#of#evaluation#practice#may#need#to#be#explored#and#potentially#integrated,#

such#as#external#review#processes#as#a#check#for#validity#of#internal#findings,#and#the#possible#use#of#

indicators#and#metrics.##

Another#constraint#identified#in#the#research#is#that#the#READ#approach#and#methods#trialled#were#

part#of#a#contextKspecific#application#designed#in#response#to#the#unique#evaluation#needs#at#the#site#

of#practice.#It#cannot#be#claimed#that#the#integrated#READ#methodology#will#suit#all#contexts#and#

community#partnerships#in#community#arts,#and#this#is#an#accepted#limitation#of#the#model#for#

future#practice.#Applying#the#READ#model#as#a#guide#to#evaluation#practice#cannot#guarantee#

consistent#learning#or#appraisal#outcomes,#validity#of#findings,#credibility,#participation,#or#the#

delivery#of#a#socially#and#politically#just#means#for#data#collection,#analysis#and#management.#Any#

future#applications#of#the#model#will#be#different#to#the#case#studied#in#this#research,#and#will#be#

shaped#by#variables#such#as#stakeholder#needs#and#dynamics,#resource#allocation,#organisational#

capacity,#and#the#rigour#of#the#contextKspecific#methodological#design.#

Despite#such#limitations#however,#the#READ#approach#was#effective#in#engaging#with#the#

complexities#of#the#community#arts#practice,#rather#than#succumb#to#an#oversimplified#or#toolkit#

methodology.#As#a#learning#and#appraisal#framework#it#demonstrated#its#potential#for#shifting#the#

focus#beyond#an#instrumental#benefits#lens,#to#be#inclusive#of#multiple#learning#and#knowledge#

systems#which#is#important#when#working#with#culturally#diverse#partners,#as#is#the#case#for#Nuclear#

Futures.#Further,#the#research#demonstrated#that#the#integrated#READ#approach#achieved#an#

alignment#between#the#practices#of#evaluation#and#community#arts,#where#the#vibrancy#and#ethos#of#

the#arts#practice#became#exemplified#and#delivered#through#the#evaluation#practice.##

As#a#practitioner#negotiating#multiple#roles#within#the#complex#creative#community#partnership#

initiative,#the#research#produced#valuable#insights#into#how#to#strategically#integrate#the#

management,#READ#practice#and#artKmaking#of#the#program#to#produce#benefits#for#each#domain.#

The#result#for#the#Balaklava#application#of#READ#was#a#blurring#of#lines#between#these#three#domains#

at#the#site#of#practice,#and#I#would#argue#that#this#contributed#to#the#model’s#effectiveness.#The#

implication#for#my#own#practice#is#that#for#future#iterations#of#using#the#READ#approach#I#will#

prioritise#establishing#strategic#links#between#these#domains#as#a#core#part#of#practice.##


# 190#

I#refer#back#to#the#relationship#illustrated#in#Figure#5.1#between#READ,#program#management#and#

creative#practice:##

#

#

#

#

#

#

#

As#well#as#working#well#across#the#above#three#spheres#of#practice,#the#research#demonstrated#that#

the#READ#model#offers#a#flexible#evaluation#approach#that#can#be#used#as#an#integrated#or#modular#

framework,#and#it#is#this#attribute#of#the#model#that#increases#its#potential#relevance#for#the#varying#

contexts#within#community#arts.#This#means#that#individual#READ#strategies#can#be#employed#

independently,#or#in#tandem,#to#cater#for#particular#learning#and#appraisal#objectives,#or#

accommodate#stakeholderKpreferred#learning#modalities.#For#example,#a#solo#creative#practitioner#

wanting#to#understand#and/or#promote#their#arts#practice#for#the#purpose#of#improvement#or#

audience#development#could#apply#Reflection#and#Documentation#as#a#combined#solution.#The#

viability#and#broader#implications#of#using#the#READ#model#in#its#integrated#or#modular#capacity#was#

critically#assessed#during#the#reflective#analysis#and#metaKevaluation#of#the#Balaklava#application,#

and#was#identified#as#an#area#of#innovation#demonstrated#by#the#model#(see#Chapter#6).#

Overall,#the#READ#model#has#the#potential#to#present#as#both#a#simple#and/or#complex#system.#In#its#

Balaklava#iteration,#READ#was#ambitiously#designed#and#comprehensively#enacted.#The#convenient#

READ#acronym#offered#a#useful#mantra#for#those#participating,#a#shared#compass#for#the#collective#to#

guide#our#evaluation#practice#that#produced#outcomes#and#impact#beyond#the#expectation#of#the#

partner#collective.#While#it#is#beyond#the#scope#of#this#research#to#verify#whether#the#efforts#and#

input#employed#for#the#wideKranging#Balaklava#application#have#or#will#be#matched#by#the#benefits,#

in#my#position#as#dual#READ#facilitator/manager#I#can#see#that#the#benefits#of#the#READ#application#

will#be#of#value#to#Alphaville#(at#the#very#least)#for#years#to#come.#Another#question#arising#from#the#

research#is#about#READ’s#integrated#approach,#and#whether#the#outcomes#of#the#READ#system#as#a#

whole#add#up#to#more#than#the#sum#of#its#parts.#For#the#DCI,#with#the#limited#industry#application#

#


# 191#

conducted,#my#conclusion#is#that#it#does.#However,#to#what#extent#this#occurs#is#a#question#for#future#

research.##

In#closing#this#section,#I#present#a#list#made#during#the#forensic#reflective#practice#(a#kind#of#‘note#to#

self’#reflection)#of#the#key#lessons#and#conclusions#from#the#evaluation#practice#that#I#will#take#with#

me#as#READ#facilitator#for#use#in#the#next#round#of#Nuclear#Futures#evaluation#practice,#and#beyond.#

While#there#are#still#some#issues#to#improve,#and#further#applications#to#implement#and#analyse,#this#

list#was#my#summation#of#the#practical#lessons#I#drew#from#the#Balaklava#application:#

•! The#internal#position#meant#previous#knowledge#of#context#was#utilised,#and#advantages#

were#created#from#this.#

•! The#dual#position#of#READ#facilitator/manager#enabled#strategic#pathways#to#be#established#

and#maintained#between#READ,#creative#practice#and#program#management.##

•! Data#collection#and#analysis#could#be#spontaneous#and#reflexive,#as#well#as#

structured/planned.#

•! Access#to#data#in#realKtime#increased#the#capacity#of#both#positions,#and#in#turn#increased#

responsiveness#and#reflexivity,#and#utility#of#READ#or#program#management#data.#

•! The#socially#constructivist#interpretative#stance#was#an#effective#means#to#deal#with#the#

diverse#range#of#evaluation#constructs#and#values#present#for#stakeholders#and#partners.#

•! Scoping#was#fundamental#for#establishing#the#terms#of#reference#and#determining#partner#

capacity#and#interest#to#participate.#

•! If#negotiated#and#agreed#with#partners,#the#terms#of#reference#for#evaluation#practice#can#

extend#to#evaluative#and/or#nonKevaluative#processes#(in#terms#of#attributing#worth).#

•! The#creative#documentation#methods#were#of#high#relevance#to#the#project#and#collective,#

producing#a#range#of#benefits#including#increased#engagement,#ownership,#participation#and#

utility,#whereby#creative#processes#became#evaluation#practice.#

•! The#culture#of#reflection#that#developed#contributed#to#individual#and#organisational#

learning.#Particular#value#was#gained#internally#for#the#creative#team#through#sharing#and#

responding#to#reflections.#Data#was#often#of#immediate#use.#

•! Reflective#practice#performed#well#as#an#evaluator’s#tool.#

•! A#range#of#developmental#outcomes#was#evident,#including#organisational#learning#

(Alphaville#and#partners),#partnership#development,#professional#and#skills#development,#

and#program#development.#

•! Unlocking#the#academies#(by#partnering#researcher#institutions)#is#a#longerKterm#strategy#

that#added#credibility,#created#a#community#of#practice#and#added#value#for#community#


# 192#

partners,#participants#and#audiences#(see#Chapter#5:#Section#5.4.4#for#background#on#

unlocking#the#academies#concept).#

•! The#methodological#flexibility#of#the#model#was#important#in#a#changing#environment,#and#

for#responding#to#project#or#creative#practice#variations.#

•! Introducing#legacy#and#archival#objectives#to#the#evaluation#practice#has#the#potential#(if#

successful)#to#extend#the#impact#of#evaluation#for#social#justice#(speculative).#

What(innovation(is(offered(for(program(evaluation(and(practice?(

Based#on#the#DCI#findings,#innovation#was#identified#in#the#following#areas.#

First,#the#READ#model#demonstrated#innovation#in#the#Nuclear#Futures#application#and#this#was#

closely#linked#to#its#originality,#efficacy#and#appropriateness#of#the#model#in#meeting#the#

requirements#of#the#evaluation#context.#This#included#methodological#innovation#that#fulfilled#the#

information#and#resource#needs#demanded#at#the#site#of#practice.#The#systematic#and#tailored#

framework#designed#for#the#Balaklava#project#used#an#innovative#suite#of#methods#drawn#from#the#

latest#academic#literature#to#substantiate#the#robust#integration#of#all#four#READ#strategies.#The#

outputs#that#were#produced#fulfilled#the#shortKterm#evaluation#and#documentation#needs#of#

partners.#The#strength#of#READ#as#a#learning#framework#complemented#the#evaluative#activities,#and#

had#significant#outcomes#for#the#continuous#improvement#of#our#evaluation#approach#and#arts#

practice,#and#in#building#the#evaluation#capacity#at#the#collective#level.##

The#second#demonstration#of#innovation#is#the#READ#infographic,#a#stylised#visual#representation#of#

the#READ#model,#which#is#presented#in#Section#6.3.3.#The#infographic#is#an#original#and#creative#

solution#to#the#challenge#of#communicating#the#READ#approach,#and#will#be#of#immediate#relevance#

at#the#site#of#practice.#By#encapsulating#a#large#volume#of#descriptive#information#about#the#model,#I#

plan#to#use#the#infographic#as#a#communication#tool#for#the#broader#dissemination#of#information#

about#READ#as#an#emergent#evaluation#approach,#in#both#industry#and#academic#contexts.#

Third,#is#the#innovative#approach#of#the#READ#model#for#establishing#processes#aimed#at#generating#

longKterm#outcomes,#and#to#create#and#disseminate#outputs#in#a#way#that#sustains#relevance#and#

accessibility#beyond#the#project#timeframe#and#site#of#practice.#Examples#include#READ’s#approach#

to#creative#documentation,#the#intent#to#unlock#the#academies,#and#production#and#public#sharing#of#

legacy#artefacts.#While#it#is#too#early#to#verify#the#developed#model’s#effectiveness#in#delivering#the#

longerKterm#and#deep#future#goals,#the#READ#approach#has#demonstrated#promise#for#this#in#the#

community#arts#context.#One#known#outcome#from#the#trial#is#that#the#pursuit#of#broader#learning#

and#legacy#goals#does#extend#the#management#and#partner#brokerage#demands#of#the#evaluation#


# 193#

practice,#and#therefore#will#be#better#suited#to#mediumKtoKlong#term#community#arts#projects#rather#

than#shortKterm#activities.#

Fourth,#while#reflection,#evaluation,#analysis#and#documentation#are#known#and#used#in#varying#

ways#by#evaluators#and#creative#practitioners,#the#innovation#of#the#READ#model#lies#in#the#

integration#of#these#core#strategies#via#a#consolidated#framework.#The#model#brings#together#data#

generated#from#four#established#knowledgeKgenerating#and#appraisal#strategies#to#deliver#a#wide#

summative#lens#for#recording,#reviewing#and#representing#community#arts#activities.#Furthermore,#

the#model#has#the#flexibility#to#be#applied#as#an#integrated#or#modular#framework,#and#to#

accommodate#the#preferred#learning#modalities,#budgets#and#interests#of#stakeholders.#It#is#

imperative#here#to#note#that#in#the#READ#model,#reflection,#analysis#and#documentation#are#not#

merely#methods#of#an#evaluation#methodology,#but#are#independent#(yet#interKrelated)#learning#and#

appraisal#strategies#across#creative#practice,#management#and#evaluation#practice.#In#this#way,#the#

developed#READ#model#approaches#evaluation#as#part#of#an#integrated#practice.#

Lastly,#innovation#was#demonstrated#within#the#READ#strategies.#For#instance,#in#the#case#of#creative#

documentation,#the#READ#model#methodology#was#innovative#in#two#ways.#First,#it#drew#on#the#

strengths#of#the#creative#practice#of#the#project#and#second,#in#utilising#documentary#art#methods#it#

enabled#new#creative#content#and#artworks#to#be#generated#through#the#evaluation#practice.#Future#

utility#of#the#creative#content#is#already#planned#as#spinKoffs#beyond#the#Balaklava#project,#including#

plans#for#a#feature#length#documentary#film.#The#prospect#that#such#artefacts#can#be#generated#from#

the#combined#creative#and#evaluative#practice#of#READ#has#the#potential#to#raise#new#goals#for#CACD#

practitioners#interested#in#creating#longKterm#outcomes#through#community#arts#and#evaluation.##

7.3! Implications0and0significance0of0the0research0

In#this#section#I#examine#the#significance#of#the#research#and#findings#across#three#spheres#of#

evaluation#practice:#as#a#practitioner,#at#the#site#of#practice#and#for#the#industry#field.#

As(a(practitioner(

The#research#is#embedded#in,#and#furthers,#my#own#professional#evaluation#practice#advancing#my#

expertise#in#the#design#and#delivery#of#reflexive#learning#and#appraisal#systems#suited#to#the#creative#

arts#context.#The#DCI#offered#a#unique#opportunity#as#an#industry#practitioner#to#contextualise#and#

critically#examine#my#own#evaluation#practice.#Throughout#the#DCI#I#have#challenged#my#espoused#

assumptions#of#evaluation#and#experimented#with#new#approaches#and#methods.#I#have#also#

developed#my#theoretical#understanding#of#both#evaluation#and#program#management.#As#a#


# 194#

consequence#of#the#research#enquiry,#my#evaluation#practice#has#transformed#and#advanced#by#

utilising#the#process#of#continuous#improvement#via#forensic#reflective#practice.#The#DCI#coursework#

has#also#had#implications#for#my#practice#through#skills#development#in#transmedia#design#and#digital#

storytelling,#and#both#have#been#incorporated#into#the#creative#approach#at#the#site#of#practice.#Of#

immediate#significance#is#the#impact#that#the#research#has#had#on#building#my#capacity#and#expertise#

to#deliver#evaluation#for#Alphaville#and#the#Nuclear#Futures#program,#and#my#ability#to#convey#new#

approaches#to#others#at#an#academic#level.#Overall,#the#new#expertise#and#experience#gained,#and#

the#resulting#evaluation#theory,#will#impact#the#quality#and#scope#of#my#future#evaluation#practice.#

Site(of(practice(

The#research#has#generated#transformative#impacts#at#the#site#of#practice,#and#the#outputs#

developed#through#the#research#process#–#including#the#clarification#of#the#program#logic,#the#

evaluation#plan,#the#READ#product#development#and#the#design#of#evaluation#instruments#and#so#on#

–#have#produced#myriad#benefits#for#host#organisation#Alphaville,#and#for#the#Nuclear#Futures#

program#and#partners.#Essentially#the#findings#have#produced#a#set#of#established#protocols#for#

evaluation#at#the#site#of#practice,#and#a#working#knowledge#of#the#baseline#steps#involved#in#

implementing#the#READ#approach.#The#findings#also#establish#an#important#foundation#for#the#

further#development#of#innovative#evaluation#practices#across#the#Nuclear#Futures#program,#

including#the#identification#of#the#model’s#strengths#and#constraints#critical#for#the#program’s#

summative#evaluation#in#2017.#At#the#site#of#practice,#the#successful#READ#approach#created#impact#

through#an#enhanced#understanding#of#the#artistic,#theoretical,#academic#and#the#wider#social#

implications#of#the#community#arts#practice,#and#promoted#engagement,#acceptance#and#

participation#in#evaluation#by#mixed#stakeholder#groups.##

Feedback#collected#from#Alphaville#stated#that#there#has#been#direct#and#significant#value#for#the#

organisation#from#the#research#throughout#the#DCI#period.#The#research#and#findings#will#also#be#

directly#relevant#for#the#evaluation#approach#to#be#implemented#during#the#final#evaluation#of#the#

initiative.#As#well#as#meeting#the#evaluation#needs#at#the#site#of#practice,#the#READ#application#

delivered#capacity#building#and#a#broad#range#of#learning#outcomes#for#Alphaville.#In#particular,#the#

culture#of#reflection#generated#amongst#the#creative#team#has#and#will#continue#to#extend#across#

other#projects,#and#the#community#of#practice#will#result#in#valuable#learning,#dissemination#and#

promotional#outcomes#for#the#organisation.#For#community#partners,#the#research#has#produced#a#

mix#of#creative#and#informationKbased#resources#that#encapsulate#the#stories,#artworks#and#

testimonies#from#the#project,#which#has#implications#for#future#campaigning#and#advocacy#work.#


# 195#

Community(arts(evaluation(

While#the#DCI#findings#are#context#specific,#the#READ#product#developed#in#the#research#is#relevant#

for#practitioners#charged#with#the#task#of#evaluating#community#arts#practice#or#interested#in#

program#evaluation.#As#an#emergent#and#tested#approach,#the#developed#READ#model#contributes#

more#broadly#to#the#discourse#on#effective#approaches#to#community#arts#evaluation#and#offers#an#

original#and#relevant#model#of#evaluation#practice,#especially#in#the#domain#of#community#

partnerships.#For#practitioners,#the#research#speaks#to#individuals#such#as#myself,#who#traverse#

multiple#professional#roles#across#program#management,#creative#practice#and/or#evaluation.#In#

detailing#the#system’s#operations#and#nuances#of#the#READ#approach,#the#DCI#research#provides#

practical#guidance#for#an#integrated#approach#to#program#evaluation,#designed#for#multiple#and#

complex#creative#partnerships#initiatives.#In#drawing#from#the#documentary#art#traditions,#the#

research#also#demonstrates#the#potential#role#of#creative#practice#in#evaluation.#Although#arising#

from#the#community#arts#context,#the#research#has#implications#for#evaluation#practice#in#other#

fields,#including#the#creative#industries,#education#and#sustainability.#

In#addressing#identified#knowledge#gaps#on#effective#community#arts#evaluation,#the#research#

engages#with#the#complexities#of#evaluation#of#creative#partnership#initiatives,#delivering#an#original#

and#practical#contribution#that#will#have#impact#within#the#field.##For#the#field,#the#READ#model#

offers:#

•! an#integrated#evaluation#method#that#can#be#done#‘in#house’;#

•! a#model#of#evaluation#best#practice#that#embraces#the#ethos#of#community#arts;#•! a#flexible#framework#for#generating#robust#data,#that#is#linked#with#management#and#

creative#arts#practice#so#that#data#can#be#fed#back,#continuously,#into#the#project#for#realKtime#impact;#

•! original#strategies#for#engagement#and#for#developing#stakeholder#ownership#of#project#evaluation,#particularly#where#trepidation#or#resistance#to#traditional#evaluation#means#exists;#

•! possibilities#for#more#general#application#in#(researchKled)#creative#management.#

When#framed#within#the#broader#discourse#on#cultural#value#currently#unfolding#in#the#field,#the#

research#advances#the#pursuit#of#alternative#approaches#to#the#current#orthodoxy#of#topKdown,#

funding#agency#oriented#evaluation.#As#discussed#in#Chapter#2,#scholars#such#as#Goldbard#(2008,#

2015),#Belfiore#(2014)#and#Badham#(2015)#concerned#by#the#current#trends#and#blinkered#

approaches#to#community#arts#evaluation,#call#for#a#new#era#of#evaluation#practice#that#can#embrace#

wider#notions#of#cultural#value#and#operate#outside#of#the#constraints#of#quantitative#impact#

measurement#alone.#The#READ#model,#as#developed#in#this#research,#responds#to#such#appeals.#The#

application#of#evaluation#as#part#of#an#integrated#approach#to#learning#and#appraisal#is#inclusive#of,#


# 196#

yet#not#limited#to,#meritKattribution#as#a#lens#for#understanding#and#representing#community#arts#

practice#and#impact.#In#addition,#the#developed#READ#model#embraces#practice#and#partnerKled#

knowledge#and#sensemaking#arising#from#creative#practice,#collective#reflection,#research#for#

broader#learning,#and#legacyKdriven#activities,#and#in#doing#so,#offers#new#rhetoric,#conceptualisation#

and#scope#for#evaluation#practice#in#community#arts.#

7.4! Dissemination0

The#two#audiences#that#dissemination#is#directed#to#are#industry#practitioners#(in#community#arts#

and#beyond)#and#academic#scholars#in#community#arts#and#evaluation.#Previous#and#futures#plans#

for#dissemination#are#provided#below.##

Academic((

Dissemination#of#early#findings#during#the#DCI#has#included:#

•! Conference#presentation:#Harnessing#evaluation#through#integration:#A#participatory#model#

for#Reflection,#Evaluation,#Analysis#and#Documentation#(the#‘READ’#model)#in#CommunityJ

based#Arts#by#E.#Barkley#at#the#Australasian#Evaluation#Society#Conference,#Darwin#10K12#September,#2014.#

•! Seminar#presentation:#Approaches#for#Reflection,#Evaluation,#Analysis#and#Documentation#by#E.#Barkley#at#Creative#Arts#and#Nuclear#Futures#Academic#Seminar,#The#Glasshouse,#QUT,#7#August#2015.##

Other#academic#events#and#forums#in#which#research#was#disseminated#with#industry#peers#and#

networks#include:#

•! Evaluation#workshop#to#CSIRO#social#research#teams#(as#part#of#a#professional#development#day)#in#Brisbane,#December#2013.#

•! Sharing#of#research#at#the#masterclass#for#postgraduate#students#prior#to#the#Spectres#of#Evaluation#conference#in#Footscray,#February#2014#

•! Networking#and#discussion#with#members#of#international#community#of#community#arts#workers#and#evaluators,#at#Spectres#of#Evaluation#conference#in#Footscray,#February#2014.#

#

Future#dissemination#opportunities#will#be#created#through#the#publication#of#papers#within#targeted#

journals.#One#example#is#a#planned#paper#for#submission#to#the#refereed#Journal#of#Arts#and#

Communities,#which#has#a#relevant#and#international#audience.#Another#example#is#the#planned#eK

book#discussed#below.#

#


# 197#

Industry(dissemination((

Dissemination#of#selected#research#findings#has#been#undertaken#with#Nuclear#Futures#stakeholders/#

partners#(for#example,#via#partner#summaries#and#presentations).#During#the#research#I#have#

communicated#key#findings#at#the#site#of#practice#for#the#purpose#of#promoting#understanding#about#

the#READ#practice#(during#and#following#the#READ#application#research)#and#for#inspiring#action.#

Dissemination#to#the#Nuclear#Futures#partnership#and#stakeholder#audience#is#important#for#our#

continued#work#on#the#model#and#for#improving#practice#for#the#summative#application#of#READ#for#

the#whole#program.#Sharing#the#findings#and#examples#of#good#practice#of#the#READ#application#

from#the#Balaklava#experience#provides#information#and#helps#spread#ideas#that#are#relevant#beyond#

the#site#of#practice.#The#dissemination#methods#utilised#include#meeting#forums,#online#information#

via#the#Nuclear#Futures#website#and#partner#summaries#at#the#completion#of#the#research.#Given#the#

collaborative#process#involved#in#this#DCI#research,#it#will#be#important#to#deliver#results#back#to#the#

participants#and#stakeholders.#While#findings#from#READ#have#been#regularly#reported#to#the#

Australia#Council#for#the#Arts,#during#the#project,#a#more#comprehensive#presentation#on#the#

developed#READ#model#and#findings#is#planned#following#the#summative#evaluation#of#Nuclear#

Futures#in#2017.#

Beyond#the#dissemination#at#my#site#of#practice,#research#findings#have#also#been#circulated#with#

selected#industry#peers.#Dissemination#of#the#research#to#date#in#the#industry#arena#has#received#

positive#external#feedback#from#affiliated#industry#networks#and#peak#funding#agencies.##

eKbook(

For#future#communication#and#dissemination#of#the#research#I#plan#to#produce#and#8K10#page#eKbook#

that#introduces#and#describes#the#READ#model#and#approach.#The#publication#will#be#primarily#aimed#

at#creative#practitioners#and#evaluators.#Included#will#be#the#infographic#and#a#basic#‘how#to’#guide#

for#applying#the#integrated#approach.#

7.5! Future0research0

As#the#application#of#READ#is#relatively#recent#and#there#is#no#other#data#published#about#the#READ#

model,#it#is#important#to#expand#knowledge#on#this#subject.#Future#research#would#benefit#in#

developing#an#understanding#of#how#the#READ#model#can#best#be#designed#and#implemented#in#

other#community#arts#settings,#and#to#what#effect.#Subsequent#research#could#also#be#a#means#to#

innovate#the#methodological#designs#and#practices,#and#further#refine#the#systems#and#practice#of#

READ.#Keeping#pace#with#the#pending#Nuclear#Futures#applications#of#the#READ#model#it#would#be#of#

interest#to#examine#the#model#in#the#international#context.#Likewise,#application#in#other#fields#would#


# 198#

also#provide#insight#into#the#functioning#and#potential#versatility#of#the#developed#evaluation#

approach.##

At#the#site#of#practice,#research#will#continue#into#the#READ#model’s#applications#across#the#Nuclear#

Futures#program,#with#the#research#questions#and#analysis#being#framed#across#a#number#of#

research#areas.#For#example,#in#a#planned#book#Australian#Art#on#the#Nuclear#Age:#Land,#Peoples,#

and#their#Futures,#to#be#written#by#Nuclear#Futures#program#collaborators,#I#will#examine#the#wider#

implications#of#the#program#revealed#by#the#READ#findings,#focusing#in#particular#on#the#legacy#

objectives#and#methods.#The#chapter#is#tentatively#titled#‘Making#the#story#go#far’.#The#second#

chapter,#which#I#will#coKauthor#with#projection#designer#Nic#Mollison,#has#the#prospective#title#

‘Staging#the#bomb’.#This#chapter#is#set#to#explore#the#implications#of#the#production#design#of#the#

immersive#projection#installation#developed#for#the#Balaklava#showcase,#and#investigates#the#role#

the#installation’s#innovative#audioKvisual#technologies#played#in#enhancing#the#community#arts#

practice#and#transforming#our#capacity#to#engage#with#community#about#the#nuclear#age.##

Finally,#there#would#be#benefits#derived#from#investigating#READ#to#understand#its#role#in#

organisational#learning#and#program#management,#without#evaluation#being#the#central#focus#of#the#

study#as#it#has#been#in#this#research.#Future#research#across#all#of#the#above#mentioned#areas#will#

increase#the#depth#of#details#and#descriptions#for#using#the#developed#READ#approach#across#varied#

contexts#and#for#many#potential#purposes.#

The#following#section#presents#the#overall#dissertation#conclusion.#

# #


# 199#

7.6! Dissertation0conclusion0

This#Doctor#of#Creative#Industries#(DCI)#research#has#focused#on#the#development,#operational#use,#

and#critical#review#of#the#READ#model#K#a#dynamic#and#rigorous#model#for#community#arts#evaluation.#

Designed#for#the#community#arts#context,#the#READ#model#extends#beyond#conventional#approaches#

to#integrate#Reflection,#Evaluation,#Analysis#and#Documentation#(READ),#to#create#a#wide#summative#

lens#for#community#arts#practice.#The#READ#approach#was#collaboratively#devised#at#the#site#of#my#

professional#practice#with#Alphaville#arts#company.#Prior#to#this#research,#the#proposed#READ#model#

existed#in#concept#form#only,#and#no#proof#of#concept#or#methodology#had#been#established.##

Through#the#DCI#my#role#as#researcher#has#involved#driving#the#twoKyear#process#of#developing#the#

READ#concept#into#to#a#substantiated,#working#model#in#the#industry#context.#I#have#achieved#this#by#

engaging#with#community#arts#partners#and#participants,#leading#collective#design#and#delivery#

processes,#articulating#and#refining#the#model,#and#evaluating#the#developed#READ#model#in#action.#

The#body#of#evidence#presented#in#this#dissertation#represents#the#first#iteration#of#critical#analysis#

on#the#model#during#the#initial#phase#of#the#Nuclear#Futures#Partnership#Initiative#(2013#K#2015),#an#

international#arts#program#funded#by#the#Australia#Council#for#the#Arts#(2014#K#2016)#and#the#site#of#

my#professional#practice.#Resulting#from#the#DCI#research#is#a#comprehensive#and#integrated#model#

of#practice#for#community#arts#evaluation,#as#well#as#a#practitionerKoriented#READ#product#for#future#

practice,#of#broader#relevance#to#the#field,#suited#to#nonKexperts#and#experts#in#evaluation.#

The#enquiry#fits#within#a#tradition#of#artists,#managers#and#evaluators#seeking#to#address#the#

persistent#epistemological#and#technical#problems#raised#by#evaluation,#in#the#face#of#rising#industry#

demands#to#demonstrate#the#value#of#community#arts#practice.#Within#the#field,#the#research#is#

positioned#within#the#unfolding#debate#on#cultural#value#and#measurement,#and#contributes#to#the#

discourse#on#effective#models#of#practice#for#community#arts#evaluation.#This#research#establishes#a#

verified#alternative#to#the#progressively#measurementKoriented#and#quantitative#approaches#gaining#

traction#in#the#field,#which#favour#economic#and#social#conceptualisations#of#cultural#value#and#lead#

to#the#further#instrumentalisation#of#the#arts.#Demonstrated#in#this#dissertation,#and#the#

accompanying#READ#in#Practice#portfolio,#are#the#diverse#accomplishments#of#the#wideKlens#READ#

approach#that#integrates#evaluation#as#part#of#a#broader#methodology#for#learning#and#appraisal.##

To#study#the#design,#efficacy#and#potential#of#the#READ#model#developed#in#this#research,#I#utilised#

forensic#reflective#practice#as#my#principal#research#strategy#for#the#DCI.#The#forensic#reflective#

practice#was#knowledgeKdriven,#involving#systematic#examination#and#critical#enquiry#of#the#model#

devised.#The#research#design#also#utilised#structured#and#reflexive#qualitative#methods,#including#

written#and#video#reflection,#semiKstructured#interviews,#digital#storytelling#and#observation.#The#


# 200#

qualitative#research#methodology#elicited#rich#data#and#siteKspecific#learnings.#Of#significance#were#a#

number#of#key#findings#on#the#emergent#evaluation#practice#and#the#consolidated#READ#model.#

The#READ#model#developed#in#the#DCI#research#operated#as#a#flexible#and#robust#framework#that#

harnessed#evaluation#and#increased#the#scope#for#learning#and#appraisal#via#its#integrated#approach.#

The#READ#methodology#tailored#for#the#Balaklava#Community#Arts#Project#addressed#stakeholder#

identified#needs,#as#well#as#facilitated#a#collective#process#of#sensemaking#about#the#creative#

practice#and#broader#implications#of#the#community#arts.#The#mix#of#evaluative#and#nonKevaluative#

methods#utilised#across#the#READ#strategies#diversified#the#opportunities#for#data#collection,#

engagement#and#participation,#and#generated#rigorous#and#credible#findings#for#the#context#and#

needs#of#the#evaluation#process.#It#was#identified#that#the#READ#model#fulfilled#a#range#of#purposes,#

including#accountability,#improvement,#broader#learning#and#instituting#legacy#outcomes.##

In#terms#of#utility,#the#multiKplatform#portfolio#of#READ#outputs#produced#during#the#Balaklava#

application#were#highly#valued#by#stakeholders.#Immediate#and#longKterm#benefits#resulted#for#host#

organisation#Alphaville#and#for#others#within#the#partnerships.#The#findings#revealed#that#the#

developed#READ#approach#generated#transformative#impacts#at#the#Nuclear#Futures#site#of#practice#

and#advanced#evaluation#practice#for#the#collective.#In#shifting#the#focus#beyond#a#conventional#

evaluationKonly#lens,#and#through#the#incorporation#of#reflective#and#creative#methods#closely#linked#

into#the#arts#practice,#the#vibrancy#and#ethos#of#the#community#arts#practice#was#exemplified#in#the#

evaluation#practice.#Furthermore,#for#some#stakeholders#harbouring#trepidation#of#conventional#

evaluation,#the#diversified#READ#approach#allowed#prior#evaluation#constructs#to#be#challenged#and#

renegotiated,#so#that#evaluative#processes#and#activities#were#contextualised#and#legitimated#as#part#

of#an#accepted#broader#framework#for#artKmaking,#learning#and#appraisal#of#value#to#all#partners.##

The#research#demonstrates#that#the#innovation#of#the#emergent#READ#model#of#practice#lies#in#the#

integration#of#the#four#key#appraisal#and#learning#strategiesK#Reflection,#Evaluation,#Analysis#and#

Documentation.#For#the#field,#the#tested#READ#model#arising#from#this#research#offers:#

•! an#integrated#evaluation#method#that#can#be#done#‘in#house’;#

•! a#model#of#evaluation#best#practice#that#embraces#the#ethos#of#community#arts;#

•! a#flexible#framework#for#generating#robust#data,#that#is#linked#with#management#and#creative#arts#practice#so#that#data#can#be#fed#back,#continuously,#into#the#project#for#realKtime#impact;#

•! original#strategies#for#engagement#and#for#developing#stakeholder#ownership#of#project#evaluation,#particularly#where#trepidation/#resistance#to#traditional#evaluation#means#exists;#

•! possibilities#for#more#general#application#in#(researchKled)#creative#management.#


# 201#

Specific#areas#demonstrating#innovation#in#evaluation#practice#were#evidenced,#including#the#use#of#

creative#documentation#and#original#creative#work#for#evaluation#purposes,#as#well#as#the#utilisation#

of#long#term#goals#and#broader#learning#outcomes#to#enhance#the#benefits#returned#to#community#

partners.#The#prospect#of#the#developed#READ#approach#being#able#to#address#longKterm#objectives#

arising#from#creative#practice#has#the#potential#to#raise#new#goals#for#CACD#practitioners#interested#

in#creating#diverse#legacy#outcomes#through#community#arts.#In#addition,#the#model#demonstrated#a#

capacity#to#produce#organisational#and#broader#learning#outcomes#through#the#established#

community#of#practice#so#as#to#‘unlock#the#academies’,#and#via#a#sustained#culture#of#reflection#by#

artists#and#community#partners#K#that#has#carried#on#across#subsequent#Nuclear#Futures#projects.#

Methodologically,#the#READ#model#demonstrates#innovation#as#an#original,#versatile#and#adaptable#

framework,#able#to#be#used#by#practitioners#as#either#an#integrated#or#modular#evaluation#approach.#

One#result#of#integrating#the#four#established#strategies#(Reflection,#Evaluation,#Analysis#and#

Documentation)#was#that#the#frame#for#evaluation#practice#was#widened#at#the#site#of#practice#and#

this#extended#READ’s#applicability#as#a#knowledgeKdriven#framework#for#learning#and#appraisal.#The#

broader#implication#is#that#the#integrated#approach#offers#the#model#a#reflexive#edge#in#meeting#

multiple#needs#as#defined#by#the#context,#partnerships#and#arts#practices#under#review.#It#also#

means#that#the#scale#and#scope#of#future#READ#applications#can#range#greatly,#from#creative#

practitioners#adapting#the#READ#approach#for#their#individual#arts#practice#to#application#by#an#

evaluation#team#across#large#complex#multiKarts#creative#partnerships.#The#capacity#of#the#model#to#

be#inclusive#of#different#learning#approaches#and#knowledge#paradigms#is#also#significant,#given#the#

role#evaluation#practice#plays#in#brokering#partnerships,#which#are#often#culturally#diverse.##

In#the#dissertation,#the#impact#of#the#research#is#evidenced#across#three#spheres#of#evaluation#

practice:#as#a#practitioner,#at#the#site#of#practice#and#for#the#industry#field.#In#my#own#professional#

evaluation#practice,#the#research#has#advanced#my#expertise#in#the#design#and#delivery#of#reflexive#

learning#and#appraisal#systems#suited#to#the#creative#arts#context#and#provides#a#valuable#evaluation#

tool#for#future#practice.#At#the#site#of#practice,#the#evaluation#practice#and#READ#model#developed#in#

the#research#created#impact#by#generating#an#enhanced#understanding#of#the#artistic,#theoretical,#

academic#and#wider#social#implications#of#the#community#arts#practice.#The#research#findings#also#

establish#an#important#foundation#for#the#further#development#of#innovative#evaluation#practices#

across#the#Nuclear#Futures#program,#including#the#identification#of#the#model’s#strengths#and#

constraints#critical#for#the#program’s#summative#evaluation#in#2017.#For#the#field,#the#DCI#research#

provides#a#timely#contribution#to#the#discourse#on#effective#evaluation#of#community#arts#by#

demonstrating#a#number#of#original#developments#in#evaluation#practice#that#respond#to#calls#from#

within#the#field#to#diversify#approaches#to#community#arts#evaluation.#Practical#insights#into#an#


# 202#

integrated#stakeholderKoriented#approach#to#evaluation#are#revealed,#as#well#as#demonstration#of#

how#documentary#art,#collective#reflection,#community#archiving,#and#academic#enquiry#can#offer#

adept#methods#for#understanding,#appraising#and#representing#community#arts#practice.##

In#summing#up,#the#findings#within#this#dissertation#identify#a#number#of#key#areas#of#innovation#for#

evaluation#practice,#particularly#for#the#evaluation#of#complex#creative#partnershipKoriented#

programs.#The#READ#model#developed#and#presented#in#the#DCI#demonstrates#innovation#in#the#

originality,#efficacy#and#appropriateness#of#the#evaluation#methods#and#conduct#that#it#brought#to#

delivering#the#evaluation#needs#at#the#site#of#practice.#Of#particular#interest#were#the#benefits#

resulting#from#the#use#of#creative#documentation#and#the#inclusion#of#legacy#goals#as#a#longKterm#

purpose#for#evaluation.#Both#of#these#examples#drew#on#the#strengths#of#the#creative#practice#and#

partners#of#the#project,#and#extended#the#scope#of#evaluation#practice,#resulting#in#original#artefacts#

and#creative#works#that#are#of#relevance#to#partners#beyond#the#evaluation#phase.#As#a#body#of#

work,#the#DCI#findings,#the#developed#READ#model,#and#the#consolidated#READ#product,#contribute#

original#knowledge#to#the#field#and#to#the#discourse#on#the#effective#evaluation#of#community#arts#

and#impact#measurement.#For#my#own#professional#practice,#the#research#has#transformed#my#

approach#to,#and#definition#of,#evaluation,#and#elucidated#the#benefits#of#cultivating#interKlinked#

opportunities#for#learning#and#appraisal#across#the#domains#of#creative#practice,#management,#and#

evaluation.#At#the#site#of#practice,#the#research#represents#a#genuine#effort#to#engage#with#the#

complexities#of#community#arts#evaluation#in#a#practical#way,#and#has#resulted#in#improved#

evaluation#practice#and#the#establishment#of#protocols#for#the#future#use#of#the#READ#model.#

Beyond#the#site#of#practice,#the#findings#have#broader#relevance#and#will#be#of#interest#to#scholars#

and#practitioners#in#community#arts,#the#creative#industries,#and#other#fields#with#an#interest#in#

program#evaluation,#artsKbased#and/or#stakeholderKoriented#evaluation,#community#engagement,#

organisational#learning,#and#program#management#more#generally.#

#

*************************#


# 203#

Appendix#1## Interview#design#for#scoping#interviews###Example:0Interview0Design0Schedule0

Code0 Code0category0 Aim0of0Data0 Questions0to0generate0data0 Relevance0Partner0000000Industry0

Notes0for0analysis0

StakHs0 Stakeholder0profiles0

Gather0data0on0background0of0participant0and0positioning0within0CBA0stakeholder0groupings0

Could0you0please0briefly0describe0the0field0that0you0work0in0and0the0type0of0work0you0do?0(i.e.0your0current0role)0What0 type0 of0 organization0 do0 you0 work0for?0

*0 *0 Participants0to0be0analysed0in0terms0of0stakeholder0group0and0as0individuals.0Analysis0may0be0relevant0relating0to0years0of0experience0and0evaluation0lessons0etc.0

How0long0have0you0worked0in0this0area?0 *0 *0

EvUse0 Evaluation0Uses0 Gather0a0multiple0perspective0snapshot0of0evaluation0uses0and0dissemination0

What0 is0 your0 experience0 in0 participating0in,0 implementing,0 reviewing0 or0 using0evaluation?0

*0 *0 Provides0a0gauge0to0participant’s0experience0with0evaluation0

Are0you0involved0with0evaluation0practice0in0your0current0role?00If0so,0what0type0of0evaluative0activities0do0you0do?0If0not,0how0does0your0organization0use0or0implement0evaluation?0

*0 *0 Analysis0on0dominant0evaluation0uses0and0dissemination0relevant0to0participants0

Does0 your0 organization0 (or0 yourself)0disseminate0 or0 share0 evaluation0 findings0to0other0practitioners0or0organisations?0If0so,0how0and0why?0

*0 *0

EvDef0 Evaluation0Definitions0

Gather0a0multiple0perspective0snapshot0of0evaluation0definitions0

How0do0you0define0evaluation?0What0is0the0purpose0of0evaluation?0

*0 *0 Analysis0aims0to0identify0the0dominant0underlying0values,0assumptions0and0constructs0of0CBA0evaluation0by0key0stakeholder0groups0to0provide0insight0into0the0partnership0context0of0CCPI0and0CBA.0

Does0 this0 definition0 reflect0 how0 your0organization0defines0evaluation?0

*0 *0 Examine0if0there0is0conflict0or0difference0in0personal0vs0organizational0agendas.0

EvVal0 Evaluation0Value0

Identify0what0participants0value0about0evaluation00

What0 is0 your0 personal0 opinion0 of0 the0value0of0evaluation?0

*0 *0 Analysis0of0dominant0uses,0values0and0barriers0of0CBA0evaluation0by0key0stakeholder0groups0to0provide0insight0into0the0functionality0points0for0NFs0evaluation0design00

How0 does0 your0 organization0 view0evaluation0practice0do0you0think?0

*0 *0

What0are0some0of0the0negative0aspects0of0evaluation?0

*0 *0

Beyond0 the0 evaluation0 findings0themselves,0 what0 other0 benefits0 does0evaluation0offer,0if0any?0

0 *0


# 204#

Code0 Code0category0 Aim0of0Data0 Questions0to0generate0data0 Relevance0Partner0000000Industry0

Notes0for0analysis0

EvCBA0 Evaluation0Community)based0Arts0

Gather0a0multiple0perspective0snapshot0of0evaluation0within0the0Community)based0arts0context;00Identify0what0principles0or0factors0make0for0‘good’0evaluation0

Do0you0think0 it0 is0possible0 to0successfully0evaluate0 Community)based0 Arts0 projects0or0 initiatives0 (or0 CCPI0 or0 relevant0language)?0

*0 *0 Analysis0of0context0specific0issues0for0undertaking0evaluation,0may0be0approached0by0stakeholder0groupings0or0theme0based0analysis.0

What0do0 you0 consider0 to0be0 the0barriers0to0 successful0 evaluation0 in0 Community)based0Arts?0

*0 *0

What0 aspects0 of0 community)based0 arts0are0 important0 to0 ‘measure’0 or0 evaluate?00Why?0

*0 *0

Are0 evaluation0 reports0 from0 other0Community)based0 Arts0 projects0 of0interest0to0you?00Do0you0get0 an0opportunity0 to0 learn0 from0other0projects0via0evaluations?0

0 *0

What0do0you0think0are0key0elements0that0you0 think0 makes0 a0 ‘good’0 evaluation0 in0the0Community)based0Arts0context?0

*0 *0 Analysis0for0building0criteria0for0successful0design0(can0be0analysed0in0terms0of0meta)evaluation0scholarship)0

PartEv0 Participatory0Evaluation0

Identify0dominant0definitions0and0levels0of0experience/0familiarity0of0participants0

Have0 you0 heard0 of0 ‘participatory0evaluation?’00If0 so,0 what0 is0 your0 understanding0 of0 this0approach?0If0not,0define0participatory0evaluation0

*0 *0 Analysis0of0key0features0or0examples0provided0of0participatory0approaches0and0levels0of0experience0with0participatory0approaches0by0partners0

Have0 you0 been0 involved0 in0 evaluation0before0 that0 used0 participatory0approaches?0If0 so,0what0were0 the0 key0 benefits0 of0 this0type0of0approach?0If0 so,0 what0 were0 the0 issues0 or0disadvantages0 arising0 from0 using0 a0participatory0approach?0

*0 *0 Looking0for0evidence/0links0between0participatory0approaches0and0benefits0(or0potential0benefits)0to0the0program0that0will0assist0in00theory0development0on0participatory0approaches0given0constraints0of0CBA0projects0

What0do0you0think0the0benefit0might0be0for0using0a0participatory0approach0for0a0program0like0NFs?00Is0there0any0evidence0of0this0so0far?0

*0 0

What0do0you0think0the0benefit0might0be0for0using0a0participatory0approach0for0Community)based0arts0or0CCPI0programs?0Do0you0know0of0examples0or0case0studies0where0this0has0been0the0case?0

0 *0 Analysis0of0potential0strengths0and0constraints0of0participatory0approaches0for0particular0elements0or0complexities0within0the0CCPI0case0study0

0


# 205#

Bibliography0

Alkin,#M.#C.#(2004).#ContextKadapted#utilization:#A#personal#journey.#In#M.#C.#Alkin#(Ed.),#Evaluation#Roots:#Tracing#theorist’s#view#and#influences#(pp.#293–303).#Thousand#Oaks,#CA:#Sage.#

Alphaville.#(2015).#Nuclear#Futures:#Exposing#the#legacies#of#the#atomic#age#through#creative#arts.#[Website].#Retrieved#February#13,#2015,#from#http://nuclearfutures.org.#

Alvesson,#M.#&#Skoldberg,#K.#(1999).#Reflexive#Methodology:#new#vistas#for#qualitative#research.#London:#Sage.##

Anderson,#L.#W.,#&#Postlethwaite,#T.#N.#(2007).#Program#evaluation:#LargeKscale#and#smallKscale#studies.#International#Institute#for#Educational#Planning#Series#Booklet,#(8).#

Archer,#B.#(1995).#The#Nature#of#Research.#CoJdesign,#January#issue,#pp.#6K13.#[Paper#transcribed#from#a#photocopy#of#the#original#by#Chris#Rust#on#17.1.09.]#Retrieved#April#20,#2014#from#http://www.metu.edu.tr/~baykan/arch586/Archer95.pdf#(no#longer#available).#See#alternative#link,#https://archive.org/stream/TheNatureOfResearch/Archer1995Codesign#page/n0/mode/2up##

Arvanitakis,#J.#(2008).#Staging#Maralinga#and#Looking#for#Community#(Or#Why#We#Must#Desire#Community#Before#We#Can#Find#It).#Research#in#Drama#Education,#13(3),#295K306.#

Australasian#Evaluation#Society#(AES).#(2010).#Guidelines#for#the#Ethical#Conduct#of#Evaluations.#[Booklet].#Retrieved#May#20,#2013#from#http://www.aes.asn.au/images/stories/files/membership/AES_Guidelines_web.pdf##

The#Australia#Business#Arts#Foundation#(AbaF)#&#Australia#Council#for#the#Arts.#(2010).#Arts#and#business:#partnerships#that#work#[Factsheet].#Sydney.#Retrieved#May,#5,#2013#from#

http://www.creativepartnershipsaustralia.org.au#(no#longer#available).#See#full#report,#http://apo.org.au/resource/artsKandKbusinessKpartnershipsKwork##

Australia#Council#for#Arts.#(2011).#Community#Partnerships#sector#plan#2010J2012#for#community#arts#and#

cultural#development#(revised#version#May#2011).#Retrieved#May,#15,#2013#from#http://www.australiacouncil.gov.au/__data/assets/pdf_file/0018/103482/COMMUNITY_PARTNERSHIPS_2011_Sector_Plan.pdf#(no#longer#available).#See#alternative#link,#http://static.placestories.com/pool/project/0001/0008319/docs/docK2Z.pdf##

Australian#Government.#(2000).##Does#Your#Project#Make#a#Difference?#A#Guide#to#evaluating#environmental#education#

projects#and#programs.#Sydney:#Department#of#Environment#and#Conservation#(NSW).#

Babbie,#E.#(1992).#The#Practice#of#Social#Research#(6th#ed.).#Belmont:#Wadsworth#Publishing#Company.#

Badham,#M.#(2010).#The#case#for#‘socially#engaged#arts’:#navigating#art#history,#cultural#development#and#arts#funding#narratives.#Local#Global:#Identity,#Security,#Community,#7,#84K99.#

Badham,#M.#(2015).#Democratising#Cultural#Indicators:#Developing#a#Shared#Sense#of#Progress.#In#L.MacDowall,#M.Badham,#E.Blomkamp,#&#K.Dunphy#(Eds.).#Making#Culture#Count:#The#Politics#of#

Cultural#Measurement#(pp.195K213).#London:#Palgrave#Macmillan.##

Bambino,#D.#(2002).#Critical#friends.#Educational#leadership,#59(6),#25K27.#

Becker,#H.#S.#(1996).#The#Epistemology#of#Qualitative#Research.#In#R.Jessor,#A.Colby#&#R.#A.Shweder#(Eds.),#Ethnography#and#Human#Development:#Context#and#Meaning#in#Social#Inquiry#(p.530).#Chicago:#University#of#Chicago#Press.##

Belenky,#M.,#Clinchy,#B.,#Goldberger,#N.,#and#Tarule,#J.#(1986).#Women’s#Ways#of#Knowing.#New#York:#Basic#Books.#


# 206#

Belfiore,#E.#(2014).#‘Impact’,#‘value’#and#‘bad#economics’:#Making#sense#of#the#problem#of#value#in#the#arts#and#humanities.#Arts#&#Humanities#in#Higher#Education,#14(1),#95–110.#

Belfiore,#E.#&#Bennett,#O.#(2008).#The#Social#Impact#of#the#Arts:#An#Intellectual#History.#New#York:#Palgrave#Macmilllan.#

Belfiore,#E.#&#Bennett,#O.#(2010).#Beyond#the#Toolkit#Approach:#Arts#Impact#Evaluation#Research#and#the#realities#of#Cultural#PolicyKmaking.#Journal#for#Cultural#Research#14,#2#(2010),#121K142#from#http://dx.doi.org/10.1080/14797580903481280.#

BetterEvaluation.#(2013).#Rainbow#Framework#Overview.#[website]#Retrieved#September#4,#2013#from:#www.betterevaluation.org/plan###

Big#hART.#(2010).#Ngapartji#Ngapartji#Memory#Basket.#[Boxed#creative#documentation#with#information#and#artefact#resources].#Available#via#Big#hART#organisation.#

Blichfeldt,#B.#S.,#&#Andersen,#J.#R.#(2006).#Creating#a#wider#audience#for#action#research:#Learning#from#caseKstudy#research.#Journal#of#Research#Practice,#2(1),#ArticleKD2.#

Blomkamp,#E.#(2015).#A#Critical#History#of#Cultural#Indicators.#In#L.MacDowall,#M.Badham,#E.Blomkamp,#&#K.Dunphy#(Eds.).#Making#Culture#Count:#The#Politics#of#Cultural#Measurement#(pp.11K26).#London:#Palgrave#Macmillan.##

Bolton,#G.#(2010).##Reflective#Practice:#Writing#&#Professional#Development#(3rd#ed.).#London:#Sage.##

Boud,#D.#(2001).#Using#Journal#Writing#to#Enhance#Reflective#Practice.#New#Directions#For#Adult#and#Continuing#Education,#90#(Summer),#9K17.##

Boud,#D.,#&#Walker,#D.#(1998).#Promoting#reflection#in#professional#courses:#The#challenge#of#context.#Studies#in#higher#education,#23(2),#191K206.#

Bourner,#T.#(2003).#Assessing#reflective#learning.#Education+#Training,#45(5),#267K272.#

Braun,#V.,#&#Clarke,#V.#(2006).#Using#thematic#analysis#in#psychology.#Qualitative#research#in#psychology,#3(2),#77K101.#

Brown,#P.F.#(2006).#Maralinga:#Theatre#from#a#Place#of#War.#In#G.McAuley#(Ed.),#Ground#Work:#The#

Politics#of#Place#and#Performance.#Berne:#Peter#Lang.#

Brown,#P.F.#(2007).#British#nuclear#testing#in#Australia:#Performing#the#Maralinga#Experiment#through#verbatim#theatre.#Journal#of#the#Royal#Society#of#New#South#Wales,#139,#39K50.#

Brown,#P.F.#(Ed.).#(2010).#Verbatim:#staging#memory#and#community.#Sydney:#Currency#Press.#

Brown,#P.F.#and#the#Maralinga#Research#Group.#(2006).#Half#a#Life#[Stage#Play#based#on#testimonies#of#Australian#and#British#Nuclear#Test#Veterans].#Available#from#Alphaville#arts#organisation.#

Burgess,#J.#(2006).#Hearing#ordinary#voices:#Cultural#studies,#vernacular#creativity#and#digital#storytelling.#Continuum:#Journal#of#Media#&#Cultural#Studies,#20(2),#201K214.#

Burgess,#J.#&#Klaebe,#H.#(2009).#Digital#storytelling#as#participatory#public#history#in#Australia.#In#J.Hartley#&#K.McWilliam,#(Eds.)#Story#Circle:#Digital#Storytelling#Around#the#World#(pp.#155K166).#Chichester,#UK;#Malden,#MA:#WileyKBlackwell.##

Burnaford,#G.#(2006).#Moving#Toward#a#Culture#of#Evidence:#Documentation#and#Action#Research#inside#

CAPE#Veteran#Partnerships#[Veteran#Partnerships#Report#2005K6].#Retrieved#May,#21,#2013#from#http://www.capeweb.org/##


# 207#

Butterfoss,#F.#D.,#Francisco,#V.,#&#Capwell,#E.#M.#(2001).#Stakeholder#participation#in#evaluation.#Health#Promotion#Practice,#2(2),#114K119.#

Campbell,#D.#T.,#Stanley,#J.#C.,#&#Gage,#N.#L.#(1963).#Experimental#and#quasiJexperimental#designs#for#research#(No.#04;#Q175,#C3.).#Boston:#Houghton#Mifflin.#

Carter,#P.#(2004).#Material#thinking:#The#theory#and#practice#of#creative#research.#Melbourne#University#Press.#

Carter,#P.#(2010).#Symposium:#‘The#Artist#as#Researcher’#[Recorded#lecture].#Royal#Academy#of#the#Art#Conference,#The#Hague#(5K6th#February,#2010).#Retrieved#November#20,#2015#from#https://vimeo.com/23632430##

Centre#for#Digital#Storytelling.#(2014).#How#it#all#began.#[Website].#Retrieved#January#4,#2014#from#http://storycenter.org/history/#(no#longer#available).#See#‘Our#Story’#on#alternative#link,#http://www.storycenter.org/press/##

Charmaz,#K.#(2000).#Grounded#Theory:#Objectivist#and#Constructivist#methods.#In#N.K.Denzin#&#Y.S.Lincoln#(Eds.),#Handbook#of#Qualitative#Research#(2nd#ed.)#(pp.509K535).#Thousand#Oaks,#CA:#Sage.#

Charmaz,#K.#(2006).#Constructing#Grounded#Theory:#A#practical#guide#through#Qualitative#Analysis.#Newbury#Park,#CA:#Pine#Forge#Press.#

Christie,#C.#A.,#&#Alkin,#M.#C.#(2008).#Evaluation#theory#tree#reKexamined.#Studies#in#Educational#Evaluation,#34(3),#131K135.#

Clarke,#A.#(1999).#Evaluation#Research.#An#introduction#to#principle,#methods#and#practice.#London:#Sage.#

Clements,#P.#(2007).#The#evaluation#of#community#arts#projects#and#the#problems#with#social#impact#methodology.#International#Journal#of#Art#&#Design#Education,#26(3),#325K335.#

Couldry,#N.#(2008).#Digital#storytelling,#media#research#and#democracy.#In#K.Lundby#(Ed.),#Digital#storytelling,#mediatized#stories:#SelfJrepresentations#in#new#media#(pp.#42K60).#New#York:#Peter#Lang.#

Cousins,#J.#B.,#&#Earl,#L.#M.#(1995).#The#case#for#participatory#evaluation:#Theory,#research,#practice.#In#J.B.#Cousins#&#L.M.#Earl#(Eds),#Participatory#evaluation#in#education:#Studies#in#evaluation#use#and#organizational#learning#(pp.#3K18).#London:#The#Falmer#Press.#

Cousins,#J.#B.,#&#Whitmore,#E.#(1998).#Framing#participatory#evaluation.#New#directions#for#evaluation.#(80),#5K23.#

Cox,#A.#(2005).#What#are#communities#of#practice?#A#comparative#review#of#four#seminal#works.#Journal#of#Information#Science,#31(6),#527–540.##

Cresswell,#J.#W.#(1994).#Qualitative#and#quantitative#approaches.#Qualitative#and#quantitative#approaches.#New#York:#Sage.#

Cresswell,#J.W.,#&#Clark,#V.#L.#P.#(2007).#The#Nature#of#Mixed#Methods#Research.#In#J.W.Cresswell#&#V.L.P.Clark#(Eds.),#Designing#and#Conducting#Mixed#Methods#Research#(pp.1K19).#Thousand#Oaks,#CA:#Sage.#

Cross,#R.,#&#Hudson,#A.#(2006).#Beyond#belief:#the#British#bomb#tests:#Australia's#veterans#speak#out.#Adelaide:#Wakefield#Press.##

Curthoys,#A.#(1991).#Unlocking#the#Academies:#Responses#and#Strategies#[online].#Meanjin,#50(2/3),#WinterKSpring#1991:#386K393.#Retrieved#December#4,#2015,#from#http://search.informit.com.au.ezp01.library.qut.edu.au/documentSummary;dn=136325485529146;res=IELLCC##


# 208#

Darabi,#A.#(2002).#Teaching#Program#Evaluation:#Using#a#Systems#Approach.#American#Journal#of#Evaluation,#23#(June),#219K228.#

Dart,#J.#&#Davies,#R.#(2003).#A#dialogical,#storyKbased#evaluation#tool:#The#Most#Significant#Change#Technique.#American#Journal#of#Evaluation,#24(2),#137–55.#

Davidson,#E.#J.#(Ed.).#(2005).#Evaluation#methodology#basics:#The#nuts#and#bolts#of#sound#evaluation.#London:#Sage.#

Davidson,#E.#J.#(2010).#Actionable#evaluation#for#realKworld#decision#makers.#In#10th#Australasian#Higher#Education#Evaluation#Forum,#[Proceedings].#November#2010,#Lincoln:#New#Zealand.#

Davidson,#P.,#&#Rowe,#J.#(2009).#Systematising#knowledge#management#in#projects.#International#Journal#of#Managing#Projects#in#Business,#2(4),#561K576.#

Davies,#R.#&#Dart,#J.#(2005).#The#Most#Significant#Change#(MSC)#Technique:#A#guide#to#its#use.#Retrieved#20.5.13#from#http://www.mande.co.uk/docs/MSCGuide.pdf##

Denzin,#N.K.#&#Lincoln,#Y.S.#(Eds.).#(2011).#The#SAGE#handbook#of#qualitative#research.#Thousand#Oaks,#CA:#Sage.#

Dewey,#J.#(1933).#How#we#think:#A#restatement#of#the#relation#of#reflective#thinking#to#the#educational#process.#Lexington,#MA:#Heath.#

DiCiccoKBloom,#B.,#&#Crabtree,#B.#F.#(2006).#The#qualitative#research#interview.#Medical#education,#40(4),#314K321.#

Dobson,#P.#J.#(1999).#Approaches#to#theory#use#in#interpretive#case#studies–a#critical#realist#perspective.#In#Australasian#Conference#on#Information#System,#Wellington,#New#Zealand.#Retrieved#September#20,#2013#from#http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.10.722&rep=rep1&type=pdf##

Doloughan,#F.#J.#(2002).#The#language#of#reflective#practice#in#art#and#design.#Design#Issues,#18(2),#57K64.#

Donaldson,#S.#I.,#Christie,#C.#A.,#&#Mark,#M.#(Eds.).#(2008).#What#Counts#as#Credible#Evidence#in#Evaluation#and#

EvidenceJbased#Practice?#Thousand#Oaks,#CA:#Sage.##

Donovan,#C.#(2012).#Priceless?#A#blog#on#the#very#idea#of#measuring#cultural#value,#Interactive#media#blog#on#Department#of#Culture,#Media#and#Sport,#United#Kingdom.#Retrieved#August#20,#2012#from#http://blogs.culture.gov.uk/main/2012/01/welcome_to_the_priceless_blog.html#(no#longer#available).#See#alternative#link#http://dcmsblog.uk/2012/01/welcomeKtoKtheKpricelessKblog/##

Drotner,#K.#(2008).#Boundaries#and#bridges:#Digital#storytelling#in#education#studies#and#media#studies.#In#K.#Lundby#(Ed.),#Digital#storytelling,#mediatized#stories:#SelfJrepresentations#in#new#media#(pp.#61K84).#Berne:#Peter#Lang.#

Dunphy,#K.#(2015).#A#Holistic#Framework#for#Evaluation#of#Arts#Engagement.#In#L.MacDowall,#M.Badham,#E.Blomkamp,#&#K.Dunphy#(Eds.).#Making#Culture#Count:#The#Politics#of#Cultural#

Measurement#(pp.243K263).#London:#Palgrave#Macmillan.##

Dunn,#A.#(2006).#Community#Partnerships#Scoping#Study:#Creative#Communities#[Commissioned#report]#Retrieved#May#22,#2013#from#http://www.australiacouncil.gov.au/__data/assets/pdf_file/0004/33952/cp_scoping_study.pdf#(no#longer#available).#See#alternative#link#http://static.placestories.com/pool/project/0001/0007950/docs/docK2E.pdf##


# 209#

Dunst,#C.#J.,#Trivette,#C.#M.,#&#LaPointe,#N.#(1992).#Toward#clarification#of#the#meaning#and#key#elements#of#empowerment.#Family#Science#Review,#5(1K2),#111–130.#

Edenloff,#J.#(2011).#ArtsJBased#Evaluation#Tools#for#Community#Arts#Programs:#A#Case#Study#of#Art#City’s#

‘Green#Art’#in#Winnipeg,#Manitoba#(Doctoral#dissertation,#University#of#Manitoba).#

Ezzy,#D.#(2006).#The#Research#Process.#In#M.#Walter#(Ed.),#Social#Research#Methods:#An#Australian#

Perspective#(pp.30K51).#Melbourne:#Oxford#University#Press.##

Fetterman,#D.#M.,#&#Wandersman,#A.#(Eds.).#(2005).#Empowerment#evaluation#principles#in#practice.#New#York:#Guilford#Press.#

Finlay,#L.#(2008).#Reflecting#on#‘Reflective#practice’#[PracticeKbased#Professional#Learning#Centre#paper#52.#A#discussion#paper#prepared#for#PBPL#CETL].#Retrieved#June#5,#2013,#from#http://www.open.ac.uk/opencetl/resources/pbplKresources/finlayKlK2008KreflectingKreflectiveKpracticeKpbplKpaperK52##

Fitzpatrick,#J.#L.,#Sanders,#J.#R.,#&#Worthen,#B.#R.#(2012).#Program#evaluation:#Alternative#approaches#and#practical#guidelines#(4th#ed.).#Upper#Saddle#River,#NJ:#Pearson.#

#Fonow,#M.,#and#Cook,#J.#(Eds.).(1991).#Beyond#Methodology:#Feminist#Scholarship#as#Lived#Research.#

Bloomington:#Indiana#University#Press.##

Friedman,#A.#(Ed.).#(2008).#Framework#for#Evaluating#Impacts#of#Informal#Science#Education#Projects#[OnKline],#p.#36.#Retrieved#12.12.13,#from#http://insci.org/resources/Eval_Framework.pdf#(no#longer#available).#See#alternative#link#http://www.informalscience.org/sites/default/files/2008_PI_Summit_Plenary_Framework_Introduction.pdf##

Gattenhof,#S.,#&#Seffrin,#G.#(2014).#Evaluation#of#the#Australian#Performing#Arts#Market#2014K2018KYear#One#Report.#

Geursen,#G.,#&#Rentschler,#R.#(2003).#Unraveling#cultural#value.#The#Journal#of#Arts#Management,#Law,#and#

Society,#33(3),#196K210.##

Gibbs,#G.#(1988).#Learning#by#Doing:#A#Guide#to#Teaching#and#Learning#Methods.#London:#Further#Education#Unit.#Retrieved#June#8,#2013#from:#http://www2.glos.ac.uk/gdn/gibbs/index.htm##

Glaser,#B.G#&#Strauss,#A.L.#(2009).#The#discovery#of#grounded#theory:#Strategies#for#qualitative#research.#New#Brunswick,#USA:#Transaction#Books.#

Goldbard,#A.#(1993).#Postscript#to#the#past:#Notes#toward#a#history#of#community#arts.#High#Performance,#16(4),#23K27.#

Goldbard,#A.#(2006).#New#Creative#Community:#The#Art#of#Cultural#Development.#Oakland:#New#Village#Press.#

Goldbard,#A.#(2008).#The#Metrics#Syndrome#[Online#article].#Retrieved#May#23,#2013#from#http://arlenegoldbard.com/wpKcontent/uploads/2005/12/MetricsKSyndromeK10K13K08.pdf#!

Goldbard,#A.#(2015).#The#Metrics#Syndrome:#Cultural#Scientism#and#Its#Discontents.#In#L.MacDowall,#M.Badham,#E.Blomkamp,#&#K.Dunphy#(Eds.).#Making#Culture#Count:#The#Politics#of#Cultural#

Measurement#(pp.214K227).#London:#Palgrave#Macmillan.##

Gray,#C.#(1996).#Inquiry#through#practice:#developing#appropriate#research#strategies.#No#Guru,#No#Method.##UIAH#Helsinki.#Retrieved#April#1,#2013,#from#http://carolegray.net/Papers%20PDFs/ngnm.pdf##

Greene,#J.#C.#(2005).#A#valueKengaged#approach#for#evaluating#the#BuncheKDa#Vinci#learning#academy.#New#Directions#for#Evaluation,#106,#27–47.�#


# 210#

Greenwood,#J.#(1993).#Reflective#practice:#a#critique#of#the#work#of#Argyris#and#Schon.#Journal#of#Advanced#Nursing,#18,#1183K1187.#

Guba,#E.G.,#&#Lincoln,#Y.S.#(1981).#Effective#Evaluation.#Beverly#Hills,#CA:#Sage.#

Guba,#E.G.,#&#Lincoln,#Y.S.#(1985).#Naturalistic#inquiry.#San#Francisco:#JosseyKBass.#

Guba,#E.G,#&#Lincoln,#Y.S#(2004).#The#roots#of#fourth#generation#evaluation:#Theoretical#and#methodological#origins.#In#M.Alkin#(Ed.),#Evaluation#roots:#Tracing#theorists’#views#and#influences.#Thousand#Oaks,#CA:#Sage.�#

Guijt,#I.,#Brouwers,#J.,#Kusters,#C.,#Prins,#E.#&#Zeynalova,#B.#(2011).#Evaluation#Revisited:#Improving#the#Quality#

of#Evaluative#Practice#by#Embracing#Complexity.#[Conference#Report#of#the#same#name#held#in#Utrecht,#the#Netherlands,#in#May#2010]#Retrieved#May#7,#2013#from#http://www.capacity.org/capacity/export/sites/capacity/documents/topicKreadings/110412KevaluationKrevisitedKmayK2010_smallKversion.pdf#(no#longer#available).#See#alternative#link#https://evaluationrevisited.wordpress.com/2011/04/13/conferenceKreportKonline/##

Hadley,#B.J.#&#Gattenhof,#S.J.#(2012).#Brokering#evaluations#of#partnerships#in#Australian#community#arts:#responding#to#entrepreneurial#tendencies.#Journal#of#Arts#and#Communities#(JAAC),#4(3),#231K249.#

Hartley,#S.#(2008).#Project#Management:#Principles,#processes#and#practice.#Pearson#Education#Australia,#Frenchs#Forest.#

Hartley,#J.#&#McWilliam,#K.#(Eds.).#(2009).#Story#Circle:#Digital#Storytelling#Around#the#World.#Malden,#MA:#WileyKBlackwell.#

Haseman,#B.#(2006).#Manifesto#for#Performative#Research.#Media#International#Australia#incorporating#

Culture#and#Policy,#(118),#99K106.##

Haseman,#B.#&#Mafe,#D.#(2009).#Creative#Practice#and#Research#in#Education#and#Politics.#In#H.Smith#&#R.T.#Dean#(Eds.),#PracticeJled#Research,#ResearchJled#Practice#in#the#Creative#Arts#(pp.212K228).#Edinburgh:#Edinburgh#University#Press.#

Hatton,#N.,#&#Smith,#D.#(1995).#Reflection#in#teacher#education:#Towards#definition#and#implementation.#Teaching#and#teacher#education,#11(1),#33K49.#

Hawe,#P.,#Degeling,#D.#&#Hall,#J.#(1990).#Impact#and#Evaluation#Outcome.#In#P.#Hawe,#D.#Degeling#&#J.#Hall#(Eds.),#Evaluating#Health#Promotion:#A#Health#Workers’#Guide.#Sydney:#Maclennon#&#Petty#Publishing.#

Hawkesworth,#M.#E.#(1989).#Knowers,#knowing,#known:#Feminist#theory#and#claims#of#truth.#Signs,#533K557.#

Hawkins,#G.#(1993).#From#Nimbin#to#Mardi#Gras:#Constructing#Community#Arts.#Australian#Cultural#Studies.#St#Leonards,#NSW:#Allen#&#Unwin.#

Hertzberg#Kaare,#B.#&#Lundby,#K.#(2008).#Mediatized#lives:#Autobiography#and#assumed#authenticity#in#digital#storytelling.#In#K.Lundby#(Ed.),#Digital#storytelling,#mediatized#stories:#SelfJrepresentations#in#

new#media#(pp.#105K122).#Berne:#Peter#Lang.##

Hildreth,#P.#&#Kimble,#C.#(2004).#Knowledge#Networks:#Innovation#through#Communities#of#Practice.#London:#Hershey,#Idea#Group#Inc.#

#


# 211#

Hlubinka,#M.#I.#(2003).#Behind#the#Screens:#Digital#Storytelling#as#a#Tool#for#Reflective#Practice.#[submission#for#Master#of#Science#at#the#Massachusetts#Institute#of#Technology].#Massachusetts:#Massachusetts#Institute#of#Technology.##

Hood,#D.#W.#(2002).#Feminist#Evaluation#and#the#Inclusion#of#Difference.#In#New#directions#for#evaluation,#96.96,#27K2.#

House,#E.#R.#(1993).#Professional#evaluation.#Newbury#Park,#CA:#Sage.##

House,#E.#R.,#&#Howe,#K.#R.#(2000).#Deliberative#democratic#evaluation#in#practice.#In#Evaluation#Models#(pp.#409K421).#Springer#Netherlands.#

Ilcan,#S.,#&#Basok,#T.#(2004).#Community#government:#voluntary#agencies,#social#justice,#and#the#responsibilization#of#citizens#1.#Citizenship#studies,#8(2),#129K144.#

Jackson,#M.R.#(2009).#Shifting#Expectations:#An#Urban#Planner’s#Reflections#on#Evaluation#of#CommunityJbased#

Arts#[Essay#for].#Retrieved#May#5,#2013#from#http://conference2012.culturaldevelopment.com.au/wpKcontent/uploads/2012/03/ShiftingExpectations.pdf#(no#longer#available).#See#alternative#link,#http://animatingdemocracy.org/sites/default/files/shifting_expectations.pdf##

Jackson,#M.R.,#Harranz,#J.#&#KabwasaKGreen,#F.#(2003).#Art#and#culture#in#communities:#a#framework#for#

measurement#[Policy#Brief#No.1#of#the#Culture,#Creativity#and#Communities#Program].#Washington#DC:#The#Urban#Institute.##

Jennings,#M.,#&#Baldwin,#A.#(2010).#“Filling#out#the#Forms#was#a#Nightmare”:#Project#evaluation#and#the#reflective#practitioner#in#community#theatre#in#contemporary#Northern#Ireland.#Music#and#Arts#in#Action,#2(2),#72K89.#

Johns,#C.#(2006).#Engaging#reflection#in#practice:#A#narrative#approach.#Oxford,#UK:#Blackwell#Publishing.#

Jonassen,#D.#H.#(1992).#Evaluating#constructivistic#learning.#In#T.M.Duffy#and#D.H.Jonassen#(Eds.),#Constructivism#and#the#technology#of#instruction:#A#conversation#(pp.137K148).#Hillsdale:#Lawrence#Erlbaum#Associates.#

Keating,#C.#(2002).#Evaluating#Community#Arts#and#Community#WellJBeing:#An#Evaluation#Guide#for#Community#

Arts#Practitioners.#(Eds.),#Arts#Victoria#and#Effective#Change.#Retrieved#May#5,#2013#from#http://creative.vic.gov.au/Research_Resources/Resources/Evaluating_Community_Arts_and_Wellbeing###

Kelaher,#M.,#Dunt,#D.,#Curry,#S.,#Berman,#N.,#&#Joubert,#L.#(2009).#Evaluation#of#the#Community#Arts#

Development#Scheme:#Improving#Health#and#Wellbeing#Through#Social#Connection#and#the#Arts.#Retrieved#May#15,#2014#from#https://www.vichealth.vic.gov.au/search/evaluationKofKtheKcommunityKartsKdevelopmentKscheme##

Kendall,#K.#E.,#&#Kendall,#J.#E.,#(2001).#Systems#analysis#and#design#(5th#Edn).#Upper#Saddle#River,#NJ:#PrenticeKHall.#

Klaebe,#H.,#Ellison,#E.,#&#Kirk,#G.#(2015).#Spotlight#on#animating#spaces:#Evaluation#of#Artslink#Queensland's#Animating#Spaces#2013#&#2014.#Brisbane:#Artslink#Queensland.#Available#from#http://eprints.qut.edu.au/89149/.#

Kotvojs,#F.#&#Hurworth,#R.#(2013).#Evaluating#capacity#development:#What#do#we#really#want#to#know?#Evaluation#Journal#of#Australasia,#13(1),#4.#

Kotzee,#B.#(2012).#Private#practice:#Exploring#the#missing#social#dimension#in#‘reflective#practice’.#Studies#in#Continuing#Education,#34(1),#5K16.#

Lambert,#J.#(2010).#Digital#Storytelling#Cookbook.#Berkeley:#Digital#Diner#Press.##


# 212#

Lambert,#J.#(2012).#Digital#storytelling:#Capturing#lives,#creating#community.#New#York:#Routledge.#

Lamont,#M.#(2011).#An#environment#of#collegial#reflective#dialogue#for#inservice#teacher#educators.#In#Processes#of#Inquiry#(pp.#63K88).#Rottadam:#Sense#Publishers.#

Larner,#W.,#&#Butler,#M.#(2005).#Governmentalities#of#local#partnerships:#The#rise#of#a"#partnering#state"#in#New#Zealand.#Studies#in#Political#Economy,#75.#

Lave,#J.#&#Wenger,#E.#(1991).#Situated#Learning:#Legitimate#Peripheral#Participation.#Cambridge:#Cambridge#University#Press.##

Lay,#M.,#&#Papadopoulos,#I.#(2007).#An#exploration#of#fourth#generation#evaluation#in#practice.#Evaluation,#13(4),#495K504.#

Leavy,#P.#(2009).#Method#meets#art:#Social#research#and#the#creative#arts.#London;#Guildford.#

Lehmann,#H.#(2001).#Using#grounded#theory#with#technology#cases:#distilling#critical#theory#from#a#multinational#information#systems#development#project.#Journal#of#Global#Information#Technology#

Management,#4(1),#45K60.#

Leitch,#R.,#&#Day,#C.#(2000).#Action#research#and#reflective#practice:#Towards#a#holistic#view.#Educational#Action#Research,#8(1),#179K193.#

Lennie,#J.#(2006).#Increasing#the#rigour#and#trustworthiness#of#participatory#evaluations:#learnings#from#the#field.#Evaluation#Journal#of#Australasia,#6(1),#7K35.#

Lippard,#L.R.#(1984).#Trojan#Horses:#Activist#Art#and#Power.#In#B.#Wallis#(Ed.),#Art#after#Modernism:#Rethinking#

Representation#(pp.341K358).#New#York:#New#Museum#of#Contemporary#Art.#

Loughran,#J.J.#(2002).#Effective#reflective#practice:#in#search#of#meaning#in#learning#about#teaching.#Journal#of#Teacher#Education,#53(1),#33K43.#

Lundby,#K.#(Ed.).#(2008).#Digital#storytelling,#mediatized#stories:#SelfJrepresentations#in#new#media.#Berne:#Peter#Lang.##

MacDowall,#L.#Introduction:#Making#Culture#Count.#In#L.MacDowall,#M.Badham,#E.Blomkamp,#&#K.Dunphy#(Eds.).#Making#Culture#Count:#The#Politics#of#Cultural#Measurement#(pp.1K8).#London:#Palgrave#Macmillan.##

MacDowall,#L.,#Badham,#M.,#Blomkamp,#E.,#&#Dunphy,#K.#(Eds.).#(2015).#Making#Culture#Count:#The#

Politics#of#Cultural#Measurement.#London:#Palgrave#Macmillan.##

MacDowall,#L.,#Mulligan,#M.,#Panucci,#F.#&#Badham,#M.#(2013).#Spectres#of#Evaluation#[Unpublished#workshop#paper].#Centre#for#Cultural#Partnerships,#University#of#Melbourne.##

Mark,#M.#M.,#Henry,#G.,#Julnes,#G.#(2000).#Evaluation:#An#integrated#framework#for#understanding,#guiding,#and#improving#policies#and#programs.#San#Francisco,#CA:#JosseyKBass.##

Markiewicz,#A.#(2012).#Closing#the#gap#through#respect,#relevance,#reciprocity#and#responsibility:#Issues#in#the#evaluation#of#programs#for#Indigenous#communities#in#Australia.#Evaluation#Journal#of#Australasia,#12(1),#19.#

Mark,#M.#M.#(2011).#Toward#better#research#on—And#thinking#about—Evaluation#influence,#especially#in#multisite#evaluations.#New#Directions#for#Evaluation,#2011(129),#107K119.#

Mason,#J.#(2002).#Qualitative#researching#(2nd#ed.).#London:#Sage.#


# 213#

Mathison,#S.#(Ed.).#(2005).#Encyclopedia#of#evaluation.#Thousand#Oaks,#CA:#Sage.##

Matarasso,#F.#(1996).#Defining#Values:#Evaluating#Arts#Programs#[Social#Impact#of#the#Arts:#Working#Paper#1].#Bournes#Green:#Comedia.#

Matarasso,#F.#(1997).#Use#Or#Ornament?:#The#Social#Impact#of#Participation#in#the#Arts.#Stroud:#Comedia.#

McClelland,#J.#R.#(1985).#The#Report#of#the#Royal#Commission#into#British#Nuclear#Tests#in#Australia.#Australian#Government#Publication,#Appendix#G#Chronology.#

McKay,#J.,#&#Marshall,#P.#(2001).#The#dual#imperatives#of#action#research.#Information#Technology#&#People,#14(1),#46K59.##

McNeill,#P.#(1985).#Research#Methods.#London:#Tavistock.#

McNiff,#S.#(2008).#ArtsKBased#Research.#In#J.G.#Knowles#and#A.#L.#Cole#(Eds.),#Handbook#of#the#Arts#in#Qualitative#Research#(pp.#29K40).#Thousand#Oaks,#CA:#Sage.##

Milbourne,#L.#(2009).#Remodelling#the#Third#Sector:#Advancing#collaboration#or#competition#in#communityKbased#initiatives?#Journal#of#Social#Policy,#38(02),#277K297.#

Mills,#D.#&#Brown,#P.#(2004).#Art#and#wellbeing:#a#guide#to#the#connections#between#community#cultural#

development#and#health,#ecologically#sustainable#development,#public#housing#and#place,#rural#

revitalisation,#community#strengthening,#active#citizenship,#social#inclusion#and#cultural#diversity.#Surry#Hills:#Australia#Council#for#the#Arts.#

Moffatt,#A.#C.#(2014).#Stepping#outside#the#circle:#a#reflective#practice#framework#for#creative#facilitators.#Retrieved#May#4,#2015#from#http://eprints.qut.edu.au/78130/2/Amanda_Moffatt_Practitioner_Resource.pdf##

Moon,#J.#(2001).#Learning#through#reflection.#In#Early#professional#development#for#teachers#(pp.#364K378).#London:#David#Fulton.#

Moreau,#K.#A.,#&#Cousins,#J.#(2012).#Program#evaluation#in#familyKcentred#pediatric#rehabilitation#settings:#A#review#of#evaluation#studies#and#the#potential#use#of#participatory#and#collaborative#evaluation#approaches.#Evaluation#Journal#of#Australasia,#11(2),#3.#

Mulligan,#M.#&#Smith,#P.#(2009).#Art,#Government#and#the#Turn#to#Community:#Putting#Art#at#the#Heart#

of#Local#Government.#Melbourne:#RMIT.#

National#Archives#of#Australia.#(2013).#British#nuclear#tests#at#MaralingaJFactsheet#129.#Retrieved#June#3,#2013#from#http://www.naa.gov.au/collection/factKsheets/fs129.aspx##

National#Ideas#Summit#&#Australia#Council#(1990).#Unlocking#the#academies:#discussion#paper#and#proposals#for#

action#from#forum#I#following#the#National#Ideas#Summit#(1990:#Canberra).#Redfern:#Australia#Council.#

Newman,#S.#(1999).#Constructing#and#critiquing#reflective#practice.#Educational#Action#Research,#7:1,#145K163.#

Newman,#T.,#Curtis,#K.,#&#Stephens,#J.#(2003).#Do#communityKbased#arts#projects#result#in#social#gain?#A#review#of#literature.#Community#Development#Journal,#38(4),#310K322.#

Osfield#Consultants.#(2007).#HighWater#Theatre#the#Partnership#–#Somebody’s#Daughter#Theatre#Company,#

Upper#Hume#Community#Health#Service.#Melbourne:#Victorian#Department#of#Education.##

#


# 214#

Osmotherly#J.#(2002).#The#HighWater#Youth#Arts#Program#Somebody’s#Daughter#Theatre#Company.#Paper#presented#by#Somebody’s#Daughter#Theatre#Company#at#the#Role#of#Schools#in#Crime#Prevention#

Conference#convened#in#Melbourne:#Australian#Institute#of#Criminology.##

Owen,#J.M.#&#Rogers,#P.J.#(1999).#Program#Evaluation:#Forms#and#Approaches.#St#Leonards,#NSW:#Allen#&#Unwin.#

Palmer,#D.#(2010).#Ngapartji#Ngapartji:#the#consequences#of#kindness.#Perth:#Murdoch#University.##

Parsons,#H.#(2015).#Growth#and#Depression#in#Hayek’s#Garden:#The#Emotional#Language#of#Accounting.#In#L.MacDowall,#M.Badham,#E.Blomkamp,#&#K.Dunphy#(Eds.).#Making#Culture#Count:#The#Politics#of#

Cultural#Measurement#(pp.53K66).#London:#Palgrave#Macmillan.##

Patton,#M.#Q.#(1994).#Developmental#evaluation.#Evaluation#practice,#15(3),#311K319.#

Patton,#M.#Q.#(2008).#UtilizationJfocused#evaluation.#Thousand#Oaks,#CA:#Sage.#

Patton,#M.#Q.#(2010).#Developmental#evaluation:#Applying#complexity#concepts#to#enhance#innovation#and#use.#New#York:#Guilford#Press.#

Price,#M.,#McCoy,#B.,#&#Mafi,#S.#(2012).#Progressing#the#dialogue#about#a#framework#for#Aboriginal#evaluations:#Sharing#methods#and#key#learnings.#Evaluation#Journal#of#Australasia,#12(1),#32.#

Richardson,#K.#(2004).#Introduction:#General#systems#theoryK#the#skeleton#of#science.#E:CO#Special#Double#Issue,#(6)(1K2),#127K139.#

Rist,#R.C.#(Ed.).#(1999).#Program#evaluation#and#the#management#of#government:#Patterns#and#prospects#across#

eight#nations.#New#Brunswick,#JF:#Transaction#Publishers.#

Rodgers,#C.#(2002).#Defining#reflection:#Another#look#at#John#Dewey#and#reflective#thinking.#The#Teachers#College#Record,#104(4),#842K866.#

Rowley,#J.#(2012).#Conducting#research#interviews.#Management#Research#Review,#35(3/4),#260K271.#

Ryan,#K.,#&#Cousins,#J.#B.#(Eds.).#(2009).#The#SAGE#international#handbook#of#educational#evaluation.#Newbury#Park,#CA:#Sage.#

Scanlan,#J.#M.,#&#Chernomas,#W.#M.#(1997).#Developing#the#reflective#teacher.#Journal#of#Advanced#Nursing,#25(6),#1138K1143.#

Schon,#D.#A.#(1983).#The#Crisis#of#Confidence#in#Professional#Knowledge.#In#D.#A.#Schon#(Ed.),#The#reflective#practitioner:#how#professionals#think#in#action#(pp.3K20).#New#York:#Basic#Books.##

Schratz,#M.#(1993).#From#Cooperative#Action#to#Collective#SelfKreflection:#A#sociodynamic#approach#to#educational#research.#In#M.#Schratz#(Ed.),#Qualitative#voices#in#educational#research#(pp.56K70).#London:#The#Falmer#Press.##

Schwandt,#T.#A.#(2009).#Globalization#influences#on#the#western#evaluation#imaginary.#The#SAGE#international#handbook#of#educational#evaluation,#19K36.#

Schwandt,#T.#(2000).#Three#Epistemological#Stances#for#Qualitative#Inquiry:#Interpretivism,#Hermeneutics,#and#Social#Constructionism.#In#N.K#Denzin#&#Y.#Lincoln#(Eds.),#Handbook#of#Qualitative#Research#(2nd#ed.),#(pp.189K213),#Thousand#Oaks,#CA:#Sage.##

Scriven,#M.#(1967).#The#methodology#of#evaluation.#In#R.E.#Stake#(Ed.),#Curriculum#evaluation.#(American#Educational#Research#Association#Monograph#Series#on#Evaluation),#No.#1,#pp.#39K83).#Chicago:#Rand#McNally.#


# 215#

Scriven,#M.#(1991).#Key#evaluation#checklist.#In#M.#Scriven#(Ed.),#Evaluation#thesaurus.#Thousand#Oaks,#CA:#Sage.#

Scriven,#M.#(2003a).#Evaluation#in#the#new#millennium:#The#transdisciplinary#view.#In#S.#I.#Donaldson#&#M.#Scriven#(Eds.),#Evaluating#Social#Programs#and#Problems:#Vision#for#the#New#Millennium.#Mahwah,#NJ:#Erlbaum.#pp.#19–42.�#

Scriven,#M.#(2003b).#The#Key#Evaluation#Checklist.#Available#online:#http://evaluation.wmich.edu/checklists/#

Scriven,#M.#(2012).#Conceptual#Revolutions#About#Evaluation;#Past#and#Future.#Recorded#lecture#from#The#Evaluation#Centre#(Evaluation#Café:#September#11th,#2012).#Retrieved#June#4,#2014,#from#http://vimeo.com/49308493##

Selwood,#S.#(2002).#Measuring#Culture#[Online#article#on#Spiked#website].#Retrieved#May#23,#2013#from#http://www.spikedKonline.com/articles/00000006DBAF.htm#(no#longer#available).#See#alternative#link,#http://www.spikedKonline.com/newsite/article/6851#.V4YZO5N94n0##

Sharp,#C.#(2003).#Development#of#Program#Evaluation#in#Australia#and#the#Australasian#Evaluation#Society#–#the#early#decades.#Evaluation#Journal#of#Australasia,#3(2),#6K16.#

Shaw,#I.,#Shaw,#I.#G.#R.,#Greene,#J.#C.,#&#Mark,#M.#M.#(Eds.).#(2006).#The#Sage#handbook#of#evaluation.#Thousand#Oaks,#CA:#Sage.#

SielbeckKBowen,#K.#A.,#Brisolara,#S.,#Seigart,#D.,#Tischler,#C.,#&#Whitmore,#E.#(2002).#Exploring#feminist#evaluation:#The#ground#from#which#we#rise.#New#directions#for#evaluation,#2002(96),#3K8.#

Smith,#H.#&#Dean,#R.T.#(Eds.).#(2009).#PracticeJled#Research,#ResearchJled#Practice#in#the#Creative#Arts.#Edinburgh:#Edinburgh#University#Press.#

Stake,#R.#(1967).#The#countenance#of#educational#evaluation.#The#Teachers#College#Record,#68(7),#523K540.#

Stauffer,#B.#(2015).#‘Process#Documentation’.#[An#adapted#online#summary#of:#Moriarty,#P.#(2007).#The#EMPOWERS#approach#to#water#governance:#guidelines,#methods#and#tools.#INWRDAM]#Retrieved#December#7,#2015#from#http://www.sswm.info/content/processKdocumentation#reference_book139##

Stock,#C.#(2013).#Course#Overview#and#Forensic#Reflective#Practice#Intensive#Modules#[Doctor#of#Creative#Industries,#QUT,#Unpublished#course#materials].#Retrieved#April#3,#2013#from#QUT#Blackboard#online#community#site#(password#protected).#

Sullivan,#G.#(2009).#Making#Space:#The#Purpose#and#place#of#PracticeKled#Research.#In#H.#Smith#&#R.T.#Dean,#PracticeJled#Research,#ResearchJled#Practice#in#the#Creative#Arts#(pp.#41K65).#Edinburgh:#Edinburgh#University#Press.#

Sullivan,#G.#(Ed.).#(2010).#Art#practice#as#research:#Inquiry#in#visual#arts.#Thousand#Oaks,#CA:#Sage.#

Swan,#P.,#&#Atkinson,#S.#(2012).#Managing#evaluation:#A#community#arts#organisation's#perspective.#Arts#&#Health,#4(3),#217K229.#

Tharp,#T.#(2003).#Twyla#Tharp:#The#creative#habit:#Learn#it#and#use#it#for#life:#A#practical#guide.#New#York:#Simon#&#Schuster#

Thomas,#L.#(2015).#Curating#in#uncharted#territories:#An#examination#of#a#programming#model#&#

communicative#platform#for#curation#in#nonJtraditional#art#spaces.#Masters#by#Research#thesis,#Queensland#University#of#Technology.#Retrieved#January#22,#2016#from#http://eprints.qut.edu.au/84621/##

#


# 216#

Thorpe,#K.#(2004).#Reflective#learning#journals:#From#concept#to#practice.#Reflective#practice,#5(3),#327K343.#

UNESCO#and#ICCROM.#(2015).#‘Why#Document’.#ReKorg#[website].#Retrieved#December#7,#2015#from#http://reKorg.info/en/documentationKsystem/1KwhyKdocument###

van#Erven,#E.#(2001).#Community#Theatre:#Global#Perspectives.#London:#Routledge.#

Wadsworth,#Y.#(1998).#What#is#Participatory#Action#Research?.#Action#Research#International,#Paper#2.#Retrieved#May#20,#2013#from#http://www.uq.net.au/action_research/ari/pKywadsworth98.html#(no#longer#available).#See#alternative#link#http://www.aral.com.au/ari/pKywadsworth98.html##

Wakefield#Regional#Council.#(2015).#[website].#Retrieved#April#3,#2015#from#http://www.wakefieldrc.sa.gov.au/page.aspx##

Watkins,#J.#and#Tacchi,#J.#(2008).#Participatory#Content#Creation#for#Development:#Principles#and#Practices#(Research#from#the#Finding#a#Voice#Project),#India:#UNESCO#House.#

WebFinances.#(2016).#‘Product’.#[Online#Business#Dictionary]#http://www.businessdictionary.com/definition/product.html#ixzz3w843DBlD##

Weiss,#C.H.#(1972).#Evaluation#research:#Methods#for#assessing#program#effectiveness.#Englewood#Cliffs,#NJ:#PrenticeKHall.#

Wenger,#E.,#McDermott,#R.#&#Snyder,#W.M.#(2002).#Cultivating#Communities#of#Practice#(Hardcover),#Boston:#Harvard#Business#Press.#

Wilcox,#D.#(1996).#The#Guide#to#Effective#Participation,#Retrieved#from#May#20,#2013#from#http://partnerships.org.uk/guide/main1.html##

Wilde,#J.#&#Sockey,#S.#(1995)#Evaluation#Handbook.#Albuquerque,#NM,#Evaluation#Assistance#CenterKWestern#Region,#New#Mexico:#New#Mexico#Highlands#University.#Retrieved#May#2,#2013#from#http://www.ncela.us/files/rcd/BE020502/Evaluation_Handbook.pdf##

Williams,#D.#(1997)#How#the#Arts#Measure#Up:#Australian#Research#into#Social#Impact#[Social#Impact#of#the#Arts#Working#Paper#No.8.].#Gloustershire:#Comdedia.#

Woolcock,#G.,#&#Davern,#M.#(2015).#Creative#Accounts:#Reimagining#Culture#and#Wellbeing#by#Tapping#into#the#Global#Movement#to#Redefine#Progress.#In#L.MacDowall,#M.Badham,#E.Blomkamp,#&#K.Dunphy#(Eds.).#Making#Culture#Count:#The#Politics#of#Cultural#Measurement#(pp.129K144).#London:#Palgrave#Macmillan.##

Yalata#&#Oak#Valley#Communities#with#Mattingley,#C#(2009).#Maralinga:#the#Anangu#story.#St#Leonards,#NSW:#Allen#&#Unwin.#

#

#

************************************end************************************#

An#integrated#approach#to#evaluation:## A#participatory# ...eprints.qut.edu.au/97728/3/Ellise Barkley Thesis.pdfAn#integrated#approach#to#evaluation:##...

Documents