Impact and Evaluation in Designing Social Innovation · 2019-06-21 · Impact and Evaluation in Designing Social Innovation: Insights from the DESIAP KL Workshop & Symposium DESIAP

Impact and Evaluation in Designing Social InnovationInsights from the DESIAP KL Workshop & Symposium

Yoko Akama, Joyce Yee, Regina Hill and Cyril TjahjaMay 2019

desiap.org

Impact and Evaluation in Designing Social Innovation: Insights from the DESIAP KL Workshop & SymposiumDESIAP Network B

Contents

Executive Summary p. 1

1. Background to this report p. 2

2. What is designing social innovation? p. 3 • Keys to effective evaluation in designing social innovation p. 5 • Principles of designing social innovation evaluation p. 6 • Recognising impact from designing social innovation initiatives p. 7

3. Comparisons p. 8 • Commonalities between designing social innovation and developmental evaluation p. 9 • Traditional evaluation vs designing social innovation models p. 10 •Whatdothosedifferencesreflectorimply? p.11 •Whatdoesthismeanforevaluation? p.12

4. Key points for practitioners, implementers, intermediaries p. 13 • Bridging the gap p. 13 • Surfacing existing evaluation practices in designing social innovation p. 14 • Example of tools and approaches used in ‘Learning & Evaluation Loops’ p. 15

5. What roles can you play? p. 16 • Key challenges and opportunities for exploration p. 17

6. Appendix p. 19 •DESIAPKL:Activities&Approaches p.20–30 •Participants’biographies p.31–33 •ExamplesofDesigningSocialInnovation p.34–36

Impact and Evaluation in Designing Social Innovation: Insights from the DESIAP KL Workshop & SymposiumDESIAP Network 1

Measuring social impact is hard. If we want to achieve meaningful social impact, we need to acknowledge that dominant impact evaluation models are limited. Thedominantmodelsofevaluationprioritisesquantifiableoutcomesthatpoorly take into account longer term impact related to social value and transformative potential. There is often a disconnect between funders and communities because of the chain of intermediaries caught between the two constituents that are often tasked to serve different agendas.

ImpactevaluationwasidentifiedbypractitionersinDESIAPeventsasa key theme and challenge in their work. This report is guided by this key question: How can impact evaluation be undertaken in a way that is centred on community-led, culturally grounded and iterative nature thattypifymostdesigningsocialinnovation(D&SI)projects?Weinvited12researchersandchange-makersfromMalaysia,Thailand,Myanmar,Cambodia, Indonesia and The Philippines, who are designing social innovation to share their experiences and identify challenges and opportunities related to evaluating the impact of their work. This gathering aimed to explore alternative social impact evaluations that are more suited to the dynamic and complex characteristics of community-led D&SI projects.

Thisreportisasummaryofthethreedays(19-21stDecember2017),beginningwithanintensive2-dayworkshopthatledtoinsightsandthemesthatweresharedwiththewiderpubliconthe21stofDecember2017.Thereportconsiders:

1) How designing social innovation practices shape the form and purpose of impact evaluation

2) How evaluation is embedded in designing social innovation processes

3) Questions and propositions for understanding impact evaluation

Executive Summary

Alternative and culturally grounded evaluative practices are present if we choose to recognise them. For funders and commissioners, acknowledging evaluation as a form of learning requires a change in mindset from one of monitoring to one of support.

We identify existing evaluative practices in D&SI projects, which often goes unnoticed because they differ from dominant or common models of evaluation. Highlighting and surfacing these differences is an important step forward in diversifying existing approaches. Key to undertaking effective evaluation in D&SI is to build trust among commissioners, communities and partners. This can open up discussions about how and what kind of impact could be achieved together. Adopting a culturally grounded evaluative practice enables project teams to be true to the needs of the communities they serve.

For funders and commissioners, acknowledging evaluation as a form of learning requires a change in mindset from one of monitoring to one of support. It requires trust in the organisations that they fund and to co-design evaluative practices that acknowledges the transformative potential. It involves expanding evaluation methods and approaches to include a broader spectrum of informal and qualitative evaluation approaches to complement traditional outcome-driven approaches. It is also important to build an eco-system of practitioners who have strong evaluative practices to support people who want to apply a more evaluative practice to their work.


DesignandSocialInnovationinAsia-Pacific(DESIAP)isanetworkofresearchers and change-makers led by Yoko Akama (RMIT University, Australia) and Joyce Yee (Northumbria University, UK), to enable researchers and practitioners undertaking design and social innovation (D&SI) in the region to mutually learn and support collective capacity building. DESIAP has been supported by these institutions and other funds from UK and Australia.

This network has been built on a series of workshops and public symposia to share inspiration, knowledge and learnings through practical examples.ThefirstDESIAPeventtookplaceinSingapore2015attheNationalDesignCentreanditconfirmedstronginterestandopportunityforD&SIintheAsia-Pacificregion.DESIAPBangkok2016,fundedbythe UK’s AHRC fund and hosted at the Thai Creative and Design Centre, built on that experience, drawing an international audience of over 150 academics, policy makers and practitioners across two days. More can be read on our website: www.desiap.org.

ThethirdDESIAPeventinKualaLumper2017focusedonimpactandevaluation, which this report captures. This event was funded by RMIT University’s Enabling Capabilities Platformthatbroughttogether12researchers and change-makers from Malaysia, Thailand, Myanmar, Cambodia, Indonesia and The Philippines, who are designing social innovation to share their experiences and identify challenges and opportunities related to evaluating the impact of their work. The list of DESIAP KL participants are in the Appendix section p. 31-33.

Background to this report

DESIAP KL was held at Majlis Rekabentuk Malaysia (Malaysia Design Council) in Kuala Lumpur, Malaysia. The participants were selected based on their catalytic role in social innovation and entrepreneurship networks in their localities, and the ensuing potential they had to share learnings and mentoring within their local contexts. Impact evaluation wasselectedasafocusbecauseitisasignificantissueandinterestformany change-makers across the world. Evaluating impact is a required outcome of social innovation practice, and yet it is challenging when the scale and focus of change aims to address societal problems, especially in a community-centred and culturally grounded way.

1


What is Designing Social Innovation?DESIAPembracesbroaddefinitionsofdesign-thereisnoonemodel- but it sees designing as a way of giving material and immaterial form to values and worldviews, which are often hidden from view. Design can be a method, a product, a technology, a system or a process, which takes both material and immaterial forms that expresses certain values andworldviewstowardsspecificoutcomes.Takenthisway,designingsocial innovation means giving material and immaterial form to meet certain social goals. Communities and organisations have always tackled problems and effected change to meet certain social goals. Such acts can be called designing (with a little ‘d’), which has been operating under other names, continually incorporating various blends of cultures, relationships, materials, histories, philosophies and worldviews in response to particular localities and situations.

DESIAP focuses on design to understand what enabling role it takes in tackling various issues. Awareness of design is increasing in government and businesses alike. DESIAP has been assisting practitioners who may not self-identify as ‘designers’, as they have not been formally trained in design, to understand what designing is, and to recognise how they are designing in their practices, in self-taught and intuitive ways with the communities they work with. It acknowledges that such practices can be called designing social innovation and seeks to explore the synergies betweenestablishedfieldsanddiscoursesof‘Design’(withbig‘D’)and ‘Social Innovation’. This is an asset-based approach to build inter-disciplinary capacities.

Through successive DESIAP events, common features of designing socialinnovationhavebeensharedandnoted(seeFig2).Inmanycases,communities and change-makers are working in highly participatory ways that resemble established design approaches such as co-design,

2

Fig. 1 Little ‘d’ iceberg model - illustrating the various visible and hidden outcomes of design activities in social innovation

View DESIAP KL video introducing DSI evaluative approaches

Visible e.g products, services, technologies, buildings, crafts, media, methods, process models, visual language, drawings, maps, songs, dance, stories etc

Less Visible (mindsets, approaches, relationships) e.g. improvisation, creativity, adaptive, collaborative,participatory,reflexive, respectful, sensitivity

Invisible (worldviews, assumptions, values, beliefs) e.g. trust, ethics, politics, power, justice , freedom, self-determination

participatory design and human-centred design. Both co-design and participatory design are human-centred approaches that involve people in iterative processes of problemdefinition,researchexploration,ideageneration,evaluateandprototypecreativeways of addressing challenging social, political and environmental issues. The highly collaborativenatureandsharedownershipoftheworkmeansthereissignificantinvestment of time and effort to build relationships and trust with and within the community and other relevant stakeholders towards empowerment of community. The nature of work is iterative and intuitive, providing multiple points of connection to build a shared understanding of the opportunity / problem being worked on and how best to respond to it.

https://vimeo.com/254287184


While designing social innovation observed in the DESIAP network resembles established co-design and participatory design approaches, it is important to recognise that internationally popular methods and models of human centred design are ‘western’ in origin and worldview. This means these models can carry legacies and assumptions of its industrialized Euro-US centric origins that emphasise a linear, uniform, replicable and generalisable process of problem-solving and lateral thinking as the main characteristic of Design (big ‘D’), visualised as divergent and convergent processes. Design here is often framed as neutral, objective and detachable from place, culture and relationships. It is clear from discussions within DESIAP network that there is not one model of design or ways to undertake social innovation. In other words, in contrast to theoretical models, the realities of designing social innovationisfarmessierandshapedsignificantlybyitssituatedcontexts.What are some features

of designing social innovation?

sensitivechange is hard work. Sensitivity is required of hidden, tacit and affective dimensions.

respectfulthere is a need to respect existing place, culture, practices and knowledges of the locality in order to build upon or to reorient.

improvisatoryinnovation cannot be predicted, so one needs to embed into the context, embrace learning from mistakes, exploration and happenstance, and work with what’s available.

empowering

the work involves highly participatory, iterative and collaborative work to provide multiple points of connection to build shared ownership and capacity.

timefulnesssignificant time and effort is needed to build trusting relationships with stakeholders and communities, and also to observe impacts of transformation.

hopefulrisk and precariousness are conditions of social innovation, but instead of fearing these, courage and optimism is needed to open up possibilities.

reflexiveone’s own values, power and position needs to be interrogated in order to work with differing agendas of many others.

...More to come

relationalbeyond designing artefacts or processes, social innovation also involves reconfiguring relationships towards a purposeful end.

Fig. 2 Features of designing social innovation. These values and qualities are often hidden under the waterline (see Fig. 1)

Fig. 3 Big ‘D’ iceberg model - illustrating the various visible and hidden outcomes of dominant design activities

Visible e.g Double Diamond model, Stanford d.school, ‘Design Thinking’

Less Visible e.g agile, lean, fast, replicable, convenient, scale, economical, solution-focused, effective, durable, newness

Invisible e.g neutral, logical, capitalism, individualism, objective


Keys to effective evaluation in designing social innovation

build trust & relationships

participatory collaboration

grounded in place, culture and locality

develop shared understanding

build a team to explore unknowns

create ways to make collective decisionsadapt process

and outcomes

synthesise multiple insights

pilot proposals with stakeholders

test ideas frequently & learn from mistakes share

learnings

build evidence & capacity

Build trust in order to build confidence,whichinturnwillhelpbuild commitment to the project.

Work to develop a culturally grounded and informed practice.

Look for unintended consequences and outcomes.

Aim for a balanced and grounded perspective. Ensure that the evaluation is useful, that there are purposeful elements for all stakeholders.

Build capability: Structure work and sharefindingsinawaythatinforms,enables and empowers community. Recognise impacts both large and small.

Be mindful of bias and assumptions based on culture, context, lived experience, training, role etc.; test against own and others perspectives and belief system. Notice unsaid things.

Recognise positional and personal authority and power; acknowledge, reveal, understand and work to manage unequal power relationships.

Explore the meaning of success: Invest time tounderstandanddefinewhatconstitutessuccess and understand what that might look like.

Createsafespace(s)toreflect,learnandcritique.Allowconflictbutmediate tensions.

Acknowledge that we don’t know everything.

Co-design and implement data collection tools and sensemaking approaches to support that; support community members to be active co-evaluators and co-researchers.

Take time to record and share how you have done things and what you have learnt through that with others. Celebrate effort, progress and achievements!

Be realistic and sustainable. Make things doable and in ways that it can continue or be passed on.

Be inclusive: Gather the perspectives of different stakeholders, including those who are hard to engage. Incorporate qualitative and quantitative approaches.


Principles of Designing Social Innovation Evaluation

The purpose of evaluation is to inform the focus and design of the project, understand what is happening (at a system, community and activity level) and what is being achieved. Another purpose is to provide sources of information through which those engaged in an initiativecanreflectonandadapttowhattheyarelearning,toimproveandinnovateonareal time basis. Robust evaluation processes should incorporate these four key phases:

2) Inform and support activity • Align effort and resources• Set and manage expectations• Support continuous improvements• Understand effectiveness of

methods and process

1) Understand grounded context • Surface assumptions and biases•Buildconfidence,trustandrelationships• Awareness of managing power-dynamics

While the core relationship, activity, process, performance and learning dimensions of evaluation are relevant to all facets of social innovation projects,thespecificattributesorareasthattheyfocusonwillchangewith context and the intention and orientation of the work.

4) Support reflection and learning • Build evidence• Capture and share stories• Bring learning back into community• Provide sector + system based

capacity building

build trust & relationships

participatory collaboration

grounded in place, culture and locality

develop shared understanding

build a team to explore unknowns

create ways to make collective decisionsadapt process

and outcomes

synthesise multiple insights

pilot proposals with stakeholders

test ideas frequently & learn from mistakes share

learnings

build evidence & capacity

3) Track process, activity and outcomes • Check alignments with intended

objectives and review consistently• Notice unintended consequences and

track short and long-term outcomes• Celebrate achievements


Recognising impact from designing social innovation initiatives

Process Related Outcomes

Outcomes resulting from experience of participating in designing social innovation process for:

Individuals: e.g. shifts in relationships, knowledge, skills, insights, understandings, attitudes, behaviours, agency

Groups: e.g. shifts in relationships, capability, standing, access to resources

Systems: e.g. network strength, cohesion, collaboration, capability, ability to leverage resources etc.

Project:sustainability,qualityoffunder–initiativerelationshipandcontributionto learning and value add

Project Related Outcomes

Outcomes resulting from the outputs and learnings from the project for:

Individuals: e.g. social connection, physical, emotional & spiritual health and wellbeing, access to education, economic and civic participation, agency and voice

Communities: cultural strength and cohesion, social dynamics, population health and wellbeing, economic development, environmental sustainability etc.

PROCESS related outcomes

PROJECT related outcomes

Sector or System Based Benefit

DSI INITIATIVE

Fig. 4 Process and project impact leading to sector/system benefit

Learnings shared through and from the work have the potential to support broader based sector or system based benefits.


In traditional evaluation the primary purpose of the evaluation is to assess andreportontheefficacyandoutcomeofaproject.Inmostcasestheprimary audience for the evaluation is the funding or commissioning party.

Hill & Vaughan’s report1, based on interviewing DESIAP participants, identifiesthat‘traditionalformativeandsummativemodelsofevaluationcanstifleinnovation’(p.6),arguingtheneedtobroadenevaluativeways that are more suited to the unique and complex characteristics of co-design and social innovation. We can take from the report that there are common problems with evaluative approaches where external and independent evaluators are assumed as the only experts with authority to judge and examine. This creates a ‘top-down’ power-dynamics and can be a disempowering experience for many change-makers, compounding existing power hierarchies between funders and receivers, especially if funders are from the Global North or dominant economies. It can also inhibit equal recognition of the other stakeholders, such as communities and change-agents, that arguably have as much grounded knowledge of the complex issues that are being addressed. Such concerns were similarly voiced during the DESIAP KL workshop. The power held by donors, impact investors or research councils that fund and evaluate impact can further exacerbate this judgemental approach.

The emergent nature of designing social innovation means that the primary purpose of evaluation in that context is to understand what has been done, what has been achieved to date and how best to progress the work. It has a learning, adaptation and an assessment function. The community led (or centred) nature of the work means the evaluation should be based on

parameters that make sense to the community. The primary audience for the evaluation is that community and the implementation partners (practitioners, funders and commissioners) working with the community.

Sometimes there can be a tension between the needs of the community and thefundingorcommissioningparty.Inmanycasesthesetensionsreflecthow funder or commissioning party has not fully understood the implications of adopting a co-design, participatory or human-centred design approach as opposed to a more traditional transactional project model.

The above tensions can be exacerbated when there is not an engaged relationship between the funder and the community, and also due to lack of understanding context of the project by the funder. This leads to misalignment of expectations, implementation, constrains value that can potentially be delivered through projects, and risks impact outcomes for all concerned.

Comparisons: How does designing social innovation differ from traditional ways of evaluating?3

1 Hill,R&Vaughan,L.,2017,‘DesignforSocialInnovationImpactEvaluationStudy’.Design and Creative Practice, Enabling Capability Platform, RMIT University. This report was commissioned by RMIT University and the interviews were conducted with several participants of DESIAP research.


Commonalities between designing social innovation and developmental evaluationDevelopmental Evaluation is designed to help review and evolve initiatives that are working on complex systems and issues in an emergent way, often in the early stages of their development (hence its name). As such, Developmental Evaluation is both an assessment and a learning process. It is not based on implementingfixedsetofassessmenttools,butratherinvolvestheapplicationofevaluativethinkingandareflectivelearningpractice.Itseekstoembedareflective,actionlearningpracticeinthewaythatinitiativesworkonadaytoday basis to progress their work and support adaptation and systems based change. It looks at what has been done, what has been learned and what needs to come next to progress an initiative using tools that suit the context and questions being asked.

The role of the developmental evaluator is to observe and capture the important and emergent patterns, and support the evolution of an initiative by helping to frame concepts, test quick iterations, track developments and surface issues. Because of that, developmental evaluation is often delivered by either embedding an evaluator within an initiative or establishing collaborative coaching / critical friend relationships where evaluators work with those engaged in an initiative to support them to develop and apply evaluative thinkingskillsandtobuildreflectivelearningpracticesintotheirinitiative.

The major difference between traditional and developmental evaluation is that the process of evaluation is embedded in, and through the work. This is to account for the changing context that the activities are nested within, and the rangeoffactorsthatinfluenceandinformcomplex,collaborativeinitiatives.The developmental evaluation approach also enables communities to be central in deciding how, why and what is important, so evaluative thinking is built into the work or project from the very beginning.

Key characteristics of Developmental Evaluation are:

• Cyclicalassessmentandreflective,strategic,action-orientedlearningprocess

• Built into the work from the beginning and embedded in and through the work, undertaken as a parallel activity i.e., not just summative, external, ‘objective’

• Match the pace of the work, with timely feedback• Takesintoaccountchangingcontextandrangeofinfluences• Participatory implementation methods (create, use and amend together)• Draw from multiple sources and perspectives • Observe and document changes to:

- Individuals: e.g. shifts in relationships, knowledge, skills, insights, understandings, attitudes, behaviours, personal outcomes

- Groups: activities undertaken and impacts from them- Systems: policy, program, funding, services, systemic changes

• Adaptive to dynamic environment to support systems-based change• Undertaken by internal teams in collaborative ways and shared continuously

with stakeholders• Balance of creative and traditional methods

It is important to remember that Developmental Evaluation is an established form of evaluation. The literature review by Hill and Vaughan, DESIAP KL workshop outcomes and the principles that emerged from DESIAP study, evidences that participants are undertaking evaluative practices in a way that strongly echoes a Developmental Evaluation methodology, even though the terminology is not used explicitly. In other words, their practices and methodology summarised in this report is just as rigorous and valuable for social innovation initiatives.

Key observation

There is a strong alignment between the Developmental Evaluation and designing social innovation process. Both are based on the establishment of trusting relationshipsthatsupportaparticipatory,reflective,actionlearningpractice.Developmental Evaluation can therefore be a powerful, complementary tool, particularly when building relationships and working through the steps in the designing social innovation process. More traditional, participatory formative and summative evaluation techniques can best be used to support the prototyping stages of a project to determine whether those activities are meeting the goals and objectives that are set for them.


• Projects are commissioned and undertaken by an external practitioner on behalf of (rather than by) communities. This outsider status aims to assure independence & objectivity.

• Projectgoals,objectivesandsuccessaredefinedbythecommissioning party based on how the project is framed. This means projectsrespondtoapre-definedissue,problemoropportunityidentifiedbythecommissioningparty.

• Community members act as project informants, participants and / orbeneficiaries.Evaluatorcontrolsimplementationandevaluationprocess.

• Project activity is undertaken on a single “plan, do, assess” cycle.

• Conclusions are based on traditional (Western) forms of knowledge, critique and analysis. Traditional research methods dominate in formative and summative ways to test and assure accountability.

• Evaluation is used to measure performance, focus on problem-solving anddeficit,assessquality&fidelityofimplementationandimpact.

Traditional, transactional models:

• Projects are initiated by (or at least co-commissioned with) members of the community or stakeholder group to which they relate. This values insider, lay knowledge and experience.

• Projects explore opportunities, needs, interests, priorities and potential responses at a grassroots level.

• Community members or stakeholders act as co-creators and co-researchers.

• Projectgoals,objectivesandsuccessesaredefinedthroughtheproject based on community and stakeholder input.

• Project activity is iterative, based on real time learning by doing, iteration and adaption.

• Conclusions are based on different forms of knowledge and sensemaking.

• Evaluation is used to monitor progress, inform and evolve the work that is being undertaken.

Designing social innovation model:

Traditional evaluation vs designing social innovation models


What do those differences reflect or imply?

Shifts in:Power

PrioritiesRoles

EpistemologiesPerspectives

ProcessFocus

Combined, those shifts change the fundamental nature of the work that is beingundertaken,movingitfromastructured,externallyled(ordefined)practice to being an emergent, community led (or centred) practice.

Funder (commissioner)


gap in relationship

collaborative relationship

Externally-led / defined

Linear structure (plan, do, assess)

Values external objectivity

Pre-defined aims and criteria

Traditional ‘western’ knowledge

Formative and summative

Consistency and accuracy

Focus on measuring

Community-led / centred

Emergent practice of learn by doing

Values insider and lay knowledge and experience

Co-defined aims and criteria

Diverse knowing and sensemaking

Real-time and iterative

Adaptive to dynamic environment

Focus on learning and transformation

one way power-dynamics

PractitionerImplementerIntermediary

PractitionerImplementerIntermediaryCommunity or

Stakeholder group

Community or Stakeholder group


Theshiftfromastructured,externallyled(ordefined)practicetoanemergent,communityled(orcentred)practicehassignificantimplicationsforevaluation–howitisapproached,whoandwhatisdefinedasthebasisfor success, how that is assessed, what and who the evaluation is for and the role that the evaluator is expected to play.

What does this mean for evaluation?

APPROACH

FOCUS

PURPOSE

EVALUATOR’s

ROLE

Develop evaluation framework as part of the project design stage (in advance of implementation)

Pre-defineevaluationcriteria,datacollection&analysisapproach (often with a reliance on more traditional western approaches)

Use formative and summative point in time evaluation to assess & report on implementation process, quality & impact

Evaluator is expected to maintain an arms length relationship to ensure independence

Develop evaluation framework alongside implementation

Co-defineevaluationcriteria,agreewhatcanconstituteevidence,whatandhow to collect data, and how to make sense of, analyse it (often leading to the incorporation of a broader range of knowledge and sensemaking approaches)

Useandsharerealtimeevaluationfindingstoinformandevolveimplementation, monitor impact and support learning

Evaluator is expected to act as a critical friend / partner. They need to have a closer (embedded) relationship to be able to help evolve the work

TRADITIONAL EVALUATION Structured, externally-led / defined practice

DEVELOPMENTAL EVALUATION Emergent, community-led / centred practice

It requires an openness to shift from more traditional models of evaluation to adopt an iterative developmental approach when framing the purpose and desired outcomes, developing shared understanding, co-explore and co-evolve the work together in designing social innovation process.


For the practitioners, there is a need to engage with project funders and commissioners to help them to understand the implications of adopting a designing social innovation approach, particularly when trying to address complex social challenges.

There is a role for project practitioners (implementers and intermediaries) to play in helping to build that understanding and ‘broker’ more informed and engaged relationships with funders to help them to better understand the local context, improve communication between the funder and the community and avoid a divergence in expectations and interests.

In some cases practitioners might do that by actively working to engage funders and commissioners in the work. It will be critical when doing that to be alert to and manage the potential power dynamics that might come with that involvement.

There is also potential to work to improve the capacity of communities to engage with donors.

Key points for practitioners, implementers, intermediaries

Bridging the gap

Fig. 4 The importance of involving the funder and community in the evaluative practice

Role for DSI practitioners to help broker and build more informed and engaged relationships with funders to help them to understand the implications of adopting DSI approach.

Seek to build trust, improve understanding and communication, draw funders ‘closer’ to the work, encourage them to take on risk and better align investments and expectations to community needs, priorities, objectives and expectations.




4

View DESIAP KL panel session discussing this topic



Surfacing existing evaluation practices in designing social innovation

Personal Reflection (Loop 1) What: internal Who: personal Why: learning & reflecting

Operational Insights (Loop 2) What: internal & external, informal & formal Who: team, collaborators Why: learning & feedback

Formal Evaluation (Loop 3) What: external & formal Who: team, funder, community Why: learning, justifying & reporting

What: internalWho: personalWhy: learning & reflectingExamples*: notes, diaries, sketches, photos, recordings

What: internal & external, informal & formalWho: personal, team, collaboratorsWhy: learning & feedbackExamples*: prototypes, stand-ups, sketches, project reviews, presentations

What: external & formalWho: team, funder, communityWhy: learning, justifying & reportingExamples*: programme renewal, reports, KPIs, case studies

Mapping your evaluative practice through learning loops

*These are just examples to get you thinking about how, when and with whom, but please feel free to modify them according to your own context

Fig. 5 Reflection, Learning & Evaluation Loop model (NB: Please seek permission from Yoko Akama and Joyce Yee before using)

There is a strong alignment between the work of designing social innovation and the work of evaluation. Many of the tools and approaches that are used to support designing social innovation processes can be used to support the evaluation process. They can act as data collection toolsandbeaplatformforreflection,sensemakingandanalysis.

Thekeyisoftentofindasimpleandsustainableprocesstocaptureor record the information and insights that are gathered through those processes and to allow space to synthesis and share those artefacts and observations.

Informationandinsightsgatheredthroughpersonalreflectionanddayto day operational activities can also be complemented using more formalgroupbasedreflectionandevaluationactivities.TheDESIAPKLworkshophasdevelopedaReflection,Learning&EvaluationLoopmethod(seeFigureabove)thatidentifiesthosethreedifferentmodesorlevelsofreflection.


Personal Reflection (Loop 1)Approaches:•Mindfulnesspracticesandreflections• Social media publishing (Facebook posts)• Peer coaching | supervision• MOOCs (massive open online courses)• Reading

Data & Insights Capture:• Daily diary

•Notes|fieldnotes• Voice recording• Sketches• Photographs

Operational Insights (Loop 2)Team Approaches:• Regular team stand-ups (daily and weekly meetings)•Sharinganddebriefingonsuccessesandfailuresaspartofteamreflectionorsupervisionprocess

• Holding ‘Happy Hour’ debriefs (held during social meetings)• Conducting ‘Stop, start, continue’ meetings or workshops• Conducting formal internal project reviews

Data & Insights Capture:• Time and activity recording• Stand up & meeting notes• Internal project review documents• Project reviews & presentations

Community Approaches:• Participant observation, interviews, focus groups or surveys

• Gathering community feedback at workshops and presentations etc.

• Conducting visioning, scenario and role playing activities• Prototyping• A/B testing (randomized experiment with two variants)

Data & Insights Capture:• Observation, interview and focus group notes• Survey responses• Graphic scribing | sketches• Workshop artefacts• Project feedback forms• Posters and program reports

Formal Evaluation (Loop 3)Approaches:• Eco-resilience mapping• Resource mapping• Participant observation, interviews, focus groups and surveys• Self-rating assessment of impact

•Mostsignificantchangeandothernarrativeorstorytellingbasedevaluation techniques

• Outcome mapping•Gamificationstrategies•360ºreflection/augmentedrealitymapping• Group validation / contribution analysis workshops

Data & Insights Capture:• Observation notes• Interview and focus group notes• Survey responses• Graphic scribing | sketches• Workshop artefacts• Project feedback forms• Videos• Case studies• Program reports• Social impact report• External presentations through conferences and symposiums
















Examples of tools and approaches used in ‘Learning & Evaluation Loops’


What roles can you play?




Evaluation Partner

Funder/commissioner: Spend time developing trust with evaluation partners, practitioners and community representatives, andensureyouagreeonthemostcontextspecific,culturally appropriate form of evaluation. Co-design a flexible evaluative framework to work with, one that encourages evaluation as a learning and potentially transformational activity. Consider how to acknowledge different types of impact and over what time frame.

Evaluation Partner: Act as a critical friend, one that offers valuable and relevant advice for the practitioners/implementer/intermediary during the project. The role of the evaluation partner isn’t to police, but instead to help guide the process of learning with partner stakeholders through an iterative and participatory process.

Practitioner/Implementer/Intermediary: Beconfidentin seeking a collaborative approach to evaluating social impact with funders, evaluation partners and community groups. You can also learn more about your own evaluative process using methods on p. 14, and incorporating models and approaches of Developmental Evaluation to build your capabilities. Connect with, learn and share from various evaluative approaches from other practitioners.

Community/Stakeholder group: Advocate for what community needs and build allyship with funders, evaluation partners, implementer and intermediaries. Discuss and agree with them on what impact and outcomes you want to see the community achieve.Beclearandfirmonwhat impact indicators are important to you and how you are involved in its evaluation.

5


Key challenges and opportunities for exploration

Building an eco-system of practitioners who have strong evaluative practices to support people who want to apply a more evaluative practice to their work. How do we make things sustainable?

Bridging the gap between the funder’s (commissioner’s) and community’s outcome expectations and timelines. How do you surface and understand the perspectives and requirements of different stakeholders?

Surfacing and resisting political dimensions and strategies when negotiating funding and commissioning structures. How do we manage the pressures of a competitive funding environment or market based agenda? Who is evaluation for, who should (does) it serve? How do we uncover and acknowledge the politics of evaluation?

Adopting an ethical and culturally grounded evaluative practice as a principle. What training can be provided or available for cultural competency?

Identifying appropriate requirements. What constitutes evidence? How do you avoid ‘boiling the ocean’? How much data is enough? How do we balance the assessment of process and performance and learning?

Using evaluation as a way to ensure community engagement. For example, the Free Prior Informed Consent (FPIC) can be used to ensure community engagement prior to undertaking work on ancestral land or using resources within the Indigenous Population territory.

DESIAP will continue to work with practitioners, researchers and funders to explore ways to understand impact and evaluation in designing social innovation. Please contact Yoko Akama and Joyce Yee if you are interested in working with us!

Yoko: [email protected] Joyce: [email protected]

Appropriating the discourse of evaluation to suit requirements. Expanding evaluation methods and approaches to include a broader spectrum of informal and qualitative evaluation approaches to complement traditional (western) research and evaluation approaches. What are existing discourses, frameworks and epistemologies that can be drawn upon to understand ‘relationships’ and evaluate intangible and unintended outcomes?

Celebrating success but avoiding hero worshipping or judging. How can we make sure that evaluation is used to enable experimentation rather than manage risk?

View DESIAP KL video discussing key challenges




1. DESIAP KL: Activities & Approaches p 20-302. Participants’ biographies p 31-333. Examples of Designing Social Innovation p 34-36

6. APPENDIX


DESIAP KL: Activities & Approaches

DESIAPKLranfromthe19-21stofDecember2017attheMalaysianDesignCentre.Weinvited12participantsrepresentingchangemakersandresearchersinthefieldofdesignandsocialinnovation(seeparticipantsbiogonp.31-33).Thefirsttwodayswasanintensiveworkshop followed by a day of sharing through a public symposium.

3 key questions seeded our initial discussions with participants during the2-days.

1. What is ‘evaluation’ and what could it mean?

2. What have been the challenges, or valuable benefits when evaluating your work?

3. What approaches do you use to notice, reflect, and assess impact?

Our approach assumes that the participants are already undertaking evaluative practices, so the activities focused on drawing out insights and enabling greater awareness. The workshop was structured to enable participantstomovefromthespecifictothegeneral,firstlybyaskingparticipants to draw from their own practice before moving to broader concerns and themes.

Appendix 1

21Impact and Evaluation in Designing Social Innovation: Insights from the DESIAP KL Workshop & SymposiumDESIAP Network

DAY 1: Connecting

Thefirstdayfocusedonfacilitatingparticipantstoconnectandsharewhotheyareandwhatthey do. It was a gentle step into developing more criticality and awareness of evaluative practices, as well as developing trust and rapport within the group to create a safe space to share and learn. We started with an icebreaker by each selecting a photo on their smartphones to introduce their family, background, work, interests, and anything else they wish to share. This activity enabled us to get to know one another through our histories, context and practice.

Next, participants disclosed their expectations of the workshop and what they hoped to get out of the three days. What are key questions, challenges and issues for us to addressrelatingtoimpactandevaluation?Thishelpedthefacilitators(Yoko&Joyce)andparticipants manage expectations and also foreground main interests and concerns of the group. This was pinned up and referenced throughout the two days.

Lastly,everyonetalkedthroughaspecificexampleoftheirevaluativepractices.Throughactivelisteningandnotation,thefacilitatorssynthesisedtheseintofivethematicareasandlater shared with the group:

• Purpose• Principles • Methods• Challenges (politics, cultural shifts, language, interfaces, interpretation)• Strategies

Thesefivethemesbecameaproductiveframeworktohelpusaddto,beapointofreferenceandre-configuresubsequentdiscussions.Thereport(p.1-19)containskeysummaries.


Connecting activities bridged with the Learning Loops method, which consists of 3 loops of increasing scale. This method was designed by Dr. Penny Hagen for an earlier workshop in Melbourne to test out its effectivenessinenablingreflection.Theloopisametaphor of continuous learning and evaluating, and it has resemblances with the way some design and innovation processes have been described. The loops increase in size to indicate shifts in scale, scope, duration and purpose of learning. This template was an effectivetooltotriggerreflection,notedetails,examplesand thoughts, which was then used for discussion.

See the summary of this activity on p.15.

DAY 1: Mapping

23Impact and Evaluation in Designing Social Innovation: Insights from the DESIAP KL Workshop & SymposiumDESIAP Network

Key questions were prepared and provided as A5 cards grouped into 4 categories.

Method • Whattechniquesdoyouuse?• Howdoyoudocumentyourwork?• Howdoyounoticebias,assumptions,values,worldviews,andhowaretheyaddressed?

Focus • What kinds of outcomes/goals do you look for in project, how do you know whether they havebeenachieved?

• Howdoyoudeterminewhattheimportantchangesare,orwere?

For whom• Howdoyounegotiatepriorities,conflictordifference?• Howdoyouensuremultiplesources/perspectives?• Howarethecommunity/stakeholdersparticipatinginanevaluativepractice?

Purpose • Whatareyoulearningfromyourevaluation?• Howdoyoufeed/sharethatbackintoyourteams?• Howhonestcanyoube?

DAY 1: Interrogating

The responses were discussed in groups. Participants were invited to interrogate the examples of evaluative practices that were shared, including their own, and synthesize their insightsontoeachquestionsheet.Thisencourageddiscussion,reflectionandassistedcritical examination of different dimensions to potentially add further to the learning loop notations.

Meta analysis and synthesis of the sheets (pinned up on the wall) was undertaken by the facilitatorstoaddfurtherinsightstothefivethematicareasthatemergedinthemorning(purpose, principles, methods, challenges, strategies). Synthesis and summary of this can bereadonp.5-7,11-12and15.


Day2focusedonconsolidatinginsightsfromDay1,reflectingwhatemerged, learnt and how we could share this in the Day 3 public symposium in groups. We started the morning by reviewing the expectation sheet to identify which questions and issues have been addressed and which ones may need further attention. This was a chance to recap our discussions and highlight insights from the previous day.

Thisalsogavetheopportunityfortheparticipantstoreviewthefivethemes–purpose,principles,methods,challenges,strategies–andselectonethemeoraspecificissuetoworkoningroups,thattheyfeltparticularlycommittedto.Thisthemeorspecificissuewasfleshedoutfurther and ‘pitched’ back to the group. This enabled the group to self-select themes that they wanted to explore further and make collective decisions to curate the public symposium on the following day. Nothing like a real deadline!

After intense discussion, contestation and group-work, three key themes were nominated by the group, and each group decided upon the content, who participates and how this was shared at the symposium.

1. Methods in Evaluative Practice2. Bridging the Gap between Donors and Communities3. Embedding Evaluating in Practice

DAY 2: Consolidating and proposing


As co-founders of the DESIAP network, Yoko Akama and Joyce Yee opened the public symposium by introducing current evaluation practices and why alternative approaches to evaluation is needed in Designing SocialInnovation(DSI)practices.DSIisaflourishingareaofinterestin research and practice and it is widely recognised as a complement to social entrepreneurship and sustainable development. To establish common grounds of understanding what design might mean, a co-design model was used to explain how it can take various forms, approaches and meaning, constituted by the particular place, culture, condition and communities towards tackling social challenges. This approach signalled some of the issues discussed in the report (p.3-4) aboutthedominanceof‘western’approachesanddefinitionsofdesign,in order to acknowledge the diverse range of D&SI practitioners who have participated in DESIAP events that may or may not self-identify as designers. While many have not been trained formally in design, this asset-basedapproachidentifieshowpractitionersaredesigningorusingdesign consciously, often less rigidly than established theories and models of design. The communities they work with are often actively involved in ways that resembles approaches such as co-design, human-centred design and participatory approaches. Cyril Tjahja’s PhD research illustrated key examples of designing social innovation practices and howimpactinsocialinnovationcouldbeconsidered–thecontinuationof the initiative by the community; the creation of new meaningful relationships, a change in social dynamics and highlight unforeseen and unintended consequences.

DAY 3: DESIAP Public Symposium | Reframing Evaluation

Click here for session video



Simon Baldwin, Secondmuse (Indonesia) led a psychological test and meditation session with the audience as a technique in cultivating reflexivityandself-awarenesstobecomemoremindfulofbias,assumptions, subconsciousness and belief systems, which shapes how people see the world.

Htet Yin Tun, PointB (Myanmar) shared two methods they use, one called Most Significant Change Stories to capture and evaluate impact of people’s behaviour change, and Human-Centred Governance Tool to evaluate impact of systems change.

Boonanan Natakun, Urban Futures Research Unit, Thammasat University (Thailand) presented his research on assisting urban communities for climate and disaster resilience where a Eco-Resilient Mapping Toolkit was developed iteratively to map local risks and resources to catalyse engagement,reflectionandcapacitybuilding.

Fadzilah Majid Cooke, Universiti Kebangsaan Malaysia (Malaysia) covered a range of meaty topics including ethics, politics and accessibility to ensure theoutcomesbenefitlocalcommunities.

DAY 3: DESIAP Public Symposium | Methods in Evaluative Practice




Panel discussion with Grace Santos, Pagna Ukthaun, Sokmesa Khiev, Fadzilah Majid Cooke and Klaus Oberbauer.

The panel, facilitated by Joyce Yee, was invited to respond to 4 questions:

1. What are your experiences working with donor and investors and its effectonevaluation?

2. Why aren’t donors talking and engaging with communities and vice versa?

3. Howcanwere-imaginethefundingrelationship?

4. Howdowesetrealistic/impactfuloutcomes/expectations?

This theme centred around key challenges in foregrounding community needs in traditional development process and relationships. One of the key tenets of social innovation is the place-based, culturally grounded approach to meeting social needs where the solution is often co-created with the community. This session explored how to challenge dominant model of engagement between donors and communities, what some of the obstacles are and what some possible strategies might we apply to surface community needs through alternate evaluative practices.

The following are key questions and responses: What are your experiences workingwithdonorandinvestorsanditseffectonevaluation?Whyaren’tdonorstalkingandengagingwithcommunitiesandviceversa?

– Time-bound support often hampers closer engagement between funder and communities

– Communities often do not have the capacity to engage with funders and implementers, so its important to build capacity and agency to contribute to the project

– Some international funders do not have the local knowledge and partners to turn high-level objectives into contextually sensitive outcomes that support the needs of the communities

Howcanwere-imaginethefundingrelationship?Howdowesetrealistic/impactfuloutcomes/expectations?

– Challenging some of the existing ways funding is distributed and managed through specific milestones.

– Developing better and longer term relationships with funders, have the conversation about evaluation and expected outcomes before the start of the project.

– Invest in the learning process

– Encourage donors to invest in their own innovation labs, to enable them to experiment at a smaller scale and take on riskier projects.

– Investing in the learning process and embedding it in the project plan as part of its evaluation process

DAY 3: DESIAP Public Symposium | Bridging the Gap between Donors and Communities




The panellists shared tangible, lived experiences of working with NPO, government, business and social enterprise to explain how they embedded evaluative practices in what they do. They also stressed the importance of evaluative practice as a key totheirownlearning,criticalreflectionandimprovement,tohighlightitsimportanceto their organisations, clients and communities beyond the requirements imposed upon them by their funders.

• Emma Rhule, Tandemic (Malaysia) proposed the idea of ‘data as water’ to imagine its accessibility in making effective decisions, and the strategies that they are exploring to ensure quantitative and qualitative data and intangible dimensions can be captured in their projects.

• Simon Baldwin, Secondmuse (Indonesia) focused on testing and rapid prototyping a solar lighting product to highlight the importance of iteration to improve the product, market need and potential, and develop its business model.

• Zeeda Mohammad, Water Warriors (Malaysia) addressed water conservation issues and shared how they iteratively learn-by-doing and use social media (e.g.. Facebook and Whatsapp) as a method for engagement, to crowdsource ideas and to foster discussion for wider community input.

• Htet Yin Tun, PointB (Myanmar) voiced questions of how we can involve people, in their case, wayfaring street-children, who are impacted most by the decisions made by partner organisations and case managers. This highlighted issues touched upon under the theme of ‘challenges’ during the workshop, such as accessibility, transparency, power and decision-making.

DAY 3: DESIAP Public Symposium | Embedding Evaluating in Practice




We used www.slido.com to gather questions from the audiences. Here are all that was submitted, which were shared to trigger a response or discussion, indicating the quality of thinking and interrogation by all.

• How can blockchain technology help fund social enterprise to deliver value to end users and what are the opportunities in the future?

• How can a major logistics player bring impact to the community?

• Given your experience / engagement with the various community , is there any low hanging fruits (logistics / eCommerce innovation)?

• Coming from Logistics cum eCommerce background, we feel a lot of innovations are centred towards the conveniences of urban community (e.g. Uber Eats etc)?

• You’ve told us about how great your donors are. What are your donor horror stories? How did you manage them?

• Are there any known initiatives to utilise data science in this space as part of the tools (other than focus groups, interviews etc)?

• Do you have any recommendations on literature relating to tools which can help us get started?

• What impact does using community members/the general public have on the quality of the data you collect? How does that impact the quality of your evaluation?

• How do we convince the people who live at the place to change their life through a design approach?

• How do people become good at deep reflection beyond practicing it?

DAY 3: DESIAP Public Symposium | Questions and discussion with the audience

• What are some examples of human-centred governance tools?

• How do we cultivate social innovation at the fundamental level in our society?

• What role do facilitators play in design and social innovation?

• Do you think the public’s mindset plays an important role in setting change? Is it something that can be moulded easily?

• Why is it working so well in Thailand compare to Malaysia?

• Are all social innovators designers, even if they wouldn’t call themselves such?

• How can we monetises social innovation so it’s sustainable?

• How can non-designers benefit from this symposium?

• What is the difference between service design and design and social innovation?

• How can we measure ‘social innovation’ if we can’t even define it?

All videos from the Day 3 public symposium can be viewed on this dedicated DESIAP Vimeo channel: https://vimeo.com/channels/desiap


Quotes from participants

““ Thank you again for inviting me to take part in the DESIAP workshop and symposium this week. I found the conversations really interesting and have been left with a lot to think about, especially in terms of how we can continue to develop our evaluation practice.” Emma Rhule

““ There is a great opportunity when people like this comes together, we can start to prototype ideas and test them in a safe environment.” Simon Baldwin

““ People are very upfront and open with their experience...hearing their stories and the challenges that they faced allowed me to really reflect on what I’m doing. It has really opened up my mind.” Pagna Ukthaun

““ There is a lot of politics in this kind of work and there is a lot of negotiation and reflection, and that is also a part of evaluation which I never thought could be discussed openly.” Zeeda Mohamad

Appendix 2


Participants’ Profiles

Simon is a multi-disciplinary design researcher who has spent the last 15 years working on a range of public health and human rights issues across the globe. Along the way he has also founded two companies. More recently, Simon has shifted his focus to exploring how companies can be supported in developing solutions to complex social and environmental problems. He is particularly interested in understanding how startups are creating new business models with the help of technology, and what can be learned from the way these new companies work. Simon is currently undertaking his PhD research in Indonesia where he has been living with hiswifeand2-year-olddaughterforthepastyear.Heisalsowritingabookabout Indonesian sambals.

Dr. Rhule uses design to create innovative solutions in complex, multi-stakeholder environments. Dr. Rhule has extensive experience building training programmes and multi-stakeholder, multi-institution projects. She has developed and led capacity development programmes around stakeholder management, project management, and communityengagementinternationally.AsChiefOperatingOfficerandLabLeadatTandemic,sheoverseesfieldresearchanddesignthinkingprojectsinthesocialsector.Prior to Tandemic, Emma led education projects at the Welcome Trust, one of the world’s largest foundations. Notable projects include launching DELTAS, Developing Excellence in Leadership, Training and Science, a programme based in sub-Saharan Africa as well as communications and stakeholder management workshops for organisations such as Monash University. Emma holds a Ph.D. in evolutionary genetics from the University Cambridge.SheisalsoanAMPG-certifiedPrince2ProjectManager.

Simon Baldwin SecondMuse (Indonesia)

Emma Rhule Tandemic (Malaysia)

Appendix 2

Cyril Tjahja is a PhD student at Northumbria University (UK) and a design practitioner. He is currently examining design and social innovation initiatives in Hong Kong, Bangkok and Kuala Lumpur and particularly interested in the perspectives, motivations and expectations of the different stakeholders involved. By constructing case studies, he hopes to gain insight into how and why social design projects operate the way they do, what value is created, and for whom.

Cyril Tjahja Northumbria University (UK)

KlausisasocialinnovatorwhohasbeenlivinginYangonsince2013.Hehasbeen developing and founding social enterprises and international development organizations in Asia, Europe and South America. Design-driven innovation has become an increasing focus of his work, and he strongly believes in its power to create better societies. He is currently working on designing and implementing learning and scaling systems for Impact Hub Yangon, and the Yangon social enterprise ecosystem. He does that through creating spaces for individuals to connect and support each other, through hosting learning environments, and through connecting organizations to local and international opportunities that help them to grow personally and professionally. Klaus graduated from the HPI dschool in Potsdam, Germany and has worked for and with partners like UNICEF, UNDP, DFAT and USAID.

Klaus Oberbauer Impact Hub Yangon (Myanmar)


Sokmesa joined InSTEDD iLab Southeast Asia (iLab SEA) as a Software DeveloperinApril2010.AfterearningabachelorsdegreefromtheRoyal University of Phnom Penh in Computer Science, he spent most of his time working with some software development companies and also actively involved with some social activities like BarCamp to share his skill and experience to the next generation student During hisfirstyearwithiLabSEAheworkedcloselyonthedevelopmentofResource Map, a web-based system that helps people track their work, resources, and results geographically in a simple and collaborative environment that can be accessed from anywhere. Currently he is a senior developer at InSTEDD Innovation Lab in Cambodia and he actively involve with iCamp which is a series of event created by InSTEDD iLab to help different stakeholders on design and prototype their problem. After the event each stakeholder could have some ideas on how to work on more to implement their solution. He also involved with some EpiHack (Epidemic Hackathon) events as facilitator to help health expert to design, prototype, and develop a small workable product to present about the next step solution. Sokmesa also has several years of experience working with open-source technologies. In the past, he worked on open-source web-based applications using PHP and Codeigniter frameworks, but started working with Ruby on Rails when he joined InSTEDD.

With a PhD in Environmental Sociology, Prof. Fadzilah Majid Cooke has been involved in research in the design of policy and processes in natural resource governance especially forests and land as well, more recently, in coastal and marine resources. She is member of a team designing free and prior informed consent processes in the oil palm production chain in Sabah, Malaysia. As a researcher she specialises in the design of social change, livelihood and entitlements in changing socio-ecological systems.In2017shehaspublishedasleadeditor,abookoncoastallivelihoods:Fisheries and Aquaculture Development in Sabah Implications for Society, Culture and Ecology. She is a member of the editorial council for the Southeast Asian Social Science Journal, National University of Malaysia (UKM), a fellow of the Asian Public Intellectual Programme, a member of the international steering committee for the Regional Initiative for Marginal Seas of East and South Asia (SIMSEA), based in Manila.

Sokmesa Khiev InSTEDD iLab Southeast Asia (Cambodia)

Fadzilah Majid Cooke National University Malaysia (Malaysia)

Zeeda is a Senior Lecturer at the Department of Science and Technology Studies, Faculty of Science, University of Malaya. She received her B.Sc in Ecology from Universiti Malaya; MSc in Environmental Management and Policy from Lund University and PhD in Science, Technology and Innovation (STI) Policy from SPRU, University of Sussex. Due to her interdisciplinary background, Zeeda has developed a deep interest to understand the relationship between the development of STI and sustainable development. In addition, Zeeda has is also interested in broader social engagement work. She was appointed as the coordinator of UMCares, the university’s volunteer-based environmental secretariat in 2011andlaterthefoundingDirectorwhenitwasupgradedintotheuniversity’sCommunityandSustainabilityCentrein2014.Shehassincetakenabreakfromadministrativeresponsibilities, but is currently leading the University’s Water Warrior Living Lab that looks into action-oriented research in integrated watershed management.

Zeeda Mohamad University Malaya (Malaysia)


Dr. Boonanan is an Assistant Professor in architecture at Faculty of Architecture and Planning, Thammasat University. His research focuses on the participatory planning and design at community and neighbourhood levels. He is currently the Chair of Interior Architecture Department and a vice director of Urban Futures, a research unit at the school. He has been involved in various community-based activities in Bangkok. He is particularly interested in and attempt to preserve urban cultural identities by looking at socio-spatial qualities of urban low-to-middle-income neighbourhoods in old city areas. One of his current projects, called Southeast Asian Neighbourhood Network (SEANNET) funded by Henry Luce Foundation, is a collaborative research project among 5 ASEAN countries,lookingatthewayinwhichurbanresearchersintheregionredefineurbanactors and urban phenomena by understanding community activities and lives in order to co-create pedagogical contribution to the future study of ASEAN cities.

Boonanan Natakun Thammasat University (Thailand)

Grace has been engaged in base of the pyramid (BoP) research and social innovation projectsintheASEANregionsince2007.Overthepast10years,shehashandledprojectsthat facilitated institutional building, program design, grants management, network development, and BoP community engagement towards social innovation and collective impact, working in partnership with targeted public and private institutions, and international development agencies such as Canada’s International Development Research Centre (IDRC), Rockefeller Foundation, the United Nations Development Programme (UNDP), and Oxfam GB. She is currently the executive director of UNIID-SEA, Inc. (www.uniid-sea.net; www.inclusiveinnovationhub.com),anewlyestablishednon-profitorganizationofhighereducation professionals and research institutions supporting “innovation for inclusive development” in Southeast Asia. She is also a consultant for inclusive business policy and knowledge management at the Asian Development Bank (ADB), and a lecturer at the Development Studies Program of the Ateneo de Manila University in the Philippines.

Grace Santos UNIID-SEA (Philippines)

Pagna has almost 10 years combine experienced in operations, social innovation, design thinking, supply chain management and sales to tackletoughestsocialproblems–transformingthemintosustainablesolutions. He is leading and building one of the most innovative social startups, WaterSHED Ventures, to help people drink clean water, use a toilet and wash their hands. The company is designing, innovating and distributing WASH products to bottom of the pyramid in Cambodia. Prior to his work with WaterSHED Ventures, Pagna spent his graduate studies to focus on bridging the theoretical frameworks of social innovation to apply in enterprises in Cambodia. His research was based on qualitative studieswithsocialenterprisesinCambodiatofindevidenceonhowcreative strategies will help social entrepreneurs to accelerate their impact and business growth. Pagna is actively involved with social entrepreneurs community through various activities such mentoring and training young aspiring entrepreneurs, and developing apprenticeship program to place local talents to work with social enterprises. He is also an in-house mentor and Programs Associate at Impact Hub Phnom Penh. Pagna holds a MSc in Business Innovation and Creative Entrepreneurship from London Metropolitan University, England.

Pagna Ukthaun Watershed Ventures (Cambodia)

Yin Tun is an Experienced Designer with a demonstrated history of working in the Human-Centered Research industry. Skilled in Research, English, Communication, Teamwork, and Team Facilitation. Passionate Design-Thinker with a beginner mindset. A Life Long learner with Bachelor of Arts (B.A) focused in English Language and Literature from Mawlamyine University.

Htet Yin Tun Point B Design + Training (Myanmar)


Examples of Designing Social InnovationAppendix 3

Co-create Charoenkrung was initiated by the Thai Creative and Design Centre (TCDC). It is a government organisation that is specialised in promoting design to ordinary Thai people. Until recently, they were located in the central upmarket Emporium Mall in Bangkok, but they decided to combine their need for a larger space and also move closer to lower socio-economic citizens they are supposed to serve by relocating to the neighbourhood of Charoenkrung. TCDC’s intention through this move is also to help improve the neighbourhood by co-creating alongside with the residents. They organised various sessions to learn what residents wanted to improve in their neighbourhood. This resulted in prototypes and models of some of the ideasthattheresidentscameupwith.Eventually,TCDCselectedfiveideastobedevelopedintoso-called1:1prototypes–executedonrealscale.Theprojects include providing signage to allow easier navigation through the neighbourhood, creating green pocket spaces, connecting alleys, rejuvenating and repurposing old buildings and redeveloping the river-front. Some of these 1:1 prototypes were momentary but some were tested over few weeks (like the green pocket space).

One of the effects of TCDC’s move and current presence in the neighbourhood was the development of similar initiatives in the vicinity by other organisations and companies. An example is Warehouse 30, located next to TCDC, which has been converted into a space with restaurants, design shops and other creative companies. But perhaps more importantly, TCDC’s

Co-create Charoenkrung (Thailand)co-creation project and the subsequent move to the neighbourhood has made those working and living in the neighbourhood grow closer together. Residents have mentioned that they have befriended others from the neighbourhood whom they met at the co-creation sessions. TCDC itself is currently working hard to establish themselves as a presence in the neighbourhood by organising tours for local schools to show what they are doing.

derived from Cyril Tjahja’s PhD study


CROSSs Studio is a architecture agency which started out as a volunteering organisation and developed into a collective of architects and designers. CROSSs works in rural areas of Thailand on a variety of projects where they use a participatory approach. For a city-wide urban renewal project in the town of Chumsaeng, they worked as facilitators to co-create ideas with the residents to develop proposals to the government. In other projects, such as the redesign of the co-working space Ma:D in Bangkok, they also take on the role of designing the space itself, in addition to co-creating with its users.

CROSSs (Thailand)It is easy to recognise the visible and substantial value of co-designing new spacesforresidents,butthereareotherbenefitscreatedbythisprojectthat are intangible. For example, CROSSs explain that when the community itself takes charge of the project without their involvement, this is a desirable outcome because it demonstrates ownership and sustainment. This can be as subtle as seating arrangements during meetings, which the community adopted since its introduction by CROSSs. Participants were asked to sit in a circle so that everyone sees eye to eye to one another and differences in social hierarchy can be diminished. This way of communicating was eventually adopted by the community and is now standard during meetings.


Form Society (or Hap Se in Cantonese) is a community space run by collaboration of two artists and KaCaMa, a social design collective. It is a bottom-up social enterprise and not supported or funded by an external entity. Located in a shop space in a working class neighbourhood of Sham Shui Po, the space is multi-functional and is subdivided into three areas to be a social meeting place for local residents. The front of the shop has a booth where residents can get their electronic equipments repaired for a small fee. Workshops are organised by master craftsperson so that residents can learn how to repair things themselves. The café/restaurant area at the back is a popular social space, and there is a record shop on thefirstfloor.Thesespacesactsasahubforthecommunityandaplacewhere they can gather, meet and socialise. The events, which mostly focuses on learning activity or a hobby, further serves as a catalyst for social interactions.

Form Society (Hong Kong)


We would like to thank the generous support provided by the following people and organisations: Asri Ahmad & Rekanegara, Emma Blomkamp, YBhg Prof. Dato Dr. Ahmad Haji Zainuddin, Penny Hagen, Associate Professor Dr. Mohamad Hariri Abdullah, Fuad Noor, Quzailan Abdul Rahman and the events team at Majlis Rekabentuk Malaysia, Cyril Tjahja, Sarah Kushinsky, RMIT University Enabling Capability Platform fund, and all the DESIAP KL speakers and participants.

Acknowledgments

April 2019, V1.1www.desiap.orgFor more information about this publication, please contact [email protected] citing this report: Akama Y, Yee J, Hill R, & Tjahja C, 2019, ‘Impact and Evaluation in Designing Social Innovation Insights from the DESIAP KL Workshop & Symposium’, DESIAP 2019 report, RMIT & Northumbria University.


desiap.org

Impact and Evaluation in Designing Social Innovation · 2019-06-21 · Impact and Evaluation in Designing Social Innovation: Insights from the DESIAP KL Workshop & Symposium DESIAP

Documents