By Samantha Abbato and Kate Sunners

Pictures, storytelling and fun are essential in the evaluator's change management toolbox and for evaluation capacity building. In this blog, Samantha Abbato (Visual Insights People) and Kate Sunners (ARTD) unpack why this is and provide some ideas for your engagement toolbox through a case study. 

Many evaluations require a degree of change to the organisation's goals, processes, or technology (informed by the evaluation parameters). Still, sometimes, as the evaluator, you might only know the extent of change required once you are involved. This was the case with a six-year-long project Samantha Abbato and her team at Visual Insights People recently concluded. This evaluation capacity building project aimed to embed an evaluative mindset and skills using a digital storytelling¹ approach with a non-profit organisation (which we'll call Community Care).

As Sam and the Visual Insights' team began to work with the Community Care leadership and engage with staff across the organisation, it became apparent that direct attempts at building evaluation capacity would be unsuccessful unless they addressed the barriers to change. In this project, there was already a 'powerful coalition' of organisation leaders with a clear and compelling vision of the end goal (without which Sam believes the project would have been impossible). However, there was also the usual resistance to change and fear of what embedding evaluation would mean for staff working in the organisation.

The early indication that this project required a high degree of change management was in the reluctance of staff to engage, and their sometimes quite palpable frustration in early evaluation capacity building workshops. Sam and her team noted these early signs and sought guidance from their in-house organisational psychologist and change management literature.

The literature identifies that any change, whether at the organisation or individual level, requires letting go of old beliefs and habits and adopting new ones. It typically doesn't happen without a struggle. When dealing with change, people might employ coping mechanisms of resistance (individually or in combination) such as: by focusing on a problem or a task; through avoidance, denial or disengagement; by seeking support to deal with emotions (a sympathetic ear, for example), or opportunities to vent². Conversely, for evaluators, these responses are likely indicators that it's time to draw on change management tactics.

Understanding this dynamic at play in the Community Care evaluation capacity building project helped the Visual Insights team find a less threatening approach. The process of actively listening to and engaging with resistance and adjusting actions helped staff members accept the new reality and craft joint narratives about how the change could be positive for them and their clients.  

Key literature that informed the evaluation capacity building tactics of the team

In addition to Kotter's invaluable eight steps of change management, Sam and her team also drew on several books which provided different insights on the change management process and how best to direct it.

'The Switch – How to Change Things When Change is Hard' by Chip and Dan Heath provided a helpful framework. The authors apply a metaphor about change— the rational brain is a rider (our logical self) atop an elephant (our emotional brain)³. The rider is limited in directing the elephant as the elephant may become hot, tired, or hungry and prioritises those emotions over the rider's guidance. To be successful, the rider must have the 'map' and knowledge about the journey, must motivate the elephant based on this information, and in turn, they must work together to advance.

'The Game Changer' by Melbourne author Dr Jason Fox was another source of inspiration during this project. Fox likens effective organisational change to good video game design: if the game is well designed, there will be just enough challenge in each level to keep the player engaged and learning the skills they'll need when they advance to the next level, but not so challenging that the player becomes disillusioned and gives up. Similarly, each step to creating evaluation knowledge, process and practice must be challenging, achievable, and fun.

Insights were also applied from 'How to Lead a Quest', also by Fox. When leading change, he writes, split your focus between goals-focused activities and the 'experimental phase'. Experimenting means trying different approaches or activities and using feedback when things don't work to continue innovating to get to what does (see image below). Sam's team found this a constructive approach when pioneering new and complex terrain.

The creative toolbox for change management and evaluation capacity building

Sam's approach to transforming change-resistant staff into change champions and evaluation enthusiasts was to utilise creativity, play and engaging communications. The organisation's leadership were open to running small-budget experiments and then, more broadly, applying proven approaches. This allowed the organisation to 'follow the bright spots' and find what worked best in their context for the novel evaluation initiative.

Sam's team drew on the following three multi-disciplinary and participatory techniques for engagement and to help Care Inc staff move along the change 'path'.

Visual communications – The use of visual communication strategies – such as colourful posters of program logics and theory of change and pictorial explainers of the evaluation – provided visual, simple, reminders of the evaluation and where staff fit in the process. Staff engaged with the visual explanations so much that "Can you create a picture to show that?" became a constant request when leaders and staff were trying to understand evaluation concepts. A pictures and stories toolkit (seven booklets colour-coded to link to the evaluation explainer posters and instructional videos) was integral to embedding the monitoring and evaluation process into everyday staff practice.

Playbox for engagement and learning – The 'Playbox' draws on techniques of play, games, visual arts (sculpting, painting), and applied theatre. For example, Sam and her team took a box of 'dress ups' to build interviewing skills. They had the organisation's staff members interview each other dressed in various outfits, taking them 'outside of themselves', to learn technical in-depth interviewing skills in a non-confrontational manner. They also held evaluation quizzes and gave out prizes (some of which are still spoken of with pride by trophy winners).

These methods were created by listening to staff at the different sites, and the toolbox varied in response to different staff cultures and responses. For example, the toolbox for remote site staff was very different to that for urban locations. The contents of all toolboxes were intentionally fun and just challenging enough to keep things moving in the direction of the end vision without throwing people's threat / disengage switches. They needed to appeal to the 'elephant', make logical sense to the 'rider', and create a clear vision of their future. Of course, not all the play methods worked, but this was all part of building an environment where people felt encouraged to try new things and fail without adverse repercussions⁴.

Audio-visual storytelling – As the project progressed, staff were invited to film their stories about their involvement in the process and achievements. These videos were then compiled into annual progress videos. The benefits of this were twofold. First, it made digital evaluation stories more familiar to staff and allowed them to practice their digital storytelling skills in a low-stakes environment. Second, it helped to communicate across the organisation that the steps to transformation were achievable and showed progress toward the desired change. Where people had been visibly resistant in earlier engagements, later videos showed them championing the change – a difficult achievement to capture and share in any way other than audio-visual, where the viewer can observe the energy and expressions of the storyteller.

These videos are also used in onboarding new staff into the organisation – allowing new staff to witness the organisation's transformation and how the narratives changed over time. This also builds their capacity and understanding of the evaluation concepts and how they are applied in the organisation.

Staff also created their own digital team and individual stories that were shared via an interactive online platform. Further, they participated in storyboarding scrapbooking sessions to create the basis of a combined virtual reality story of the organisation's journey. Around 50 staff members from across the organisation created a video explaining the evaluation to participants in the style of an Australian airline safety instruction film. Staff members from all sites participated in the narration and visual explainers of the evaluation toolkit videos. These digital tasks provided small, achievable team opportunities for staff to be creative and have fun. At the same time, they provided staff with opportunities to practice the data collection skills needed for the digital evaluation interviews with participants and stakeholders that formed the core of the M&E initiative. 

^{Image caption: Some of the project's visual communications; interactive audio-visual storytelling (click on a person to see film); and playbox items.}

There's good reason to draw on visual communication, audio-visual storytelling, and creative and play-based approaches when engaging adult learners. Research shows that visuals enhance understanding and learning⁵. People also tend to engage with and retain information more easily when the experience engages the senses and puts them in a good mood, i.e. involves play and is fun⁶. We experienced this with Amy Gullickson's wonderful hand-drawn keynote on Soul and Maturity, reprised recently for AES Victoria.

There's also evidence that "joy lowers the neural threshold for perceiving life events as being positive and hopeful, while raising the threshold for perceiving events as negative and hopeless"⁷. Stories can make information richer, easier to remember and motivate people to take action⁸. Given that humans cope with change by crafting narratives about whether it is positive or negative, we would argue that story, humour, fun and play are critical to successful change management that includes mastery of a new skill-set.

Play-based approaches can help to create an environment where it feels safe to fail and try again. However, not all methods work in all contexts or for all people, so there must be several creative tools in the metaphoric toolbox to test and adapt.

We're convinced that the approach of using pictures, storytelling and fun were so effective in both managing change and building evaluation capacity at Community Care because they are effective at countering some of the potential negatives people may feel about both change and evaluation. Introducing play-based learning can reduce threat levels, make understanding and retaining information easier, and create a motivational environment in which it is safe to experiment and fail. 

^{Image caption: Visual Insights People's cartoon characters showing that pictures, stories and play are tools that can help neutralise the perceived negatives of introducing evaluation and change.}

What change management literature speaks to you in your evaluation practice? What's in your toolbox? We'd love to see your comments below on what you've used, or what you want to use in future!

------------------------------------------

 ------------------------------------------

Endnotes

[1] Digital storytelling describes the practice of teaching people to tell and produce their own stories through audio-visual recordings. It's often done through workshops focussing on providing narrative frameworks to help shape stories, as well as to provide the technical know-how. For more information, see Hartley, J., & McWilliam, K. (Eds.). (2009). Story circle: Digital storytelling around the world. John Wiley & Sons.; Digital storytelling is also used as a qualitative research and stakeholder engagement tool, as well as a means of communicating research findings. For more information, see Davey, N. G., & Benjaminsen, G. (2021). Telling tales: digital storytelling as a tool for qualitative data interpretation and communication. International Journal of Qualitative Methods, 20, 16094069211022529; Sam Abbato describes the digital storytelling approach to evaluation in her article: Abbato, S. (2023). Digital Evaluation Stories: A Case Study of Implementation for Monitoring and Evaluation in an Australian Community not-for-Profit. American Journal of Evaluation, 10982140221138031.

[2] Stanisławski, K. (2019). The coping circumplex model: An integrative model of the structure of coping with stress. Frontiers in psychology, 10, 694. Accessed at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6476932/

3] This metaphor is borrowed from Professor Jonathan Haidt's book 'The Happiness Hypothesis: Finding modern truth in ancient wisdom'

[4] Fox, J. (2016). How to Lead a Quest: A handbook for pioneering executives. John Wiley and Sons Australia, Limited.

[5] Townsend, M. S., Sylva, K., Martin, A., Metz, D., & Wooten-Swanson, P. (2008). Improving readability of an evaluation tool for low-income clients using visual information processing theories. Journal of Nutrition Education and Behavior, 40(3), 181-186. Available at: https://www.sciencedirect.com/science/article/abs/pii/S1499404607008263