by AES QLD Committee Members

Evaluators in the AES network are increasingly being challenged to apply evaluative thinking, methods and tools to innovative, emergent, place-based or otherwise complex initiatives. These initiatives often seek to achieve improvements not only in individuals and institutions, but in the systems that hold 'wicked' societal problems in place. The desired systems-level outcomes are often difficult to define, predict and measure and can change and evolve as the implementing organisations learn which strategies are most effective in reaching their goal.

In response, a recent issue of the AES QLD regional committee's newsletter focussed on resources, methods and mindsets to support members to in evaluating  complex systems change initiatives. Here are the take-outs.

by Alison Rogers

Once upon a time there was a diverse range of animals working hard to run a productive farm. Among the committed and dedicated team there were five dogs. In addition to retrieving, herding, and sniffing for wild produce, their role was to guard the premises. The dogs were friendly to the milkmaid and grocer, but for some reason, they growled and barked at the postal worker.

One day, when the postal worker was due to deliver mail, four of the dogs were distracted by a commotion on the other side of the farm. No one was watching the mailbox except for the dog known as Champ. He stayed by the gate, as he was meant to do. He observed the postal worker walk closer, and when she made no attempt to enter the premises, he stayed quietly vigilant and let her get on with her job. Champ even started wagging his tail.

by Keren Winterford

Applying ethical principles in evaluation is about making fair and just choices relevant to the context, culture of participants and evaluation purpose. In fact, whenever we speak to a person – a participant or stakeholder - as part of an evaluation, we need to think about ethics.

Why? Because this type of thinking ensures that our practice, at a bare minimum, is risk management, and adheres to the fundamental principle of ‘do no harm.’ It also shapes your relationships with participants and stakeholders as one of trust, mutual responsibility and ethical equality.

It is only through such practice that evaluation provides an important contribution to effective policy and change.

by Alison Rogers

In 2020 I wrote a fable about a dog called Champ. This fable was illustrative of anecdotes I heard from evaluators when they talked about non-evaluators on their teams who helped generate momentum for change. Champ from the fable represented the participants from my doctoral research – non-evaluators who were able to effectively persuade their reluctant peers to incorporate evaluation into their routine operations. This follow-up blog shares some the research findings to help answer the question:

by Eunice Sotelo & Victoria Pilbeam

Many evaluators are familiar with realist evaluation, and have come across the realist question “what works for whom, in what circumstances and how?” The book Doing Realist Research (2018) offers a deep dive into key concepts, with insights and examples from specialists in the field.

We caught up with Brad Astbury from ARTD Consultants about his book chapter. Before diving in, we quickly toured his industrial chic coworking office on Melbourne’s Collins Street – brick walls, lounges and endless fresh coffee. As we sipped on our fruit water, he began his story with a language lesson.

By Zazie Tolmer

Late last year an opportunity came up for a Clear Horizon consultant to work full time as an embedded evaluator in a Collective Impact initiative. I jumped at the opportunity and have been part of the backbone team for the last eight months.

by Jade Maloney, Jo Farmer and Eunice Sotelo

With so many authors and approaches to evaluation, knowing what to pay attention to can be hard. Evaluation, just like the catwalk, is subject to the whims of the day. How do you know what’s a passing fad and what will remain in fashion?

At the AES Victoria regional seminar in November, Brad Astbury suggested the following 10 books will stand the test of time.