By Jade Maloney

Our world is transforming at a dizzying rate. What does this mean for evaluation and, by extension, evaluators? That’s the question posed by the 2018 Australasian Evaluation Society conference in Launceston this week. So what do our keynotes think?

Kate McKegg – well known for her work advancing developmental evaluation practice – asks us to think deeply about what we really mean when we say transformation. What might the dimensions be? What exactly is it we are trying to transform: people, places, practices, structures, systems, technologies or something else? Does it have to be global? Or does what occurs at the national, regional, local, family or individual level count? Will we recognise transformation for what it is as it happens and be able to capture it? Can we really deliver transformation or does it have to be experienced?

McKegg’s co-presenter, Michael Quinn Patton (of utilisation-focused, developmental, and now principles-focused evaluation fame), tells us that evaluating transformation means transforming evaluation and lays down a challenge. Is evaluation going to be part of the problem (maintaining the status quo) or part of the solution (supporting and enabling transformation)?

The pair’s pre-conference workshop had everyone buzzing, both those who had read Principles-Focused Evaluation from cover to cover and those who were new to the concept. Participants learned the distinctions between rules – where the focus is on compliance and there isn’t need for interpretation – and principles – which provide guidance and direction, but need to be interpreted within specific contexts. They also learned about layering principles and that less is more in both number and description.

For Lee-Anne Molony, Managing Director at Clear Horizon, who chaired the session: a quote from William Easterly (The Tyranny of Experts) neatly summed up the value of taking a principles-based approach: ‘It is critical to get the principles of acting right before acting’. This plays out most in good ‘design’ but as evaluators our role is to support the process of ensuring those ‘right principles’ are clarified well enough that they are meaningful and relevant (provide sufficient guidance for decision makers); are able to be adhered to (at least in theory); and the results they would produce if adhered to are clear (or can be determined).

For Keryn Hassall, one of the participants: principles-focused evaluation offers an opportunity for transforming evaluation practices, and for supporting more sophisticated program management. Principles are the best way to guide decisions in complex, adaptive contexts, and where there are no easy answers to how to solve problems. Programs where the journey is just as important than the destination can look like failure when evaluated using government evaluation guidelines that focus on reporting on specified outcomes. Learning about principles-focused evaluation helps evaluators deepen their role to help program managers deliver meaningful programs.

But principles-focused evaluation is only one of the ideas on the table. Penny Hagen is strengthening the relationship between co-design and evaluation, Karol Olejniczak is getting us to gamify, and Sharon Gollan and Kathleen Stacey are asking us to apply the lens of cultural accountability to ensure evaluation is culturally safe.

With all of this on offer, you’d be hard-pressed not to find a way to transform your practice by the end of the week.

Thanks to aes18 conference convenor Jess Dart for coordinating input from keynotes and Eunice Sotelo for curating the questions.

Jade is a partner at ARTD Consultants.