Share your ideas
AES Blog

Welcome to the AES Blog

Australasia has some excellent evaluators. More than that, we have an evaluation community full of ideas and a willingness to share. The AES has long provided a place for us to come together, at regional events and the annual conference, to develop our community together. Now we’re taking it online! The new AES blog will be a space for AES members – both new and experienced – to share their perspectives, reflecting on their theory... If you have an idea, please contact us on blog@aes.asn.au. Please also view our blog guidelines.

Opening space for a community of practice

Opening space for a community of practice

by Jade Maloney

Ever found yourself more engaged in the coffee break than the conference agenda? Ahead of the International Evaluation Conference #aes19SYD unconference day, Ruth McCausland, Kath Vaughan-Davies and I trialled an approach for the Australian Evaluation Society NSW meet-up that combined the best of both worlds – purposeful encounters with a coffee break vibe.

We adapted Open Space Technology, established by Harrison Owen in the 1980s, with the aim of finding “a way towards meetings that have the energy of a good coffee break combined with the substance of a carefully prepared agenda.” The approach has since been used around the world as a way of enabling people to self-organise around purpose.

At the NSW meet-up, about 30 evaluators braved the wind and cold to talk about evaluation topics that keep them up at night. For those of you in evaluation, it will be no surprise that these were many and varied:

  • managing your involvement in participatory action research
  • communicating findings effectively, particularly the negative
  • scoping evaluations effectively to meet and manage expectations
  • identifying value and dealing with attribution in an education context
  • planning for the data required for statistical analyses and the ethics of analysis
  • crafting useful and useable recommendations.

And that was before we got to our back-pocket topics

Working what we dubbed the “East Wing”, the “West Wing” and “next door”, groups took their discussions in different directions.  

The group discussing evaluation in an education context shared references: the four levels in Kirkpatrick’s Evaluating Training Programs (reaction, learning, behaviour and results), Guskey’s additional fifth level (although organisational support isn’t a level in the same way), as well as Mitchie, Stralen and West’s COM-B system (thinking about behaviour change in terms of capability, opportunity and motivation).

The participatory action research group was prompted by a question from one evaluator about whether he'd become too involved. They segued into how an evaluator’s participation can shape what is being evaluated and questioned whether this matters. The many lines between questions and the “really??” underneath the word "objective" in their record capture the connecting threads of their conversation, but you had to be there for the depth.

Instead of a traditional report back, we came together as we began – in a circle. The energy was palpably different, shifting from hesitant suggestions to each person sharing something they’d take forward and participants building on each other’s thoughts.

Some focused on practical tips, such as taking the time to clearly scope evaluations upfront and having findings meetings before delivering reports; some on tools (like the COM-B system); others on the process. One participant described it as bringing to life a community of practice in the AES. A few said the problem they’d started with might still keep them up at night, but they felt less alone in it. While we all came from diverse backgrounds, we found common ground among our experiences in NGO, government and private sector evaluations.

Not having set questions to answer gave people the freedom to discuss what they wanted and to go deep on a subject, and the process enabled all to have a voice.

Want to experience the process for yourself? Come along to the #aes19SYD unconference on Tuesday, September 17, to discuss how we might un-box evaluation to better contribute to reconciliation, social justice and a healthy planet. You don’t have to have the answers – just a question and the passion to hold a discussion with others on the subject.

If you’d like to learn more about Open Space Technology, there are a wealth of resources online. Chris Corrigan’s website has an easy-to-navigate collation. Or you could go back to the source: Harrison Owen’s Open Space Technology.


Jade is a Partner & Managing Director at ARTD Consultants.


 

The early career evaluator research project
Presenting for audiences - some handy tips for aes...

ABOUT US   |    CONTACT US   |    LEGALS
Search site
© 2023 Australian Evaluation Society Limited 
ABN 13 886 280 969 ACN 606 044 624
Address: 425 Smith St VIC 3065, Australia

We acknowledge the Australian Aboriginal and Torres Strait Islander peoples of this nation. We acknowledge the Traditional Custodians of the lands in which we conduct our business. We pay our respects to ancestors and Elders, past and present. We are committed to honouring Australian Aboriginal and Torres Strait Islander peoples’ unique cultural and spiritual relationships to the land, waters and seas and their rich contribution to society.