By Squirrel Main, Eleanor Williams, Kristy Hornby, Mandy Charman 

Everyday ethical challenges for evaluators

Ethics are part and parcel of any evaluation journey. Every evaluator, at some point, will face tricky situations where they'll need to balance ethical principles with practical decisions. While formal ethics processes usually revolve around consent and transparency in evaluation design, the real challenges often pop up beyond that. Evaluators frequently work in environments full of volatility, uncertainty, complexity, and ambiguity (known as 'VUCA' conditions), which means we need to stay flexible and responsive throughout the entire evaluation process to keep things on track.

A lot of the discussions about ethics in evaluation in Australia seem to centre on meeting the requirements of the National Statement on Ethical Conduct in Human Research and the needs of Human Research Ethics Committees (HRECs). But the less glamorous, day-to-day ethical challenges we face when we're actually designing, delivering, and reporting on evaluations tend to get less attention. This post provides early thinking on a basic framework for evaluators who want to think about their own ethical decision-making and kickstart a conversation about whether more guidance, advice, or support is needed.

By Thushara Dibley and Lena Etuk

Australia is a culturally and linguistically diverse country. According to the 2021 census nearly half of all Australians (48%) had at least one parent born overseas, and nearly one third (28%) were themselves born overseas (Australian Bureau of Statistics 2022a; 2022b). One fifth of Australians (22%) speak a language other than English at home. Of those, 15% have low English proficiency (almost 1 million people) (Australian Bureau of Statistics 2022a). These statistics make it clear that Australia is a culturally, linguistically and ethnically rich and complex country. 

By Eleanor Williams and Skye Trudgett

Most people who have worked in evaluation have some kind of picture in their head of what it feels like when an evaluation partnership is going well. There are some common threads – a clear scope of work which has been well communicated, well-mapped aims and desired outcomes for the program, and easily-accessed data to allow for evaluative judgements to be made. The relationship between commissioner and evaluator might be described as professional but friendly with regular and frank communication flowing in both directions. Everyone involved in the evaluation – including those delivering and receiving the policy or program – have the opportunity to genuinely contribute to the evaluation and benefit from its findings.

In June this year ALNAP (Active Learning Network for Accountability and Performance in humanitarian action) brought together hundreds of practitioners across the sector to validate proposed updates to the OECD-DAC criteria. Together, they have been reviewing the original ALNAP guide published in 2006, to support greater precision in humanitarian evaluations and address significant changes in the humanitarian landscape since. 

Following a successful pilot program the previous year, your NSW AES Regional Network Committee ran a second EvaluAdder program in 2023. We spoke with EvaluAdder (mentor) Jasper Odgers and Buddy (mentee) Ingrid Willenberg about their experience in 2023, as we lead into the launch of the 2024 program.

John was made a Fellow in 2023. He is Research Leader for Education and Training, based at the Batchelor Institute of Indigenous Tertiary Education based in Darwin, NT. Over the last 18 years John has conducted research and evaluation projects which have focused on remote contexts, covering all states and territories of Australia. He is the current Editor of the Evaluation Journal of Australasia.

Lyn was made a Fellow in 2021 and has been and continues to be an outstanding contributor to the AES. She has been a member since 2007, President for four years and completed six years as Editor of the Evaluation Journal of Australasia contributing 35 articles in the Journal (articles, book reviews).  Lyn has been in the evaluation profession for over 30 years. Presently she is Dean Academic Transformation University of Southern Queensland.

Nan was made a Fellow in 2021. She has been in the evaluation profession since the late 1980s. For more than 20 years Nan has run an independent evaluation company in Wellington New Zealand. Nan builds evaluation teams on a project-by-project basis. She brings together evaluators with a broad mix of skills and experience tailored to meet the evaluation, whether that is in a government, community, or tribal context. Nan comes to evaluation with business, retail management, market research and social policy experience. She decided to set up an evaluation company because of the highly variable quality of evaluations she was managing and because she wanted to make a difference for Māori. Nan has been a member of the AES since the late 80s. As well as being co-opted onto the AES Board she was part of the Wellington Chapter and has been involved in the NZ conferences and running workshops for the AES. 

By Samantha Abbato and Kate Sunners

Pictures, storytelling and fun are essential in the evaluator's change management toolbox and for evaluation capacity building. In this blog, Samantha Abbato (Visual Insights People) and Kate Sunners (ARTD) unpack why this is and provide some ideas for your engagement toolbox through a case study.