Title: Systems Evaluation Theory – Practice & Implementation

Date and time: Wednesday 1 May and Wednesday 8 May 10.00am  - 1.00pm AEST (registration from 9.45am) Registrants are to attend both sessions. (full day workshop - 2 sessions)

Venue: Via Zoom. Details will be emailed to registrants just prior to the workshop start time

Facilitators: Ralph Renger, Lewê Atkinson and Brian Keogh

Register online by: Tuesday 30 April 2024. Spaces limited to 25 participants.

Fees (GST inclusive): Members $295, Organisational member staff $415, Non-members $485, Student member $140, Student non-member $226. Students must send proof of their full-time student status to This email address is being protected from spambots. You need JavaScript enabled to view it. 

Workshop Overview

The practical application of systems thinking in a structured evaluation framework.

Based on Ralph’s recently released book – System Evaluation Theory: A Blueprint for Practitioners Evaluating Complex Interventions Operating and Functioning as Systems (Evaluation and Society)

https://www.amazon.com/System-Evaluation-Theory-Practitioners-Interventions/dp/B0BDCYFS5C

Workshop Content

Using contemporary COVID19 case study scenarios we will be facilitating participants in the following key principles of the methodology.

Objective 1:  To understand the challenges in evaluating complex interventions.

Objective 2:  To understand why an approach grounded in systems is a path forward for evaluating complex interventions.

Objective 3: To learn how to use Systems Evaluation Theory (SET) to define and evaluate complex interventions operating and functioning as a system.

The workshop will focus on evaluation case studies demonstrating application of SET for systems diagnosis and framing recommendations for system improvements in efficiency & effectiveness.

There will be some pre-work materials required to be completed by participants in order to enhance the workshop experience. There will also be encouragement and trouble-shooting advice to help apply the learnings to their own work/project context.

What are the learning outcomes / objectives of the workshop?

• Understand what it means to think in terms of systems

• Understand when a systems approach is appropriate

• Understand how to use these systems principles to complete a diagnostic analysis of efficiency and effectiveness of any system

PL competencies

This workshop aligns with the AES Evaluator’s Professional Learning Competency Framework competencies. The identified domains are:

• Domain 2 – Evaluation theory

• Domain 3 – Culture, stakeholders and context

• Domain 4 – Research methods and systematic inquiry

• Domain 6 – Interpersonal skills

• Domain 7 – Evaluation Activities

Who should attend?

Anyone facing the challenges of achieving efficient and effective collaboration and coordination of any system.

People who want to move their evaluation practice beyond the limitations of program logic. The workshop is Foundational and Intermediate.

Workshop start times

• VIC, NSW, ACT, TAS, QLD: 10.00am 

• SA, NT: 9.30am

• WA: 8.00am

• New Zealand: 12.00pm

• For other time zones please go to https://www.timeanddate.com/worldclock/converter.htm

About the facilitators

For the last decade Dr Renger has been a Professor of Public Health in the Schools of Medicine at the University of Arizona and the University of North Dakota. He has spent over twenty-five years being responsible for evaluating local, state, national and international programs. These evaluations have covered a broad range of interventions. These evaluations have included:

• Mining injuries prevention programs in Zambia
• Deforestation alleviation programs in Brazil
• Seasonal affect disorder programs in Alaska
• Children’s social skills programs in Germany

Dr Renger has also carried out innumerable evaluations across the US and Canada. Dr Renger has published over 70 research articles in peer reviewed journals around the world. His experience in the field has made him aware and frustrated with the limitations surrounding the foundations to evaluation being established through program logics. Dr Ralph Renger also has a new book available here.

In 2014, he developed Systems Evaluation Theory. Since his first article on SET, he has collaborated and developed his theory with evaluation practitioners across the globe. 

Dr Lewis Atkinson is a consultant, facilitator and trainer.  Lewe is experienced in evaluating NGO-style companies to help them enhance their efficiency and improve their effectiveness. Particularly in terms of improving their understanding of how their work makes a difference in the lives of the people that they support. NPO clients include; 139 Club Inc, Montrose Access, Australasian Kidney Trials Network Brisbane Diamantina Health Partners, The Burnet Institute, Livestock Biosecurity Network Ltd, National Road Transport Authority Ltd, QCOSS, Mater Foundation Ltd, Micah Projects Inc, QMUSIC, Australian Wagyu Association, Red Angus Association of America, Jabiru, Autism Queensland, Agricultural Business Research Institute, and JBS Australia.  Recent government clients include; Queensland Department of Housing and Public Works, WA Government Housing Authority, The WA Corruption & Crime Commission, Queensland Commissioner for Mental Health (NDIS implementation), Brisbane City Council Community Development and Capacity Building Program, innovation and ideation sessions on behalf of Queensland Government Advance Queensland – Innovate Queensland Program, Industry Challenge sessions for Regional Development Australia & Department of Environment & Heritage, Queensland Alliance for Agriculture & Food Innovation, and Department of Food and Agriculture WA.

Mr Brian Keogh has been involved in systems theory and evaluations. This has both developed and refined his approaches. With the Sydney Catchment Authority he developed the riparian assessment tool. This was a risk assessment tool that measured riparian improvement with ordinal data and mapped it to a total catchment approach. Since that time, he has been working on 'difficult to evaluate' situations in water and the environment. He was the lead for the last two Sydney Catchment Audits, and partnered with ARTD in evaluating environmental water use across NSW.

He has been part of the team, most notably with Lewe Atkinson, in providing on-the-ground experience and an EJA article for "Systems Evaluation Theory" - an evaluation theory developed by Dr Ralph Renger and published in "Systems Evaluation Theory: A blueprint for practitioners evaluating complex interventions operating and functioning as systems", 2022. He worked with Dr Renger for two months over 8 mid-west states in the US evaluating health programs using Systems Evaluation Theor

For this workshop, he will be supported by Brian Keogh & Dr Lewe Atkinson, two Australian collaborators with Dr Renger in the development of SET.