Workshop: Foundations of Evaluation: past, present and future

Date and time: Friday 23 August and Friday 30 August 10.00am - 1.00pm AEST (registration from 9.45am). Registrants are to attend both sessions. (full day workshop - 2 sessions)

Venue: Via Zoom. Details will be emailed to registrants just prior to the workshop start time

Facilitators: Andrew Hawkins & Brad Astbury

Register online by: Thursday 22 August 2024. Spaces limited to 25 participants

Fees (GST inclusive): Members $295, Organisational member staff $415, Non-members $485, Student member $140, Student non-member $226. Students must send proof of their full-time student status to This email address is being protected from spambots. You need JavaScript enabled to view it. 

Workshop Overview

The purpose of this workshop is to provide evaluators with a firm foundation and appreciation of choices that exist for evaluation. We will cover a vast span of history and philosophy but always with the unshakeable belief that evaluation is an applied discipline that seeks to inform decision making. We plan to provide access to a fascinating world of ideas to help evaluators make better decisions, or even to blaze a new path for evaluation.

‘If you don’t know where you have been, you won’t know where you are going’. Participants may be surprised at just how deep evaluation runs through the course of human history, prior to the professionalisation of evaluation as a discipline in the mid-20th century. This rich history underpins the practice of evaluation in Australia today and provides evaluators with an enormous corpus of ideas. We will focus on the history of western approaches to evaluation and the discipline of program evaluation with which we are most familiar but acknowledge that all cultures include a wide range of approaches that emphasise different aspects and knowledge traditions as well as transformative and emancipatory approaches to evaluation.

Workshop Content

We will start by asking participants to distinguish between statements they consider evaluation and not evaluation, and why. We will then take a whirlwind tour of history to set out and compare the ontological, epistemological, and axiological differences between five key forms of evaluation in practice today: positivist, realist, pragmatist, constructivist and systems. We will explore the implications for data analysis and synthesis to produce findings and evaluative judgements that are coherent with each approach. To do this, we will begin with the end in mind, the final report and work backwards, for it is imperative to have an explicit understanding of the kinds of judgments or findings that different approaches can provide while remaining theoretically coherent.

The workshop will include interactive role plays, debates and analysis from the perspective of different paradigms. We will support participants to draw on all they have learned to consider the challenges for evaluation in a volatile, unpredictable, complex and ambiguous world (VUCA - ontology) in a post-truth era (fraught epistemology) that must incorporate a plurality of values (contested axiology).

Workshop Objectives

Intended course outcomes are:

1.  knowledge of the history of evaluation in the western intellectual tradition,
2.  appreciation of evaluation as a discipline and profession distinct from others,
3. understanding of the philosophical bases of major approaches to evaluation and
4. ability to lead conversations about different approaches and methodological considerations for an evaluation.

PL competencies

This workshop aligns with competencies in the AES Evaluator’s Professional Learning Competency Framework. The identified domains are:

• Domain 1 – Evaluative attitude and professional practice

• Domain 2 – Evaluation theory

• Domain 3 – Culture, stakeholders and context

• Domain 4 – Research methods and systematic inquiry

• Domain 7 – Evaluation Activities

Who should attend?

Many evaluators have practical wisdom that they do not always have the words to describe to others. Some evaluators have expertise in a particular paradigm for evaluation but are less aware of other approaches that might be better suited to the demands of a particular situation. Other evaluators may seek to change the very foundations on which evaluation rests. But all evaluators should have an appreciation of the history of their discipline and at some familiarity with the major ideas that evaluation theorists have debated and an on which they have diverged.

This course is foundational and concerned with history and the application of theory to practice. This means we talk about theory in sometimes philosophical ways because it is necessary to understand theory before it can be applied. While there is no assumed knowledge of evaluation or social science necessary, a willingness and appreciation of the value of thinking deeply will ensure you get the most out of this course. This is not a ‘how to course’ but a course designed to unlock your creative potential for high quality evaluation that makes a difference. Foundational evaluators may be dazzled by the array of ideas with much to follow up on, but some that resonates more or less with their background. Intermediate evaluators may discover new perspectives and ways to solve difficult problems for evaluation. Advanced evaluators may be stimulated to develop new ideas about the nature and progress of evaluation. If you think it is time for a deep dive into evaluation, one that we trust will be refreshing, we will be keen to have your insights as we continue building paths for evaluation.

Workshop start times

• VIC, NSW, ACT, TAS, QLD: 10.00am 

• SA, NT: 9.30am

• WA: 8.00am

• New Zealand: 12.00pm

• For other time zones please go to https://www.timeanddate.com/worldclock/converter.html

About the facilitators

Andrew Hawkins & Brad Astbury.

Brad and Andrew worked together at ARTD for five years and have run several internal and external workshops together during and after their time together at ARTD. Brad has an encyclopaedic knowledge of the history and the span of current ideas in our discipline, Andrew armed with a pick and shovel is determined to expose and stimulate ancient roots for a new flowering of evaluation as the alpha discipline to address key challenges for modern society.

Andrew considers evaluation as intertwined with reasoned action and strategy in a complex and uncertain world. Andrew brings a focus on reasoned action to strategy and program design as well as to the development of monitoring and evaluation frameworks and approaches for impact evaluation. Andrew focuses on logical analysis, explicating and testing the rationale that underpins a policy or program, and cost-effective means of gathering evidence and insight for decision making. Andrew’s clients value his ability to listen deeply, probe, anticipate their needs and facilitate new understandings. In his preferred line of work he is less focused on the scientific question of ‘what works’ and more on the practical question ‘how can we make it work?’ Most recently Andrew has been active in the development of a theory of evaluation that shifts the focus from social science research and the accumulation of knowledge to reasoned action and managing the risk of failure https://www.propositionalevaluation.org/

Andrew is Chief Evaluator at ARTD where he has worked for 17 years, a board member of the AES, a member of the editorial advisory board for the Australasian Journal of Evaluation. He is the founding Co-Chair of the Australian Evaluation Society’s (AES) Systems Evaluation Special Interest Group (SIG) and is an honorary fellow with Charles Darwin University’s Northern Institute where he published Realist evaluation and randomised controlled trials for testing program theory in complex social systems and was a contributor to RAMESES II reporting standards for realist evaluations.

Qualifications: Masters of Administrative Law and Public Policy (2007, University of Sydney); Bachelor of Arts, Psychology (Hons) (2000, University of Sydney)

Associate Professor Brad Astbury is a mixed method evaluator and health systems researcher with expertise in combining diverse forms of evidence to improve the quality and use of evaluation findings.

For the past two decades Brad has worked with local, state and national government agencies, industry and the not-for-profit sector to design and deliver evaluations and enhance integration of evaluation processes into organisational systems. His primary areas of evaluation practice are health care, mental health, education, community services and justice. He is interested in the practical application of evaluation theory and methodology to support continuous improvement and evidence-informed decision making about what works, for whom, under what circumstances, why and at what cost.

Brad is passionate about advancing the theory, practice and use of evaluation through:

• situationally responsive evaluation that is tailored to the level of complexity and stage of program development, consumer needs and equity considerations, intended uses of the evaluation, and available time, budget and data
• delivering high-quality training and capacity-building, including the development of policies and tools that support individuals and organisations to embed evaluation into everyday practice
• application of methods to strengthen causal inference in impact evaluations, such as theory-based evaluation, realist evaluation and Qualitative Comparative Analysis (QCA)
• the use of evaluation-specific methodology, such as rubrics to synthesise evidence and provide evaluative conclusions about the merit, worth and significance of a program or policy
• strategies to enhance program sustainability and scale up innovations
• systematic reviews using mixed method approaches to build knowledge of effective solutions to policy challenges
• meta-evaluation to improve the quality of evaluation processes and findings.

Brad is an active contributor to professional societies for evaluators in Australia, Europe and North America and publishes regularly in leading evaluation journals. He currently serves on the editorial advisory board for Evaluation – The International Journal of Theory, Research and Practice, the Evaluation Journal of Australasia and New Directions for Evaluation.