Workshop: Experiments and quasi-experiments: understanding causality and the evidence requirements for evaluating outcomes

Date and time: Wednesday 5 February 2025, 10.00 am to 1.30pm AEDT (registration from 9.45am). Half day workshop - 1 session

Venue: Via Zoom. Details will be emailed to registrants just prior to the workshop start time

Facilitator: Scott Bayley

Register online by: Tuesday 4 February 2025, unless sold out prior. Spaces limited to 25 participants

Fees (GST inclusive): Members $215.00, AES Organisational member staff $345.00, Non-members $385.00, Student member $125.00, Student non-member $180.00* (GST inclusive) * Students must send proof of their full-time student status to This email address is being protected from spambots. You need JavaScript enabled to view it.  

Workshop Overview

The past 15-20 years has seen considerable debate in the evaluation profession about RTCs and the topic of the gold standard. Since method choices must always made in a specific context (e.g. the type of program, its history and degree of maturity, availability of data, etc), debates in the professional literature about preferred methods can at best only provide partial guidance to evaluation practitioners. The way to break out of this methods impasse is by focusing on the evidentiary requirements for assessing causal outcomes.
In this innovative skills building session participants will:

• become familiar with the evidentiary requirements for undertaking outcome evaluations

• be encouraged to adopt a systematic and evidence-based approach to undertaking outcome evaluations

• gain practical experience in the application of evidentiary criteria for casual analysis

• develop their skills in critiquing outcome evaluation reports.

The workshop will be interactive and involve sharing experiences and practical activities.

Workshop Content

This foundational workshop will introduce participants to theories of causality and the evidence requirements for evaluating program outcomes. This unique session will present a brief summary of the literature on the philosophy and principles of causal analysis and relate these to some common evaluation approaches. A framework for applying four key evidentiary criteria will be examined and participants will be guided through examples and exercises illustrating how to apply these criteria in typical evaluation situations.

The workshop’s content will include the following topics:

• the distinction between evaluation and research

• outcome evaluation is about analysing causal relationships

• theories of causality and relevant models from both philosophy and evaluation

• the evidence requirements for outcome evaluation

• types of research design, their defining characteristics and strengths / weaknesses

• examples of stronger research designs for causal analysis (outcome evaluation)

• a case study exercise and discussion of a controversial outcome evaluation.

The presenter will encourage participants to apply the knowledge they gain to their own contexts with an emphasis on practical applications. Participants will be provided with resource materials to support their evaluation practices back in the workplace.

Workshop Outcomes

This workshop will introduce participants to the foundational concepts and terminology of outcome evaluation while sharing resources for further self-study. After completing this workshop participants will:

• be familiar with the main theoretical approaches to studying causal relationships

• have a basic knowledge of relevant terminology.

• understand the evidentiary requirements for causal analysis

• have an appreciation of the different types of research designs, and

• have the knowledge to undertake a preliminary assessment of the credibility of a given outcome evaluation.

PL competencies 

• Domain 1 - Evaluative Attitude and Professional Practice 

• Domain 2 - Evaluation Theory 

• Domain 4 - Research Methods and Systematic Inquiry 

Evaluators' Professional Learning Competency Framework  

Who should attend?

Foundational – Foundational sessions assume no previous knowledge from the audience – they are beginner-friendly. The topic is presented at an introductory level using accessible language that can be understood by non-expert audiences.

This workshop is the first of a three-part series of planned workshops to run in 2025. The second workshop (one day, Intermediate level) will undertake a more in-depth analysis of research designs, appropriate outcome evaluation questions, and the typical threats to reaching defensible conclusions about program outcomes. The third workshop (one day Intermediate / Advanced) will examine the technical aspects of conducting and analysing some common experimental and quasi-experimental designs, and contrast these with theory-based passive observational studies.

• VIC, NSW, ACT, TAS: 10.00am

• QLD: 9.00am

• SA: 9.30am

• NT: 8.30am

• WA: 7.00am

• New Zealand: 12.00pm

For other time zones please go to https://www.timeanddate.com/worldclock/converter.html

About the facilitator

Scott Bayley manages his own evaluation consultancy business and holds a MA in Public Policy majoring in evaluation and social measurement. He has over 30 years of experience in evaluation and is a Fellow of the Australian Evaluation Society. Prior to having his own consultancy Scott held senior positions with Oxford Policy Management, the Australian Department of Foreign Affairs and Trade, AusAID, the Asian Development Bank, the Victorian Auditor General’s Office, the Northern Territory Department of Health, and the Aboriginal and Torres Strait Islander Commission.

Scott has served as an advisor to the United Nations Secretariat in New York on performance measurement; the Evaluation Capacity Development Group in Geneva; UNICEF in New York and the Philippines; Valuing Voices: Sustainable Solutions in the Czech Republic; the UNDP in Vietnam; and the world’s peak evaluation body, the International Organization for Cooperation in Evaluation.

Scott has published over 70 evaluation reports across a number of fields including international development, health, education, children’s services, indigenous affairs, the environment, infrastructure, capacity building, public sector financial management, and governance. Scott is particularly interested in impact evaluation methods, evaluation capacity building, and in using performance feedback to drive continuous program improvement while strengthening stakeholder relationships.