Date and time: Monday 18th March 2019, 9am to 5pm (registration from 8.30am)
Location: Canberra CBD
Facilitator: Dr Mark Griffin
Register online by: 11 March 2019
Fees (GST inclusive): Members $440, Non-members $605, Student member $220, Student non-member $302.50
Purpose of Workshop
A robust evaluation makes use of both qualitative and quantitative research methods. At the same time many people commissioning or conducting evaluations have little training or understanding of quantitative methods such as survey design and statistics. Indeed some colleagues may even face some anxiety thinking about such methods. This workshop is not intended to turn evaluation practitioners into hard-core data scientists, but the goal instead is to give evaluation practitioners the tools necessary to work productively and in close collaboration with data scientists and to give evaluation commissioners the tools necessary to scope out projects involving statistical components, to assess the value of subsequent bids from potential statistical consultants, and to maximize the potential for statistical work to lead to true insights and business value within the commissioner’s organization. With such a goal this workshop will not focus on the specific technical intricacies of the mathematical techniques discussed, instead it will focus on a series of case studies where statistical methods have been applied in a sophisticated manner (and in so doing will introduce a range of statistical methods and the types of research questions that can be asked using each method, and some basic guidelines in proper statistical practice such as the importance of checking statistical properties of a dataset prior to conducting the analysis).
During this workshop the following topics will be presented:
Designing an evaluation project including a statistical component
- The role of quantitative and qualitative research methods in evaluation frameworks, and the types of questions that can be answered using these different research methods (it is commonly said that quantitative answers the what and qualitative the why, but is this simple statement really the case?)
Survey design and data collection
- A comparison of the value of using administrative data (data that have already been collected) versus collecting a new dataset from scratch
- Scale development including question wording, validity, reliability, and reproducibility
- Survey sampling – how to choose participants from a population to complete a survey
- Comparing the strengths and weaknesses of different data collection strategies including face to face interviews, mail and telephone
Statistical methods that could be employed
- Initial data checking – what properties of a dataset should be checked during the initial analysis stages, and what are the implications if these checks are not made
- T-tests and p-values – could these results have happened simply by random chance?
- Non-parametric tests – what do I do if my data doesn’t follow a normal distribution?
- Linear and logistic regression – is there a relationship between variables?
- Analysis of clustered and longitudinal data – what are the implications if the participants belong to different groups, or if a participants is measured at more than one timepoint?
- Factor analysis – how do I combine a number of questions into one underlying score?
- Structural Equation Modelling – can I use regression and factor analysis in the same model?
- GIS spatial mapping – can I visualize the variation in a variable over an entire country or region?
- Data linkage – can I combine two or more different datasets, possibly collected by different organizations?
Writing up and presenting statistical results
- Is it ever appropriate to say “The numbers speak for themselves”, and if not then how can we address that?
- In statistics we talk about turning data into information, but is information the same as business value? If not then how can we convert information into business value?
Approximately two thirds of the workshop will be spent in the presentation of PowerPoint slides, and the remaining third will be spent dividing into small groups where each group will discuss how the statistical methods presented have or could be used within the projects that the participants are involved in.
- To give an evaluation practitioner the required skills to work productively and in close collaboration with data scientists, to choose statistical methods that offer true value to a commissioner, and to report statistical findings in a way that offers true business value.
- To give an evaluation commissioner the required skills to scope out projects including statistical components, to assess bids by statistical consultants, and to convert reports written by consultants into true business value.
This workshop aligns with competencies in the AES Evaluator’s Professional Learning Competency Framework. The identified domains are:
- Domain 4 – Research methods and systematic inquiry
- Domain 7 – Evaluation Activities
Who should attend?
This workshop will be of interest to evaluation commissioners and practitioners who are using statistical methods within their work. No previous knowledge about statistical methods is assumed within this workshop. This workshop might also be of interest to data scientists who are already familiar with these statistical methods, but would like to know more about how they relate to evaluation work.
About the facilitator
Dr Mark Griffin is the Founding Director of Insight Research Services Associated (www.insightrsa.com), and holds academic appointments at the University of Queensland and the University of Sydney. Mark serves on the Executive Committee for the Statistical Society of Australia, and is the Founder and Co-Chair of the Special Interest Group for Business Analytics within the International Institute of Business Analysis. Mark has been the primary statistician for a number of large surveys (including a survey of 140,000 parents receiving the Positive Parenting Program in Queensland), and Insight is a member of a number of government panels including that for the Therapeutic Goods Association within the Australian Department of Health. Since the formation of Insight Mark has presented over 90 two-day and 15 five-day workshops in statistics around Australia, and has recently started an annual international speaking tour.