by Anthea Rutter
Although a number of AES members have founded consultancies to channel their evaluation work, it is another thing to think about – and actually achieve – the founding of a professional society. This is exactly what Emeritus Professor Anona Armstrong did. Through her company Evaluation Training & Services, the fledging society was born in the early 80s. Not only did Anona found the AES, she had the honour and distinction of having a piece of music written for her and performed at the AES International Conference in 1992.
Unfortunately, I wasn’t able to interview Anona in person, but I am very aware of her achievements – Anona and I go back a long way! Though I know some of the earlier history of the society, it was important to get an accurate record, so the first question on our string of emails was:
Forming a society is no mean feat…how did it happen?
I guess I sort of anticipated her answer (something on the lines of ’well…Rome wasn’t built in a day’).
| Like all good things the society was formed by slow steps. It started with the first ‘National Evaluation Conference’ which was held in 1982 by the Department of Psychology and the Program in Public Policy Studies at the University of Melbourne.
I compiled a Directory of Australian Evaluators in 1983, and 99 people responded, identifying 52 different categories in which they were conducting evaluation. This Directory became the foundation for a mailing list, and the building blocks for the development of the Australasian Evaluation Society (AES). |
Anona also acknowledged that the growth in the membership of the AES was due “in no small way to formal teaching but also to evaluation training provided by AES members.”
Like many evaluators, I was curious about how others began their careers, particularly in the ‘80s when evaluation was still a fledging field in Australia.
So how did you get into evaluation?
|
In the 80s most people had never heard of evaluation. I was regularly asked ‘What do you mean?’ I remember receiving phone calls from people asking if I could do an evaluation of their property. At the time, evaluation was a novelty. Many government program managers thought that setting performance objectives for government programs could not be done. |
Anona was referring to her foray into impact assessment, when she realised that there were no accepted measures. So she looked into the ‘new’ research on social indicators, which gave her the “foundation for measuring impact as well as performance management in the public sector.” It’s no less than what I would expect from someone who founded the AES.
As an evaluator I was also curious about Anona’s thoughts on some of the early writers who influenced the field of evaluation in Australia.
I was not disappointed – I got a valuable history lesson on her take on the assent of evaluation in this country.
|
Evaluation in Australia owes its origins to the US. Madaus, Stufflebeam and Scriven (1983) traced evaluation in the US back to the educational reforms of the 19th Century, but modern evaluation is usually associated with the 1960s when the US Government’s General Accounting Office (GAO) sponsored evaluation of the large-scale curriculum development projects introduced under President Lyndon Johnson’s ‘War on Poverty’ (Madeus et al, 1983). By the 70s, new conceptual frameworks were introduced that were specific to evaluation (e.g. goal-free evaluation, Scriven, 1974; naturalistic, Guber and Lincoln, 1981; needs assessment, Stufflebeam, 1977; systematic, Rossi, Freeman and Wright, 1979; utilisation focused, qualitative evaluation, Patton, 1978, 1980. |
As a practitioner who has been in the evaluation game for a long time, I was very keen to get Anona’s thoughts on the changes she had seen in the field of evaluation.
| A lot has changed! The discipline of evaluation in Australia in those early years was owned by academia and generally regarded as a form of research. The main purposes for evaluation in those days were to improve government programs or to monitor performance.
Evaluation has expanded: from a focus on government programs to all areas of endeavour; from a small field to a core competency of professional roles; and from academia to consulting. There’s also an increased focus on internal organisation evaluation and performance measurement. |
Another area I was keen to get Anona’s input were skills that evaluators need now.
| The basic evaluation skills have not changed very much, but new skills are required for the use of technology, implementing agile organisations and data mining. |
In her speech to the Australasian Evaluation Society International Conference in 2015 on the future of evaluation, Anona expanded on this theme.
| Governments are coping with much more complex international and local environments. They face new challenges: globalisation of trade, global conflict, climate change, the gap between rich and poor, rapid advances in information and communications technology, and generational shifts in values. Locally, Australian governments are experiencing a combination of slower economic growth, an aging population, shrinking PAYE taxes, and growth in global companies that can manipulate their finances to minimise their tax. Society, too, is experiencing massive changes evident in global migration, and the influence of social media. At the same time, a more informed electorate is emerging, composed of a diversity of communities with many different values, but all with rising expectations |
Anona then went on from the original question on what skills are needed to what could be the role of evaluators in the present day.
| In this environment, what is the role of evaluators? Well, we still have a role within organisations as designers of programs, determining needs and evaluating performance |
I then took this question further and asked about the areas in which evaluators and the field of evaluation could make a difference.
Anona suggested five directions in which evaluators could have a major impact:
- Addressing social issues
Now is the time to extend the focus of our activities and use our skills to address some of the larger social issues. Evaluators have a role in establishing the value to society, and to organisations, of actions that count as corporate social responsibility and sustainability. While sustainability has different meanings in different contexts, there is a need for measures that allow comparability at least between entities engaged in the same industry or service.
- Flexibility of methods
Evaluators have traditionally used social science methods to determine the worth of programs. The debate over qualitative versus quantitative methodologies is surely over. We need the different data provided by both methodologies. The new debate is probably about social science methods versus financial methods. Value for money is becoming a standard component of evaluation. Whether in health, education or social services, it is time that evaluators addressed the reality that financial decisions drive governments as much as political ones and answering financial questions requires financial analysis.
- New technology
Evaluators need to take advantage of new data mining and other IT systems that are powerful tools for analysis and communication. This means adding financial analysis, modelling and data mining to the evaluator’s competency frameworks.
- Growing importance of governance
Across the world, there is a greater emphasis on best practice governance. Corporate governance is defined as the overarching framework of rules, relationships and processes within and by which authority is exercised and controlled in order to achieve objectives. Standards for governance are now issued by governments and professional associations. In the higher education system, universities must meet the governance standards, set by TESQA, called the Threshold standards.
- Growing need for evaluation
Trade agreements with New Zealand, several of the 10 ASEAN countries, India, South Korea and China, etc. are opening up new avenues for Australian trade and services. Services generate 72.7% of Australia’s GDP. The services that will be required are financial, education, business, transport, health, and government services. Every new government program will require an evaluation.
As a final point, I asked Anona what role the AES should have in ensuring the future of evaluation.
Her response was very much Anona – visionary and straight to the point.
|
The AES must address some of the big issues facing society, not only Indigenous and failing organisational issues; not only failures, but how to achieve success. The AES may need to fund some significant projects to market itself. There’s an opportunity to tie the image of AES to the use of agile technology and to the investigation of the new organisation structures, modes of employment, threats to democracy and of unstable government, and cross-border cultural conflicts. |
-------------------------------------------------------------
Emeritus Professor Anona Armstrong is the Director of the Centre for Corporate Governance Research – Victoria University of Technology, and Chair of the Board of Southern Cross Institute of Higher Education. She received an AM Order of Australia, in recognition of her contribution to community and education.
The material for this article is taken from interview questions completed by Anona as well as extracts from her welcome address to the Australian Evaluation Society Conference, Reaching Across Borders Melbourne, 5-9 September 2015.