by Anthea Rutter
All of us in the AES were greatly shocked and saddened by the sudden death of Jenny Neale. Jenny had been a member of the Australian Evaluation Society for over 20 years and was an active contributor to the society both in her local Regional Network Committee in Wellington as well as a regular contributor at the AES International Conferences.
Jenny was a Senior Research Fellow Health Services Research Centre, Faculty of Health, University of Wellington, New Zealand.
I interviewed Jenny last year and was rewarded by a frank discussion of life in the field of evaluation, its ups and downs and its frustrations!
The first question I asked her to reflect on was, what brought her into the field of evaluation?
| I have swapped between being a researcher and then working in the evaluation field. I first got into evaluation through the Wellington Evaluation Group. I think it would be correct to say that in those days they were a fairly loose list of people. At that stage I was teaching an applied research degree, and I guess that would have been the 90’s. |
The Fellows have a diverse range of evaluation or research interests, which keep them involved in the profession. I was, therefore, interested to find out what had been Jenny’s main areas of interest?
| My role for the last 8 years has been that of evaluator in the health services research unit at the University. We evaluate a range of health services initiatives. My main interest has been the social justice area. So, I think I am in the right space, as working in the health services field fits well into that.\ |
Like most of us, a career spanning a number of years usually has a few challenges – some have more than others. So I asked Jenny what have been the major challenges to her practice?
| I think it is something that we are facing again at the moment – people’s understanding of evaluation, what it is and what it does, as well as what it is not. Sometimes people ask you for an evaluation but at the end of the project they decide they want something different. So, it’s that whole issue around what evaluation means. Then there is an idea out there that you can evaluate impact immediately.
Another challenge is that people underestimate the amount of time it takes to actually do an evaluation as people want things yesterday. The other sort of challenge for us is that the main evaluators in NZ are contractors or work for government departments. So, people move in and out of evaluation and research. If you are a more mature person, you sort of drifted into evaluation (as it was a new field in the 80s). So new evaluators, who possibly have been trained in evaluation, talk about new methods and some of them are things we knew years ago. I try very hard to not say “In my day”! |
Apart from challenges to practice, careers also have a number of highlights. It was pleasing to note that highlights for Jenny involved the bringing together of evaluators from Australia and New Zealand.
| One of my early highlights was teaching with John (Owen) on a course in Wellington. The second time he brought his course over to NZ he also brought Ros Hurworth with him. Then we had some feedback which suggested that it would be good to have local content and, subsequently, I provided that. And John ran a course with my master’s students. Another highlight was joining the AES and then getting to know lot of people in the field. Ralph Straton was over here as a visiting scholar at the time of 9/11. He was due to head to the US. But the insurance companies would not cover US travel. So, Ralph stayed in Wellington and we all learnt a lot. It was a very good interchange of ideas and skills. |
Evaluation practice is defined by many factors, including people, evaluation models and cultures. For Jenny, her practice was influenced a lot by Māori and Pacifica.
|
I think a lot comes from where I sit with Māori and Pacifica, there is also a strong social justice factor, particular because we are bi-cultural (the Treaty) but becoming multi-cultural. So, treaties of understanding and friendship with other Pacific nations. We have to be clear that’s it’s important for them as well as Māori. We don’t want evaluations to just be a ‘tick box’, which leaves people worse off than before. I am not a theorist; my research was always applied. There have been other influences which have shaped my practice: listening to people talk at AES conferences. Michael Quinn Patton was also influential, John Owen’s teaching and book was very influential and useful. Also, the Community of Practice within AES was very important.. |
I wanted to find out from Jenny in what ways the field of evaluation has changed during her career.
| Things in evaluation become fashionable and then go out of fashion. But I think that people are realising that it is important. I think it has changed. We now have a number of different theories and practices, for example, realist evaluation and developmental evaluation. The profession has changed from being a broad field where it had practitioners who had a research background to people looking to apply what they know. |
With these changes what skills and competencies are required to keep pace with emerging trends in evaluation practice. Jenny’s response was spot on!
| I think it’s a bit like life skills, you need to be open to different ideas. Listen to people and ideas. Then craft it in the field you are working in. I remember a short debate a while ago about RCTs as being the gold standard, other countries still talk about it as the only method. In social services it does not always work. You need theoreticians and practitioners and the people in between. So, you need both ends of the spectrum. Most of us are in the middle. Both sides ensure that it is a lively debate. You need to be a good listener and adapt key ideas. Problems are the same in several countries, but the context differs. Making sure there is open debate. . |
Jenny was very definite when asked about the key issues that evaluators ought to be thinking as about and seeking to resolve in the next decade.
| The main one which AES is tackling is the professionalism aspect, and quite a bit of work has been done in this area already, and in particular what we need to make it a recognised profession. In government jobs both in NZ and Australia people move between policy, research or evaluation.
The other thing which we must do is to undertake an educative job in explaining what evaluation and monitoring means. |
Finally, I asked Jenny to comment on what she wished she had known before she set out on the evaluation path.
| Certainly, wishing I had known more… But my comment is really a wish – wishing other people understood what evaluation could do and what it was! I guess I assumed that if someone wanted an evaluation, they knew what they would get. Certainly, I think that moving between research and evaluation was advantageous in terms of methodologies and seeing what others were doing. |
This piece is a tribute to a Fellow of the Australian Evaluation Society who made her mark on the work of the Society as well as for the profession of evaluation. I personally regarded her as a friend, and she will be missed.