In conversation with AES Fellow Dorothy Lucks

Coming into evaluation was not a conscious pathway. I was then on the management team of a national not for profit in Sri Lanka and had been working in projects we needed to monitor, measure and report on. It was getting bigger and bigger and more complex, so I had to think of different ways to evaluate so I could report. I was also learning to deal with different funders, reporting styles, as well as different demands from them in terms of measurement. Then I found I was doing it – I was doing monitoring and evaluation! To be honest I didn’t really realise it until I moved to Australia and looked for like-minded people and accidently found the AES. The AES was the first time that I connected with other people who were doing the same thing.

My passion is sustainable development. I use ‘sustainable development’ broadly, i.e., the connection between the environment, the economic and the social. My PhD topic was at the intersect of those three. This often brings me to into evaluations that are complex, covering the intersections between for instance, social justice and regional development, first nations’ development aspirations, eco-system development and agriculture in fragile locations, economic regeneration after civil conflicts and many others. This means applying a blend of evaluation methodologies and always very diverse stakeholder relationships.

There are many challenges, and how do you distil those down into different aspects? From a practical point of view, the real challenge is the uncertainty of the workflow. Working as an independent evaluator, you are reliant on contracts. How do you keep the balance between keeping the work flowing with ensuring good evaluation practice. The challenge is, how do you deliver high quality evaluations within the realism of what can be achieved in the contracts you have. I would love to have the luxury of doing some research into evaluation approaches, but that is not funded, and you have to survive, so there is always an element of compromise, which is not always comfortable. That balance between saying you want to do your best, and to answer the questions that need to be asked, and draw out the learnings, means there is a tension between quality and depth and realism, and knowing when to make that call, both for outcomes and sanity.

There are many good points in evaluation practice but it’s hard to pick highlights; the most important aspect is later on when the evaluation is finished when you can see people working on your recommendations. Projects then become programs, and the terminology you have used is adopted to build mutual understanding between different stakeholders, then your analyses are used for the next stage of planning.

In terms of personal highlights, being part of the AES has been important, and that allowed me to become engaged with the international evaluation sector, through the International Organisation for Cooperation in Evaluation (IOCE). I had the privilege of sitting on the IOCE committee for five years, which brought me into contact with evaluators all over the world. I learned a lot then that took me into EvalPartners, and I was one of the authors of the global EvalAgenda 2015-2020. This was an interesting process, bringing together all of the organisations in the world involved in evaluation, and then watching how that was taken into planning for the next 5 years. I have recently been contacted as a key informant for the development of the next EvalAgenda 2030, and it is interesting to see how the next stage is unfolding. I was a founding Co-Chair of the EVALSDGs network globally and it is great to see work coming out of that, which is really important. I think I am now in the process of stepping back from that arena. I have always been committed to capacity building for sustainable development and evaluation, so that is one of my key interests at the moment.

My field-based experience and being at the coal face of making things happen has influenced how I do every evaluation. I look at the theoretical and practical sides. So, I ask questions which others may not be able to ask. When I ask a question – I can ask in-depth questions that people understand, which allows you to go deeper, when you know the realities of implementation, and people understand what you are talking about.

When I talk to others who have not come from the trenches, sometimes they can be harsh, and overly critical i.e. work in Afghanistan they will put down as unsatisfactory, but I know that people are working under incredible situations. So, I have gone in to bat a number of times, told them that this is harsh, you have to give credence to the situations they live and work in. I think there is an element of advocacy needed so people understand that evaluation has to reflect the realities of what is happening on the ground.

Talking about competencies (for AES) was a real defining point in practice, because it influenced me a lot about what competencies are required. That helped me in my self-improvement. For a good evaluator you have to make sure that all those competencies are covered. Competencies have been a huge benefit to the sector. And I feel a lot of ownership of the AES ones, as they are good.
I think that evaluators need to be aware of all of the competencies and need to cover all of those in their team. The most critical competency is ethics: if it is not ethical then how does one deal with that? It’s about knowing when to push and say, we can do this in a better way.

For example, I was asked by an organization in the field of homelessness for an evaluation, but the terms of reference was put in a way that was not ethical, and the methodology – on which they were inflexible – was inappropriate. What do I do there, should I put in a submission and try adjusting the methodology? No, I actually needed to call them on it. I told them that it was not an ethical approach – that I wanted to do it, but not in its present form. As an evaluator your ethics are continually challenged, and we need strategies to address those issues when they arise.
Another required skill is team management – managing teams is important and dealing with different sources of data coming from different people.

The last thing is keeping up to date - the field changes, and if you don’t keep up to date, your practice suffers.

It has moved away from just monitoring and reporting, more from output to outcome and even into impact. Also, from the project and program level evaluation to organisation, thematic and meta evaluations, which involves moving from simple approaches to complexity. I do a lot in that area.

There is also more demand for evaluation now. I am currently involved in an evaluation of the OECD, and this is the first time they have recognized a need for an evaluation, whereas previously they might have just asked for an audit. There is generally more interest and understanding of the potential benefits of evaluation.

I come back to where we started, which was focusing on sustainable development, about balancing the social, environmental and the economic. This is not sufficiently addressed. Look at the NDIS for example. It’s an economic program for people who are highly vulnerable and living in certain contexts. So, how do you find a pathway through that which is ethical, productive and gives the best outcomes and also the best results of taxpayers’ resources?

Evaluators can make a difference by bringing the attention of the decision makers to opportunities that improve the balance between the social and the economic, as well as seeking to resolve issues and result in better outcomes.

Pivoting is good. Having the membership committee is a critical way of doing that. As is having a designated committee which is in touch with members., and leadership on the Board who have a relationship across the world. I find that there is a lot going on in the space of environmental evaluation, although I have to go out of Australia to find that - in Germany, the Netherlands, Switzerland, Canada, as well as some in the US. There’s not much here, in the climate change and environmental evaluation space.

Let me also do a plug for young evaluators! One of the things SDF Global have tried to do is to bring interns into the organisation, and include younger evaluators in evaluation teams to give them real experience in evaluation. One of the highlights is seeing them moving on to work in evaluation. We don’t do enough to create space to bring in new evaluators, some don’t go on, as they cannot get a foot in the door. It’s great to see other organisations taking on young evaluators. The work that AES has started to do around young and emerging evaluators is important.