by Charlie Tulloch

As we move into 2021 after an interrupted 2020, it is a good time to reflect on the place of evaluators in the working world. It is clear that many sectors and vocations have been forced to significantly upscale, downscale or adapt to changing economic and global circumstances.

Fortunately for us, there remains a central role for evaluation to play in the face of increasing challenges, demanding an ongoing need for analysis of policy and program successes and failures. Indeed, evaluators now face an increasingly diverse set of choices when it comes to defining their career directions.

The final Australian Evaluation Society's Victorian seminar of 2020 explored this topic in depth, drawing on the wisdom and experiences of six fantastic evaluators of different ages, genders, study backgrounds and vocational sectors (academia, private, government, international development, philanthropy). This article reflects on the insights from this session.

Panel members reflected that career pathways into evaluation are often muddy and non-linear. Most evaluators are thrown into evaluations without really knowing much about the field or its techniques, let alone the theory and methodological choices.

For many of the panellists, good timing was important for landing jobs in the field. Equally important were the networks of passionate evaluators and mentors who helped them to get a start in the field (and often to provide opportunities for growth).

Lesson one is to take the good opportunities when they present themselves. And lesson two is to back the judgement of those career mentors who see your potential and give you the chance to learn and grow in the evaluation field.

In relation to networking, panellists noted a challenge that the evaluation profession faces – that it is not strictly a ‘profession’ at all. Evaluators do not benefit from being a professionalised discipline (such as accounting, law, trades) and so interpersonal networks and professional learning pathways are relatively informal. On the upside, there are few (if any) barriers to entry into the profession. And the AES of course does a wonderful job in linking evaluators together harmoniously and collegially.

Returning to the earlier point about evaluators being in the deep end from the start, this leap need not be extreme. Most panel members found that there were career opportunities that married their personal interests/skills with their evaluation interests. Naturally, evaluators tended to work within or across sectors where they had past career experience or deeper interests and insight. This tended to short-circuit the long road to developing expertise both as an evaluator, as well as in a particular field, and helped them to evaluate more effectively.

Lesson three is to ‘know thyself’ and your personal and professional interests, as there is likely an evaluation role in demand for your passion and knowledge.

Panellists observed that what each of them do day-to-day as evaluators is quite different. This is a refreshing advantage of work in this field. For example, in international development, work tasks relied on fostering trusting relationships and planning and undertaking often manual (rather than technological) data collection and analysis.

In applied evaluation consulting roles, daily pressures varied based on project workloads, the skills/availability of the broader team and the complexities of the subject area under review. Academic roles faced a juggle between teaching, project and research workloads. Some roles involved extensive travel while others were more consistent in their locale. As a rule, more quantitative work is desk-bound, while qualitative research could see evaluators attending focus groups and interviews in various locations. 

Lesson four is to understand the ‘day in the life’ for evaluators in different settings and match this as closely as possible to your working preferences.

There were differences in content needed too. It was felt that knowledge of evaluation theory was far less demanded in some applied consulting roles than in academic circles, but that this theory could nonetheless provide a solid foundation for methodology design and evaluation conduct.

Across the board, panellists felt that evaluators need to try to safeguard the quality of evaluation work, whether our own or others. Panellists in internal evaluation roles reflected that completed evaluations were highly variable in their quality and methodological rigor.

Lesson five is to drive quality across the board to improve the reputation of evaluation work.

Other summary reflections from the panellists included: 

• The work of evaluation needs to find its influence and place in the political environment, while maintaining independence and objectivity.
• There is often a juggle between validity and utility of evaluation work, with a need to strike a balance between rigor and time, budget, data and other constraints that often define what is possible for evaluators to achieve.
• There needs to be an ability to share evaluation messages in a compelling way via reports, but this needs to be done in a way that does not dilute the robustness of findings.

My personal reflection on the session is that “there has never been a better time to be an evaluator”. Why is this? In my observation, the career options for evaluators have been growing steadily over recent years, both in their quantity, diversity and seniority. Internal evaluation roles have been created within government departments/agencies, not-for-profit organisations, private sector consultancies, academic faculties and philanthropic organisations. The roles we are able to pursue are many and varied.

There is also a growing recognition about the important role of evaluation and evidence-related monitoring and research in advancing good policy and government. The focus on evidence has arguably been extended due to COVID-19, with an emphasis on testing, policy responses and achievement of shared community-wide outcomes.

Policy challenges are often met with calls for better evaluation. The Productivity Commission has been a strong advocate for better practice. And there are significant evaluation-related practice resources available online.

My sixth and final lesson therefore is that evaluators have many options and that interesting and diverse career pathways await those who dedicate themselves to learning about and applying their skills in the evaluation field.

Just remember that good work takes time, effort and dedication, but there are lifelong rewards from building and applying your expertise. And if all else fails, you will have a great network of evaluators to share your war stories with via the AES!

----------------------------

Charlie Tulloch is a policy and evaluation consultant who supports public sector leaders to plan, implement and evaluate effective policies. This email address is being protected from spambots. You need JavaScript enabled to view it.