In conversation with AES Fellow Nan Wehipeihana

I fell into evaluation! I was working for a research company, and they also had a social research division – evaluation was very new, and I was really interested in it. I liked the idea that publicly funded programs and services should be scrutinised to make sure they were doing a good job and applauded the idea of accountability – to government, the taxpayer but mostly to the service users or program participants. I also wanted to find out how to support programs and agencies to deliver high quality services and programs to Māori.

I think I’m probably best known for three main areas: Indigenous Evaluation and Kaupapa Māori Evaluation, Developmental Evaluation and developing culturally grounded evaluation and outcome frameworks.

I think there have been many challenges. Two main challenges come to mind. Firstly, advocating for the validity and legitimacy of Kaupapa Māori and Indigenous knowledge systems has been a priority. In the last decade particularly we’ve seen the visibility and position of Indigenous people and cultures come to the fore nationally and internationally, and this has positively influenced our practice as evaluator.

I think the other challenge that we all face is being a reflective practitioner. How do we keep learning and how do we incorporate this learning into our practice? Evaluation conferences, evaluation and research journals, and evaluation networks or communities of practices have been valuable for my learning. Also, my Kinnect group colleagues have been an important forum to grapple with new ideas and issues affecting evaluation and our practices.

I guess a highlight for me was my 2013 AES keynote. I shared a reflective model, now called shifting the locus of power; that helps people to reflect on their positioning in evaluation, and in different contexts and settings such as disability, education, health, and with indigenous peoples. The ripple effects are still being felt and it’s influencing thinking in education, health, and international development.

Being one of the early adopters of Developmental Evaluation (DE) has also been a highlight. I’ve presented on DE, run DE workshops across the globe and was a co-editor of the text Developmental Evaluation Exemplars: Principles in Practice with Michael Quinn Patton and Kate McKegg. Michael was a generous and supportive co-editor and working alongside C. Deb Laughton from Guilford publishing was a valuable learning experience.

My major influence has come from my parents. They were highly involved in iwi (tribal) activities and active contributors to school and community initiatives. I remember having foster kids coming to stay with us, embodying manaakitanga or an ethic of care my parents lived by. My parents modelled the responsibility we had to work for the good of our community and iwi (tribe). My mother was involved in the Catholic Church, so I was exposed to structural analysis and liberation theology in my late teens.

John Owen, Patricia Rogers and Sue Funnell were part of my initial evaluation grounding and the works of Michael Scriven, Michael Patton, Carole Weiss and Eleanor Chelimsky.  In New Zealand people such as Linda Smith, Fiona Cram, and Helen Moewaka-Barnes advocating for Kaupapa Māori evaluation. I had the privilege of meeting and getting to know Jennifer Greene, Ernie House, Donna Mertens, Karen Kirkhart and Rodney Hopson and Stafford Hood etc. Their thinking was particularly impactful, in part because their thinking reflected and affirmed my own values and what I thought was important in evaluation. My Māori evaluation colleagues Kataraina Pipi, Louise Were, Debbie Goodwin and Laurie Porima have been supportive and kept me anchored to Māori and tribal ways of knowing and being.

In my time, we’ve seen more explicit values-anchored approaches to evaluation such as Empowerment Evaluation, Transformative evaluation, and Culturally Responsive Evaluation. And we’ve extended our view of evaluation as a technocratic craft to evaluation serving the public good. There has been a more intentional response to issues such as social justice, equity, and racism.

Also in the last decade, we’ve seen the emergence of systems thinking and complexity-informed methods and approaches such as developmental evaluation. So, the scope of our evaluation practice sees us grappling with local, national, and global ‘wicked problems’ such as fresh water, food sovereignty, climate change, and pandemics.

I don’t think you get into or stay in evaluation unless you have an enquiring mind. So critical thinking or evaluative thinking as both a disposition and skillset is important.

My practice is more intentionally mixed methods, so evaluators need to have both quantitative and qualitative skills. Also, good facilitation skills are needed both for data collection and for engaging with communities and with our clients. A deep methodological toolbox and strong facilitation skills are needed.

Our clients are typically time poor. A key skill is communicating our findings and evaluative judgements simply and clearly. This includes different reporting formats such as single page info-graphic reports, summary reports, as well as full end-of-project evaluation reports.

Decision makers are often under time pressure when having to make program decisions either part way through the program or towards the end. They need information on the run, so progressive insights (e.g., oral presentations or written reports) are needed to inform decision-making. This is different from ‘the old’ mid and end-of-project reporting.

Increasingly evaluations need to provide economic, financial, and cost modelling information. We don’t have to have or be expert in all those areas ourselves, but we need to build teams and work with others – not always evaluators –  to include these elements in the design and conduct of the evaluation.

Equity and the environment. Equity is about contributing to better world, not just for Māori but for all peoples and communities. The economic divide between the rich and poor is widening. I think it’s important to make a start. While planning is important, I think it’s important to get started and not wait for the perfect plan. The world is changing so quickly, that plans can rapidly become outdated. There is an environmental imperative – climate change and degradation threaten the environmental legacy we leave for future generations. Achieving the deliberate inclusion of environmental or climate change in non-climate or environment-specific evaluations is still a work in progress.

Membership in AES has been very beneficial. It has been an important learning community and evaluation community of practice. My early formal evaluation education was through AES conference and branch workshops as well as week-long courses in New Zealand. John Owen, Patricia Rogers and Sue Funnell were key educators. The branch workshops and networking events were important support and community-building activities, particularly in the late 80’s and early 90’s when evaluation was new to the NZ public sector.

I think it’s challenging. In New Zealand we have a high demand for evaluation. However, there are other competing fields such as co-design, data analytics and data science which carry out evaluation-like activities e.g., assess performance or effectiveness without the grounding or theory that is core to evaluation. I think AES needs to help members stay focused on our core function and purposes – making evidence-based evaluative judgements and evaluation as a learning mechanism.

Perhaps there are opportunities for the AES to collaborate with similar professional bodies, beengaged in state and national conversations, planning and strategy around evaluation – and can support member involvement and contributions to these forums. More difficult I think is for AES to be more visible and finding ways which serve the profession and the world we want to create.

Our theories and methods, particularly our methods of valuing, are a core attribute of our practice. This is what we utilise to make our evaluation judgments. However they have to be sound so hone your craft and do the basics. You also need to read broadly, not just evaluation as it is important to understand the world and the issues, so whether it is social justice, climate change, innovation –it is important to be informed. In the past I’ve been part of various networking groups which meet monthly or bi-monthly: each person picks a topic, a chapter, a paper and a key question or questions guide the reading and reflection. Often it is something that I would not been aware of or picked up and read. Also, it is important to build a network of people that you can have conversations with; about specific evaluation topics or more broadly. Read widely in the field and outside of it. Go to workshops. Be challenged in your thinking!