By Thushara Dibley and Lena Etuk

Australia is a culturally and linguistically diverse country. According to the 2021 census nearly half of all Australians (48%) had at least one parent born overseas, and nearly one third (28%) were themselves born overseas (Australian Bureau of Statistics 2022a; 2022b). One fifth of Australians (22%) speak a language other than English at home. Of those, 15% have low English proficiency (almost 1 million people) (Australian Bureau of Statistics 2022a). These statistics make it clear that Australia is a culturally, linguistically and ethnically rich and complex country. 

Despite this rich cultural diversity, current evaluation methods and practices do not fully reflect, take advantage of or cater to this diversity. Key sources of information about evaluation in Australia are devoid of guidance or literature about best practice approaches to engaging culturally diverse population. While many valuable resources exist for appropriately engaging First Nations communities in evaluation, the considerations required to undertake evaluations that engage with non-English speaking or non-Anglo migrant and refugee communities in Australia are quite different. We currently see this omission in a variety of places and spaces, including the resources provided on the Australian Evaluation Society website, the Commonwealth's Australian Centre for Evaluation website and in the articles published in the Evaluation Journal of Australia (EJA). In the EJA, for example, with the exception of one or two articles, there is almost no indication of how evaluators have engaged with migrant, refugee or other multicultural communities in this country.

In the context of this cultural diversity, it is striking that so little guidance is provided on how to undertake evaluations within Australia that are responsive to its cultural diversity. In this short piece, we explore the key elements of culturally responsive evaluation, and then reflect on how these can be put into practice within the Australian evaluation eco-system, which we define as consisting of three levels – the macro (policies and practices at the government level), meso (organisational level) and micro (individual level). We argue that there is a role, and a responsibility, of actors at each of these levels to create an enabling environment for culturally responsive evaluation in Australia. Doing so requires government organisations, non-government organisations and evaluators to become more aware of culture and cultural difference, and to invest time and funding into better understanding these differences and what they can offer to program implementation and evaluation. The effects of investing resources into more culturally responsive approaches will be felt beyond just the programs themselves – these practices have the potential to create more inclusive pathways for people from culturally diverse backgrounds to more fully contribute and participate in society. 

Culturally responsive evaluation

Culture is "learned and shared behaviour, values, customs and beliefs common to a particular group" (Frierson et al. 2010, 75). These groups could be bound by shared ethnicity, ability, religion, age or even a shared hobby. In the context of thinking about evaluation, it is important to understand the way that culture shapes how knowledge is collected and understood. Culture is not only "an object of research and evaluation, it also informs and shapes the values, philosophies, and methodologies of research, including the researchers themselves" (Acree and Chouinard 2020, 203). By thinking of culture as both the norms and values that hold a group together, but also the framework for approaching knowledge collection and analysis, we can see how evaluation that responds to culture can take many diverse forms.

The growing field of research about culturally responsive evaluation offers one approach for how evaluations undertaken in Australia can be better informed by and responsive to the cultural diversity that characterises this country. Originating within the United States, culturally responsive evaluation aims to centre marginalised voices in evaluation and thus to democratise the evaluation process. In particular, the approach emerged from a desire to ensure that the lived experiences of African Americans informed the design and implementation of evaluations that concerned their communities (Hopson 2012). The evaluation practice has also been shaped considerably by the commitment of Indigenous evaluators across the globe to decolonising approaches that find ways to draw on Indigenous ways of knowing, being and doing. These approaches challenge the assumption that there is one true and objective approach to evaluation, and reframe Western methods of evaluation as one alongside many other options (LaFrance and Nichols 2008; Smith 2021)

These historical roots have contributed to this approach to evaluation being associated with emancipatory values and practices that draw attention to the presence and influence of power. Culturally responsive evaluation has been characterised by values such as social justice and equity (Greene 2024; Hopson 2012), and some users of this approach to evaluation are driven by a desire to address past injustice and promote social change (Chouinard and Cram 2019). In addition, culturally responsive approaches tend to involve stakeholders in ways that are sensitive to the dynamics of power within the context that the evaluation is occurring (Kushnier et al. 2023; Frierson et al. 2010). Engaging community stakeholders throughout the evaluation has been identified as an important component of ensuring that the existing dynamics of power are not reinforced and replicated through the evaluation.

Methodologically, evaluators using a culturally responsive approach have tended to draw on practices that encourage the active participation of marginalised participants in the evaluation process (Acree and Chouinard 2020; Frierson et al. 2010; Kushnier et al. 2023). In many cases, these methods have been the more participatory approaches from within the largely Western based 'toolbox' of research and evaluation tools. Though, increasingly, there is exploration of how non-Western ways of thinking and knowing intersect with and can be used as part of culturally responsive evaluations (Dinh, Worth, and Haire 2019; Dinh et al. 2019; Zaveri and Nandi 2023). 

Creating a more culturally aware evaluation ecosystem in Australia

Australia as a leader in culturally responsive evaluation

Australia's unique cultural and linguistic landscape means as a country we have an opportunity to take a leading role in championing and showcasing innovations in culturally responsive evaluation. Stepping up to this challenge requires change at the policy and organisational levels of the evaluation ecosystem, and will be driven by both individual people working at these different levels and the collective efforts of people who are passionate about this issue. Individual choices to be more curious, to be more self-reflective and to be advocates for culturally responsive approaches to evaluation, alongside collective efforts to change practices and power structures, can potentially lead to changes to our evaluation landscape over time so that it more fully reflects and responds to the cultural and linguistic wealth of our country. 

-------------------------------------

REFERENCES
Acree, Jeremy, and Jill Anne Chouinard. 2020. "Exploring Use and Influence in Culturally Responsive Approaches to Evaluation: A Review of the Empirical Literature." American Journal of Evaluation 41 (2): 201–15.https://doi.org/10.1177/1098214019879505.
Australian Bureau of Statistics. 2022a. "2021 Census: Nearly Half of Australians Have a Parent Born Overseas." June 28, 2022. https://www.abs.gov.au/media-centre/media-releases/2021-census-nearly-half-australians-have-parent-born-overseas.
———. 2022b. "Cultural Diversity: Census, 2021." December 1, 2022.https://www.abs.gov.au/statistics/people/people-and-communities/cultural-diversity-census/latest-release.
Chouinard, Jill Anne, and Fiona Cram. 2019. Situating Culturally Responsive Approaches to Evaluation: Empirical Implications for Theory and Practice. Thousand Oaks, CA: Sage.
Dinh, Kathryn, Heather Worth, and Bridget Haire. 2019. "Buddhist Evaluation: Applying a Buddhist World View to the Most Significant Change Technique." Evaluation 25 (4): 477–95.https://doi.org/10.1177/1356389019841654.
Dinh, Kathryn, Heather Worth, Bridget Haire, and Khuat Thu Hong. 2019. "Confucian Evaluation: Reframing Contribution Analysis Using a Confucian Lens." American Journal of Evaluation 40 (4): 562–74.https://doi.org/10.1177/1098214018813008.
Frierson, Henry T., Stafford Hood, Gerunda B. Hughes, and Veronica G. Thomas. 2010. "A Guide to Conducting Culturally Responsive Evaluations." In The 2010 User-Friendly Handbook for Project Evaluation, edited by Joy Frechtling. Alexandria, V.A.: National Science Foundation.
Greene, Jennifer C. 2024. "The SAGE Handbook of Evaluation." In , by pages 119-140. SAGE Publications Ltd.   https://doi.org/10.4135/9781848608078.
Hopson, Rodney K. 2012. "Reclaiming Knowledge at the Margins: Culturally Responsive Evaluation in the Current Evaluation Moment." In The SAGE International Handbook of Educational Evaluation, edited by Katherine E. Ryan and J. Bradley Cousins, 429–46. Thousand Oaks, CA: Sage.
Kirkhart, Karen E. 2011. "Culture and Influence in Multisite Evaluation." New Directions for Evaluation 2011 (129): 73–85. https://doi.org/10.1002/ev.356.
Kushnier, Lauren, Shevaun Nadin, Mary Ellen Hill, Mischa Taylor, Shelly Jun, Christopher J. Mushquash, Giulia Puinean, and Rebecca Gokiert. 2023. "Culturally Responsive Evaluation: A Scoping Review of the Evaluation Literature." Evaluation and Program Planning 100 (October):102322.https://doi.org/10.1016/j.evalprogplan.2023.102322.
LaFrance, Joan, and Richard Nichols. 2008. "Reframing Evaluation: Defining an Indigenous Evaluation Framework." Canadian Journal of Program Evaluation 23 (2): 13–31.https://doi.org/10.3138/cjpe.23.003.
Smith, Linda Tuhwai-Smithi. 2021. Decolonizing Methodologies: Research and Indigenous Peoples. London: Zed Books.
Zaveri, Sonal, and Rajib Nandi. 2023. Local Wisdom Matters: Reflections on Evaluation Theory and Practice from South Asia. GENSA COE South Asia. Local Wisdom Matters: Reflections on Evaluation Theory and Practice from South Asia. https://www.researchgate.net/publication/371226543_Local_Wisdom_Matters_Reflections_on_Evaluation_Theory_and_Practice_from_South_Asia.

-------------------------------------

ABOUT THE AUTHORS