Public Sector Evaluation Award awarded to Te Ihuwaka | Education Evaluation Centre, Education Review Office for Evaluation of learning in residential care
The Education Review Office is the public service department of New Zealand charged with reviewing and publicly reporting on the quality of education and care of students in all New Zealand schools and early childhood services.
This evaluation study looked at the quality of education for students in Oranga Tamariki Care and Protection and Youth Justice residences and how it can be improved. Children and young people who are placed in Oranga Tamariki residential care are among the most at risk of poor outcomes later in life. This was the first nationwide, system-level evaluation undertaken by the ERO. The study developed an excellent theory of change based on a published systems framework and used this effectively to develop the evaluation methodology which was robust and of a very high quality. The study developed a unique set of indicators of education performance for the residences, using a 4-point rubric to define the quality of practice, which will be useful in future evaluation studies and service delivery. The rubric was developed in consultation with the education providers to ensure their support for its use. Given the target population of school aged children, the evaluation team developed processes to give voice to these students including an appropriate reporting process. The evaluation team recognized the need to strongly engage stakeholders in the design and implementation of the study. Importantly, the team communicated regularly with the stakeholders on the study findings and developed a range of hard copies and online publications including the substantive report, a summary document, and four guides for students; parents and whanau (community members); leaders and teachers; and social workers. The evaluation team used a range of innovative approaches to enhance utilization of the study findings and products. As a result, there are a number of impacts that have already resulted from the study both in developing improved evaluation practice and decisions about the future of the residential care program.