Developmental Evaluation
Case Study: From food security to life security – Developmental Evaluation with St. Matthew’s House
April 12, 2023
St. Matthew’s House is a multi service agency providing childcare and seniors support services in Hamilton, Ontario. Looking to do things differently to address systemic barriers to food security, Executive Director Renée Wetselaar decided to apply for Developmental Evaluation (DE) coaching support and started a process with Innoweave coach Paul Bakker in 2021. Having a hard time seeing their overall impact through their emergency services, St. Matthew’s House made the bold decision to close their food bank. With Paul’s guidance, they started exploring assumptions around the root causes of food insecurity and then quick-testing them with seniors they serve. They wanted to understand if their food security intervention was effective or if there were other things they needed to be doing down the line that would be more impactful.
“Working with St. Matthews House was a great experience. They were truly open to a Developmental Evaluation mindset; asking not just “how can we improve our current efforts,” but also asking, “should we be doing different things.”
– Paul Bakker, Innoweave Developmental Evaluation coach
Credit: St. Matthew’s House
They decided to test a food delivery and mobile market system and observed what happened. “As we gained more data, Paul said, it became clearer to the staff that senior’s food insecurity related more to their mobility and transportation limitations than to their food literacy.” Staff also observed that bringing food to seniors at their home creates an opportunity for connection, where people are comfortable opening up about their needs, sometimes relating to eviction threats or even abuse. Renée describes having an “aha” moment when they realized: “It’s not only about food security, it’s about life security”, where food is not the only parameter for impact. This realization reinforced their decision to shift their focus, and they’ve since been offering more services, including working with paralegals and social workers.
The developmental evaluation approach allowed us to look at not just a short-term intervention, but a medium- and long-term intervention. There are aspects of poverty that are intertwined, and we need to use an approach to bring systems together.
– Renée Wetselaar, Executive Director, St. Matthew’s House
Credit: St. Matthew’s House
Listening to those most impacted by inequity
Gathering data through the DE process allowed St. Matthew’s House to discover sector needs and to adapt to those needs. As Paul shared, “survey data also suggested that there was a lack of awareness of their services among many seniors, especially among equity-deserving groups. That insight led to St. Matthew’s seeking partnerships to better communicate with and service different cultural groups.”
When asked if Renée had any learnings to share with organizations thinking of engaging in a DE process, she offered some advice: “You have to be open to change, to question your past success and to take risks. Don’t assume that everything you have been doing is the right way to do it. Listen to the voices of people who are the most impacted by inequity.”
A simplified tool to help organizations use a Developmental Evaluation approach
Working with St. Matthew’s furthered Paul’s own DE practice, as he developed a template summarizing the whole DE process into 2 pages. The template can be used at regular team meetings to review how your organization’s programs and the systems you address are developing, make sense of data collected, update your evaluative questions, and plan for next steps for data collection and programming.
What is Developmental Evaluation?
Developmental Evaluation (DE) serves a different purpose than traditional forms of evaluation. In contrast to formative or summative evaluation, which looks back at completed work, DE supports the creation, development, or radical adaptation of a model in real-time, as the work or project unfolds. DE It is particularly well-suited to initiatives that are being implemented in quickly changing contexts.
Many thanks to Renée Wetselaar and Paul Bakker. For more information about Saint-Matthew’s house, visit St. Matthew’s House | St. Matthew’s House is a place of hope that supports people in need in the City of Hamilton. (stmatthewshouse.ca)