Shifting from prove to improve: Transitioning to an impact-oriented learning and reporting approach

November 26, 2018

Innoweave recently realized that some of our systems may not be contributing to a healthy evaluation environment for our grantees, and we’d like to make some adaptations to address this.

While taking part in a workshop led by InWithForward, we explored the role ethnographic evaluation tools can play in helping community sector organizations learn from their clients and improve the impact of their services. InWithForward is a leading national expert and practitioner in this field. Content was shared that outlined how this highly beneficial technique is often overlooked or not prioritized over the more traditional forms of evaluation that organizations feel their funders want (whether the funders realize this or not).

When it comes to driving impact, outcomes, learning, and improvements in the community sector, it has become increasingly clear that funders have a critical role to play. But many actors in the funding environment—both grantmakers and grantees alike—have created systems that may inadvertently entrench stagnation and repetition over innovation and growth.

Given that many of the evaluation efforts taking place in the community sector are driven by the needs of funders, as grantmakers, we have a huge opportunity to shift how our grantees evaluate the work we support. Indeed, many funders and community sector organizations are already leading the way and doing this with great success.

Given our experience, Innoweave is in a good position to support promising practices and emerging patterns in this space. Innoweave aims to help community organizations clarify their intended outcomes, and then become capable of achieving those outcomes more effectively, and we know we can use our own evaluation systems to support that objective.

To that end, we want to put ourselves in the shoes of the groups we support. We are taking a look at the evaluation and reporting requirements we place on our grantees through the lens of who benefits from these activities. We’d like to support community organizations to shift away from evaluating simply to meet reporting needs and move towards evaluation that primarily benefits grantees, their beneficiaries, and the communities and important issues they serve.

The outcome of this is a renewed commitment to improve Innoweave’s processes. We are very much interested in evaluation and reporting techniques as learning tools that are employed on an ongoing basis in order to constantly observe and learn, then iterate and improve programming through regular feedback loops, with the goal of deepening impact. As a funder, we should encourage groups to take risks and be bold, to innovate and be creative, and to fail forward by learning from their mistakes and sharing their shortfalls openly. Rather than our reporting tools being geared to sharing successes and outputs, we should place more value on learnings, failures, iterations, and progress toward outcomes.

A high-level comparison of this shift in evaluation and reporting approaches could look something like this:

Evaluation and reporting dynamics	We want to shift away from…	We want to shift toward…
We report on our work because	We have to	We want to
Motivation	Reporting requirements	Program improvements
Benefit	Accountability	Adapting and deepening impact
Beneficiary	Funders	Audiences and issues we serve
Relationship with funders	Funders set reporting expectations	Funders are partners that encourage and support useful evaluation
We want to	Prove	Improve
Workflow	Most evaluation and reporting take place after programming ends (and mid-program when funders require this)	Learning and sharing takes place throughout all stages of program development and execution
We mainly track and report on	Outputs	Progress toward outcomes, changes we had to make, and failures that generated learnings
Processes	Summative evaluation, primarily quantitative	Reflection, observation, learning, Developmental Evaluation, ethnography, primarily qualitative, etc.
Often led by	Development/fundraising staff	Program staff, supported by senior leaders
Competency level	We totally know how to do this	We may need help to do this well
Main storyline	Look how great we did, please give us more money to keep doing this program the same way	Here’s what’s going well, and here’s what we can do to improve our programming, better serve our audience, and deepen our impact
Risk tolerance	Low	High
Creativity, innovation, experimentation, and iteration	Low	High
Failure mindset	Avoid at all costs, high propensity to disregard, environment of consequences	Fail forward (learn from failure and adapt, iterate, grow), acceptance, supportive environment
Feedback loops	Rarely built into programming, perhaps only at the end	Intentionally built in throughout programming or at regular intervals

As we recently announced, we are in the process of implementing a number of important changes at Innoweave, beginning with our Social Enterprise stream.

Without adding to the amount of time or energy we expect our grantees to invest in these activities, Innoweave will now ask groups to report on how they reflected on their work along the way, what they learned, and how they looped that feedback into program improvements to deepen their impact.

We aim to support organizations to adopt learning-based processes that they will continue to utilize after our support ends because they see inherent value in them. Not because we are encouraging them to.

We will also welcome groups reporting that not everything went to plan—indeed this is our expectation. And this will not adversely affect their ability to receive further support from us. Indeed, if a group can demonstrate that their learnings have led to significant program improvements that will deepen future impact, that is exactly the kind of work we want to support.

For groups that may need help developing a learning plan that accomplishes this, Innoweave will welcome exploring how to help them with this, as well, including providing additional coaching supports as needed.

We are excited about this transition and—in keeping with the spirit of this approach—welcome your input, feedback, and questions so that we, too, can continue to learn along the way. Please feel free to contact us at info@innoweave.ca to share your insights anytime.