Fostering Evidence-Based Decision-Making in a Nonprofit: Family Services Ottawa

February 23, 2016

Family Services Ottawa (FSO) provides counselling services and educational programs to help children, adults and their families gain improved mental health, emotional well-being and social connections.  FSO’s leadership team recently completed Innoweave’s Impact and Strategic Clarity module – a 6-month coaching process in which organizations clarify what they aim to achieve, how they will achieve it, and how to measure their success.

In this interview, FSO’s Executive Director Kathryn Hill talks to their ISC coach, Linda Graupner, about how the process has helped to break down program silos, fostered an organizational culture of evidence-based decision-making, and given the organization confidence to communicate clearly the impact it is having.

What motivated you to seek Impact and Strategic Clarity coaching?  

Kathryn:  Like most organizations, we had a strategic plan with all the usual strategies:  raise more funds, be more innovative, provide excellent service and so on.  We collected data on our various programs, and did customer satisfaction surveys. We collected different data for different programs, which made it difficult to communicate our impact across all programs. We wanted more clarity about how our different programs work together and what sort of impact we were having on people’s lives.   We wondered about the many different programs we offered; external sources often challenged us to be more singularly focused in our service delivery offerings. We had our own assumptions about this, but wanted to be able to base our decisions on hard evidence.  We liked the way that the Innoweave module would help us develop a theory of change and then test it using real data and external evidence.

Your leadership team put a lot of effort into developing a theory of change.  What did you learn from this?  

Kathryn:  At first it was really hard. How could we possibly define one impact across all of our programs?  But with our coach’s help we developed an intended impact statement and a theory of change that clarified what outcomes we hold ourselves accountable for, and how everything we do leads ultimately to increased resiliency for children, adults and their families.   We had our first big “aha” moment when we discovered that it wasn’t just a bunch of different programs we were running.  There are actually two closely-connected sets of activities –intervention and prevention.  We realized that our clients needed both to achieve the ultimate impact of resiliency.   The success of our organization was because we offered much more than counselling – in fact, our prevention programs are critical to achieving our outcomes. We were also really excited by the prospect of articulating a clear impact statement that we could then hold ourselves accountable to across all our programs.

What kind of research did you do?

Kathryn:  First of all, we wanted to validate that our activities did indeed lead to our ultimate impact of increased resiliency.  We received a small grant from a donor that was interested in supporting research, and used this to hire 2 researchers.  They did a literature review to help us understand and define resilience, and helped us identify 12 protective factors that lead to increased resilience.   We were particularly concerned that our prevention programs were “outliers” and would not contribute to building resilience.  However, the research showed this was not the case.

During the Innoweave process we developed a hypothesis about which programs lead to which outcomes.  We are now testing that hypothesis using client surveys to find out whether our clients are experiencing the kinds of improvements we think they are.  For example, do clients have improved coping skills?  Do they feel less isolated?  We are just about finished our first pilot implementing surveys across all programs and services, and are looking forward to analyzing the data to validate which programs are achieving which outcomes.

What specific changes have you made as a result of this work?

Kathryn:  The main changes to date relate to our internal culture and our external confidence.  Our leadership team is fostering evidence-based decision-making and an understanding of outcomes.  We have started to see small changes in organizational culture.  We’re focused on understanding how everyone’s work fits into the bigger picture and how it creates the desired outcomes.  We feel more accountable now for these outcomes.  These changes can create some anxiety, but it also strengthens our commitment to excellence and our desire for learning and improvement.  We recognize that we don’t have all the answers, but we have the processes in place to use data and evidence to continually improve.   We are creating a culture of evidenced-based decision making.   I think it’s fair to say that the Innoweave work has brought our organization closer together and helped us to see the bigger picture, outside of our own program silos.  This includes the Board, who were engaged in every step of the process.

Externally we have found ourselves on much stronger footing when explaining to funders and other stakeholders what we do.   We can speak with confidence about the impact we are having, knowing that our model is backed up with real evidence.

 

Interviewer:  What’s next for Family Services Ottawa?

Kathryn:  Very soon we will have completed customer surveys for all our programs.  We are really excited about analyzing this data (yes, we’re actually excited about data!!).   The data will enable us to validate our theory of change by confirming which programs lead to which outcomes.   We are revamping our intake and referral process to ensure that clients get access to the programs and services needed to achieve the outcomes that match their particular needs.   We are undertaking a lean process review in one of our programs and looking at our organizational structure to ensure it supports our new understanding of FSO programs and activities, particularly the distinction between prevention and intervention programs.   We’ve also had some recent successes on the funding front, which we attribute to our ability to speak with clarity and confidence about our outcomes.

The Innoweave journey was just the beginning for us.  We find ourselves at a new juncture as an organization.  We have lots more to do, but we feel much more confident now that we have the tools and culture to understand, measure and ultimately strengthen our impact.