Archive for November, 2014

Using Developmental Evaluation (DE) to support Social Innovation

by Ken Hoffman, Partner One World Inc  and Innoweave DE Coach.

It is an exciting – and somewhat unsettling – time for governments and social agencies.  Increasingly, they are realizing that conventional program approaches are reaching their limits in dealing with broad, complex issues like homelessness or childhood obesity.  They are seeking out new and innovative approaches and working with new partners, often in broad coalitions.  They are exploring new approaches to funding these initiatives.  To do this kind of innovative work, however, they often find they are in unfamiliar territory where the usual tools just don’t do the job anymore, like being lost in a strange city with a compass but no map.

The more conventional approaches to evaluation – progress and summative evaluation – are not very useful in a situation where you don’t have a full understanding of the problem you are trying to address, much less the solution.  Social innovators need an approach that can help them to better understand the complex issues on which they are working, and give them timely feedback that they can use in rapid prototyping of solutions. Michael Quinn Patton developed the Developmental Evaluation (DE) approach to deal with these exact issues.

DE is an approach that emphasizes learning throughout a process rather than “passing judgment” on whether something was a success or failure.  As such, it is best suited to organizations or groups that are, themselves, interested in learning and innovation.  This innovation mindset involves an openness to new perspectives about an issue, and a willingness to try and learn from different approaches.  This mindset also tolerates failure, as long as you can learn from the experience.

Jamie Gamble describes three main stages in a developmental evaluation.  First, is framing the issue. The people involved need to clarify how they understand the issue they wish to address.  Members of a diverse group of stakeholders may, in fact, have different ways of looking at the same issue; different perspectives can be very useful, providing a more “three-dimensional” understanding of the issue.  The group also needs to put their assumptions about the issue on the table so they can be made explicit.  The stakeholders can then state, the goal they wish to achieve and the conditions that need to be in place in order for that goal to be achieved. This is known as a Theory of Change. This kind of exercise can be very valuable in helping everyone involved to share their thinking and perspectives about an issue, and then to design the intervention.

Second, testing of quick iterations. Sometimes the best way to develop new interventions is to test them out on a small scale, with the intent of learning as much as possible from the testing process. Developmental evaluation is an approach that purposefully captures the learning so it can help to inform improvements.

Third, is tracking the trajectory of the innovation. The path to developing an innovation is seldom straightforward.  Assumptions are tested through the quick iterations.  Learnings from this testing are incorporated into future iterations, and also help to build a better understanding of the issue. Developmental evaluation tracks this path to help to clarify when and how key decisions were made. This information can be particularly useful when trying to disseminate innovations to new settings.

A couple of examples where the DE approach has been used:

  • Eva’s Initiatives, a Toronto social agency working in the area of Youth Homelessness developed the Mobilizing Local Capacity initiative as a strategy to support Canadian communities to develop their own strategies to address Youth Homelessness.  Very few Canadian communities have such strategies.  The model to support these communities was to work with a small number of communities by providing a “coach” and access to expertise from across Canada, as well as a small amount of seed money.  The communities also learned from each other through the process, building a much broader base of experiences.  Eva’s used DE to develop a Theory of Change for the initiative, as well as a process to learn from the participating communities about the effectiveness of the model of support.
  • Childhood obesity and lack of physical activity are significant health problems.  PHE Canada, a national membership-based organization that works with Health and Physical Education teachers, wanted to support the uptake of a whole new approach to supporting the development of school environments that promote health.  Health Promoting Schools helps schools to change their cultures so that everything that happens in a school – from classroom curricula to fundraising – can be viewed through a health promoting lens.  DE was used to develop a social movement approach for disseminating the Health Promoting Schools initiative across Canada.

In both of these cases, DE helped organizations in the process of developing, testing and implementing totally new approaches to addressing complex issues.  DE is a tool that helps these organizations to map a new course across the challenging terrain that they find around them.

ABC Life Literacy Canada partners with Innoweave

Jim Warrington, ABC Life Literacy Canada Board Chair, reflects on their mission and why they chose to work with Innoweave to develop a Theory of Change through the Impact and Strategic Clarity module.

The Theory of Change process, introduced to ABC Life literacy by Lisa Watson, was both timely and mission critical for future growth. The Innoweave Impact and Strategic Clarity Module made sense to us because of four key factors:

  • ABC’s current strategic plan was in its third and final year
  • Our primary funder, the Government of Canada, was potentially changing its funding model from core to project-based support Tuned out to be the case as of July 1, 2014)
  • Gillian Mason, a skilled and committed not-for-profit executive, had recently assumed the leadership of the organization and was familiar with this planning process
  • The ABC Board wanted to see a refreshed mission that focused on greater, measurable outcomes and labour market impacts

We believe this process has been very successful because the “design team” of staff and Board members has worked closely together in mapping our Theory of Change, enabled very professionally by Lisa Watson using McConnell Foundation and Bridgespan Group principles and best practices. In a very short period of time, the organization has created an Impact Statement and clarity for how we’ll get there. A one-page summary of the work has been successfully shared with major stakeholders, including the Government of Canada.

Having this strength of strategic insight has allowed us to continue to plan for growth and a more sustainable future, without relying on core government funding.  This work has allowed ABC Life Literacy Canada to think of itself and its impact in an outcomes-driven fashion, ensuring that literacy and numeracy rates will increase in Canada over the next 10 years. To have all our stakeholders, our Board, our volunteers and our staff march to the same drummer has been a critical component here, and it might not have happened so quickly or so effectively without the Theory of Change process, or Lisa Watson’s informed and focused leadership.

The best example I can think of this process’ impact is an all-day meeting we held on February 1st, 2014. It was a Saturday, and we managed to get all but one Board members out with key staff for a discussion and review of the draft Theory of Change. This enlightened the findings and drove the principles of the final outcome. In my 9 years of volunteering for the organization, I had never seen this level of participation before, especially on personal time.

As we transform the organization post-core funding, this journey ends up being mission critical for our success. We are grateful to the McConnell Foundation for the support they have given us, and the outcomes we will see as a result.

In the end, it is all about adult learners and better, more valuable life experiences, thanks to upskilling. This will impact thousands of Canadians, so thank you.

Gillian Mason, President, ABC Life Literacy Canada, offers her reflections on the development of a Theory of Change:  

ABC found enormous support working with Innoweave through the Theory of Change at this crucial time in our history. Lisa Watson with the insight and support of Sally Fazal, facilitated a process that grounded our staff and board in our key values and our long term vision. It has given us a chance to signal to our stakeholders and partners very clearly, who we are, what we believe and where we are going. We are in a time of unprecedented change and extraordinary possibility at ABC and our experience with Lisa and Sally, has positioned us to adapt and take advantage of some of the exciting opportunities. We are currently growing and changing at a rapid rate while still remaining guided by our Theory of Change and focused on our ultimate goal of increasing adult literacy in Canada.

Additional Notes and Evidence that helped to frame ABC Literacy’s work with Innoweave.

from Jim Warrington, ABC Life Literacy Canada Board Chair.

Literacy and Numeracy in Canada is Declining and Only Action Will Reverse This Trend

  • Over the last 10 years, Canadian literacy rates have dropped: in 2003 we ranked above average compared to other countries, we are now just average
  • Percentages of Canadians with below desired literacy and numeracy rates are staggering: 49% for literacy and 55% for numeracy; this means almost 6 in 10 Canadians do not have the desired level of numeracy skills
  • In the 2013 OECD survey Canada is ranked eleventh in the world, behind the Slovak Republic, Estonia and Australia
  • Canadian youth (16 to 24 years) are under-performing in literacy compared to their OECD counterparts

(Statics from OECD PIAAC 2013 survey)

Essential Skills are Critical to Business Success

  • Essential skills training provides solutions for:
  • Changes in the business
  • Health and safety concerns
  • Paperwork completion and document-use challenges
  • Communication, teamwork, and leadership requirements

(Advancing Workplace Learning study, 2013)

  • Essential skills are the building blocks for all other learning; they include: reading text, document use, writing, numeracy, oral communication, computer use, thinking skills, working with others, and continuous learning
  • Essential skills proficiency is strongly linked to positive labour outcomes (employability and earnings potential) (TD Economics Special Report Dec. 2013)