Using a highly participatory Theory of Change approach for on-going monitoring and learning
Vision for a new approach to learning, monitoring and evaluation
In Summer 2017 the Avina Foundation in collaboration with other donors launched a program to improve conditions of migrants through a specific focus on their access to dignified, just job opportunities. The program is being implemented in six countries: Brazil, Chile, Guatemala, Costa Rica, El Salvador and Mexico, with strong connections to a U.S. sister program. Program designers understood that a traditional approach to M&E wouldn't serve the needs of a program being implemented in highly diverse contexts, with partner organizations pursuing a spectrum of advocacy, organizing and humanitarian strategies.
Overview
The program is currently working with 20 grantees, in the six different countries. The grantees are either groups of faith, such as the Jesuit Migrant Service, or migrant-led groups such as the Movimiento de Acción Migrante (MAM).
The LME strategy aims to encourage and facilitate learning and three different levels. 1)The partner organizations themselves, 2)amongst the partner organizations and at 3)the donor level, in order to evolve and adapt the program based on realities on the ground.
The strategy is based on yearly cycles, with both grantees and donors, which begin with a highly participatory Theory of Change/collective learning workshop, followed by periodic opportunities for learning, ending the yearly cycle with Monitoring and Reflection workshops.
The framework for data collection and monitoring is established by the grantees themselves; they articulate their visions, and the expected longer and shorter term outcomes, with specific focus on the upcoming year.
Data is collected quarterly through focus groups conducted by program staff. Then, yearly monitoring and reflection workshops provide an opportunity for partners, program staff and donors to reflect and assess progress, and based on emerging learning, plan for the upcoming year.
INITIAL THEORY OF CHANGE WORKSHOPS
Workshops were held with each partner grantee, with the Avina staff & other donors. When possible, the workshops were collective, or had a block of time for collective work. Workshops were generally one day, with the collective workshops going from 1-3 days. The different components are below, and sample agendas can be found here.
VISIONING
Partners, program staff and donors imagine the changes and transformations ten years from now, in relationships, laws/policies, access to resources, social norms and people's understanding, skills and sense of agency.
AVINA STAFF/DONORS VISIONING
Visioning- Theory of Change, Santiago Chile, Summer 2017
PARTNERS' VISION FOR THE FUTURE
Partners discussion their visions for the future
ANALYSIS ACTIVITIES
Some workshops were focused on the work of individual partner organizations, while others were collective, and allowed for a rich exchange of ideas, knowledge and feedback as well as strengthening opportunities for collaborative work.
Fish and boulders - Participants analyze the assets and forces (sh) and the boulders (obstacles) as part of the path to reaching their visions. In some cases, we had each group do their individual analysis, and then shared with other the other groups.
Opportunities and challenges in the context - Working in pairs, then small groups, participants identied which factors they needed to take into considering in building their strategies. Then, they voted, using dots, on those that were most critical.
Power mapping - There are lots of ways to do this, one is do use different size circles to represent the importance of actors, based on who the organization or alliance needed to inuence.
Workshops in Costa Rica, Brazil, Guatemala and Chile
MAPPING
Participants discussed migrant flows and types, presence and activities of organizations and locations of refuges.
ANALYSIS - UNDERSTANDING ROLES
In collective workshops, we had organizations share where they saw others strengths and assets in the movement.
Participants were asked to identify the strengths/roles of ally organizations and then discussed how the different strengths were complementary anyhow they might be leveraged.
MAPPING OUTCOMES
A few key points...
The traditional "backwards mapping" we've found to be complicated and burdensome, so often have people working in different configurations (see 1-2-3-4 all and other Liberating Structures) to identify strategic priorities.
We ask people to make sure to draw from the prior work analyzes their context and organizations. It is helpful to keep the artifacts from the activities (drawings, maps, etc.) in clear view.
Often it is more comfortable for groups to talk about the external changes they want to achieve, ignoring the internal issues (capacities, staff issues, etc.) that need to be addressed. It's also important for them to think specically about changes that they need to contribute to in collaboration with other actors. Have participants use different colors for external changes, internal changes, and collaborative changes they hope to contribute to. We put most of the focus on the upcoming year. In advocacy and structural change work, there needs to be regular opportunities for reflecting and adapting plans based on changing circumstances and the environment.
UNDERSTANDING OUTCOMES FOR SOCIAL CHANGE & MOVEMENT BUILDING
We nd that participants have been taught to shy away from discussing changes difcult to measure, or they feel somehow that qualitative changes are not legitimate! (When actually they are critical). It's important to take time to discuss these qualitative changes, outcomes related to relationships and connections, to understanding, etc. There are different frameworks that can help with this - see here. Discussion around power are incredibly important - and Just Associates has many tools and resources to support engaging participants around what it means to shift power.
DOCUMENTING THEORY OF CHANGE WORKSHOPS
Post workshop, visions and short and long term outcomes are documented visually with an accompanying narrative. These documented outcomes are then used as the basis for the monitoring and evaluation workshops one year later.
LEARNING AT THE PROGRAM LEVEL
It is often a challenge for program ndings and insights to make their way into the planning and development process. For example, program staff have extremely limited time. Also, donor funder requirements are such that the program strategies and criteria are expected to adhere to the original logic model and guidelines; this makes adaptation challenging!
We have piloted and developed a number of strategies which are promising:
Establishment of a set of learning questions important for the program's development. Regular individual phone calls with program staff exploring learning questions. Documentation and summarizing of findings. Periodic program "learning calls" to explore themes, and discuss implications for the program. In-person reflection and planning meetings, held every 6-12 months, to explore program progress, and changes and adaptions that need to be made. (The in-person meetings are critical)
MONITORING AND REFLECTION WORKSHOPS!
At the end of the first year, workshops, conducted both with grantee partners and Avina staff and donors, serve as key opportunities to reflect and provide evidence for progress, discuss challenges and learning, and plan for the future.
PART 1- MONITORING & REFLECTION WORKSHOP
KEY COMPONENTS:
The Theory of Change created by the partner is placed on the floor (Short-term to longer-term outcomes).
Participants break into smaller groups, and discuss where they have seen progress, based on what they established last year. They also discuss new, unexpected outcomes (which they put on the floor in a different color).
For each outcome where they have seen progress, they discuss the evidence (what they have seen, heard, observed) and the key factors that are related to progress.
Then, in two groups they vote on progress (between 0-3), in two groups, and then discuss the ratings, and either come to an agreement, or if not, we average the numbers). They also place dots on those most important for them to focus on for the future.
Lastly, participants discuss priorities for the upcoming year (the large green circles). (With some organizations it is helpful to have the participants work individually, and create post-its so that anonymously priorities can be organized by themes)
FEEDBACK FOR THE BACKBONE ORGANIZATION
As part of the program learning process, we asked groups to rate and discuss the value of the capacity building activities and accompaniment role played by Avina, including hopes and recommendations for the future.
PART 2 - MONITORING & REFLECTION WORKSHOP
Sharing learnings and challenges between groups and using the collective intelligence strengthen strategies.
KEY COMPONENTS:
Individually groups discussed their key learnings and challenges Learnings and challenges were then shared between the groups.
Using an activity called "Caravan" we divided into groups working on two main themes, such as "development of membership" and then people rotating asking questions and giving suggestions until a set of recommendations and action items were established.
PROGRAM STAFF MONITORING AND REFLECTION WORKSHOP
The Labor Migration program staff also did a workshop, reflecting on the previous year, identifying where they had seen progress, and establishing priorities for the upcoming year.
UNDERSTANDING DATA ACROSS THE PROGRAM
We are using Impact Mapper a platform built to track, evaluate and visualize social impact. We did an early pilot of this rst monitoring and reection workshop with program staff, and are continuing to use the platform. Data collection consists of:
Quarterly feedback from program around program changes (from focus groups conducted by program managers).
Monitoring and Reflection workshops.
Data is collected automatically and the platform facilities coding around advocacy and movement building indicators. We are then able to explore trends and changes across the program.
REVISION OF THE THEORY OF CHANGE
Thinking about priorities and establishing plans for the future.
Program staff went through a number of different collective analysis and learning exercises to help them consider underlying values that need to drive decision-making, their own sense of what they were proudest of in the program, opportunities and priorities for the future, and specific action items on the role of Avina as a back-bone organization, accompanying partners and creating opportunities for building alliances and movement.
REVISED PROGRAM THEORY OF CHANGE
REPORTING
As mentioned above, we are using Impact Mapper to support with analysis across the program. Here are two examples of charts generated based on analysis of data to date.
Comments