Photo of a group of women smiling and holding water bottles

Evaluating progress is essential for accountability and for making adjustments along the way. This involves monitoring activities and accomplishments, assessing the impact and merit, and using the information to improve the effort.

Key Questions to Consider

  • What is the framework (logic model, or pathway) by which activities (e.g., policy and program changes) in different sectors will lead to improvement in desired outcomes?
  • What questions do stakeholders hope to answer with the evaluation?
  • What methods will be used to answer the evaluation questions? How will monitoring and evaluation be used to see how the effort is being implemented and whether it is achieving intended effects?
  • How will stakeholders from participating groups and sectors make sense of the data? How will those responsible and affected be involved in reviewing the data and contributing to dialogue about what it means?
  • How will the initiative use the evaluation information for celebration, accountability, and improvement?
  • How will the initiative use an equity lens to see whether the effects of the effort benefit all people, including socially excluded groups that have had worse outcomes?

 

Some Recommended Actions

  1. Describe the program or initiative’s framework or logic model (i.e., pathway by which activities will lead to intended outcomes). Include information about:
    • Purpose or mission (i.e., the problem or goal to which the effort or initiative is addressed).
    • Context or conditions (i.e., the situation in which the effort will take place; factors that may affect outcomes)
    • Inputs: Resources and barriers (i.e., Resources may include time, talent, equipment, information, money, etc.; Barriers may include history of conflict, environmental factors, economic conditions, etc.)
    • Activities or interventions (i.e., what the multisectoral action will do to effect change and improvement) (e.g., provide information and enhance skills; enhance services and support; modify access, barriers, and opportunities; change the consequences; modify policies and broader systems)
    • Outputs (i.e., direct evidence of having performed the activities) (e.g., targeted actions to those most affected; number of services provided)
    • Intended effects or outcomes
      • Short-term (e.g., community and systems changes)
      • Intermediate (e.g., changes in behavior)
      • Longer-term (e.g., improvement in outcomes; reduced inequities)
  2. Focus the evaluation questions and methods. Include:
    • Evaluation questions – What information about the effort is important to stakeholders? (e.g., What changes in communities and systems were brought about in different sectors? How much/what kind of difference has the initiative made in the community as a whole? For those groups living in vulnerable situations?)
    • Methods – What methods will be used to evaluate the effects of the program or initiative? (e.g., Behavioral surveys; Interviews with key participants; Archival records; Observations; Monitoring and evaluation system)
    • Indicators of success – What indicators will be used to judge the success of the program or initiative? Match the indicators to the evaluation questions. Potential indicators include:
      • Program outputs
      • Participation rates
      • Levels of satisfaction
      • Changes in behavior
      • Community or system changes
      • Improvements in indicators of outcomes and inequities
  3. Outline and implement the evaluation plan. Indicate:
    • How you will involve all stakeholders from participating sectors and groups in identifying indicators of success, documenting evidence of progress, and sense making about the overall initiative and how it can be improved.
    • How you will track implementation of the intervention.
    • How you will assess ongoing changes in specific objectives and outcomes (e.g., indicators; inequities).
    • Ethical implications of the initiative.
  4. Make sense of the data and justify conclusions. This includes engaging stakeholders (including those most affected) in:
    • Sensemaking and interpretation – How will we engage those responsible, and those most affected, in making sense of the data (i.e., what participants are seeing, what it means)? How will we use the information to help answer the evaluation questions?
    • Judgments – statements of worth or merit of the initiative. How will we communicate what the findings suggest about the value added by the effort?
    • Recommendations – How will we identify recommendations based on the results of the evaluation?
  5. Use the information from the evaluation, including to:
    • Celebrate accomplishments
    • Make adjustments in activities and interventions
    • Communicate lessons learned to stakeholders, the community, and other relevant audiences

 

Examples of Evaluation

Evaluating and Celebrating Sustainable Transportation Efforts

Icon image of Argentina flagBUENOS AIRES, ARGENTINA

 

Buenos Aires, the capital and largest city in Argentina, worked to make its urban center more inclusive and sustainable. The city implemented innovative approaches to urban planning and advances to its public transportation system.

 

For example, as part of a citywide Sustainable Mobility Plan initiated in 2009, the city gave its 9 de Julio avenue, the widest avenue in the world, an impressive transit makeover. The avenue’s more than 20 lanes of traffic were expanded to include 11 bus lanes that improved travel for 200,000 passengers per day. The city also worked to transform the city center into an environment that encouraged walking and cycling over driving, and promoted a culture that prioritizes the health of its citizens.

 

Evaluations showed the success of these transportation efforts: passengers reduced their travel time by an average of 30 minutes per bus ride, and the commute across the city was reduced from 40 minutes to 14.

 

Sources:

    1. Buenos Aires, Argentina Wins 2014 Sustainable Transport Award. Available from: https://www.itdp.org/buenos-aires-argentina-wins-2014-sustainable-transport-award/
    2. 2014 Sustainable Transport Award: Buenos Aires, Argentina. Available from: http://staward.org/winners/2014-buenos-aires-argentina/

Some Resources to Help You Evaluate

Tools from the Community Tool Box

 

Resources from the World Health Organization/Pan American Health Organization