“Amazing and targetful Workshop. I got many new ideas on the areas of Impact Evaluation design and methods, and this inspired me to do many things for my country and the world. Thanks World Bank and DIME for your support!!!” Elias Asfaw—Ethiopia
The Africa Region Health, Nutrition, and Population Units (AFTHE and AFTHW), the Development Impact Evaluation Initiative (DIME), the Human Development Network (HDN), and the Development Economics Research Group (DECRG) are organizing an innovative workshop to pilot a new approach to linking evidence with health project design in Africa. Innovating in Design: Evidence for Impact in Health will be held at the Protea Fire & Ice! Hotel in Cape Town, South Africa, from December 10–14, 2012.
The workshop will help teams to incorporate the latest evidence from rigorous impact evaluations into project design and implementation, and to leverage project design and implementation as a learning tool produce state-of-the-art evidence. The workshop has the following objectives:
Provide new evidence on mechanisms that can improve health project effectiveness. The workshop will feature a well-structured collection of evidence which can be adopted and operationalized in your project to improve its impact on the ground. Clinics will be held to link researchers and practitioners with project teams to discuss the incorporation of this evidence in each specific project context..
Assist teams in developing impact evaluations to improve project effectiveness and further expand our knowledge of health project design and implementation. The workshop will provide basic training on impact evaluation methods, including experimental and non-experimental designs. The workshop will develop preliminary concept notes (or enhance existing ones where they already exist), which will become the basis of a program of technical assistance to incorporate impact evaluation into the project..
The workshop agenda comprises four main types of session:
Panels. These sessions will be structured as a conversation between a number of presenters who will discuss the practical relevance of innovation, research, and economics in health policy and programming.
Methods. These sessions will introduce workshop participants to impact evaluation methods, and discuss implementing impact evaluations in practice.
Mechanisms. These sessions will present evidence from rigorous impact evaluations, focusing on the information, motivation, and ability mechanisms on both the demand and supply sides of health systems.
Clinics. These sessions will pair each country team with a dedicated facilitator to discuss the relevance of the material presented at the workshop in the context of each project. Teams will consider which specific mechanisms can best improve the effectiveness of their project, and may choose to begin work on an impact evaluation to test specific mechanisms or other innovations to create project-specific actionable evidence to improve project impacts. This preliminary work will then become the basis for a program of technical assistance for the implementation of the impact evaluations.
All teams will make a short presentation of their work on Thursday, December 13.
The workshop content is structured around pre-identified mechanisms that interact on demand/supply sides to produce good health outcomes.
Mechanisms are the underlying factors that facilitate or deter human behavior in markets, including healthcare markets. Evidence to be presented at the workshop is structured around three categories of mechanism — information, motivation, and ability — which are relevant to both demand and supply sides. These categories of mechanism are present at the various points of interaction between producers and consumers of health services which are required for quality health services and improved health outcomes.
Briefly, information refers to the knowledge that consumers and producers of health services have at each point of interaction. Motivation, then, is what is required to translate that information into action. Finally, ability dictates whether or not action can be carried out successfully. By identifying the constraints to information, motivation, and ability on both demand and supply sides at each point of interaction, we can begin to isolate key bottlenecks within the healthcare market. We can then target these with specific activities which, though small in relation to a project as a whole, may have big implications on our overall ability to achieve target outcomes.
For example, if we identify motivation on either the demand or the supply side of clinic visits as an impediment to achieving positive final health outcomes, at any of the points of interaction between the consumers and the producers of these services, we can begin to think about the kinds of specific mechanisms we may put in place to address this bottleneck. Incentives, financial or non-financial, can be the solution. Research shows that providing incentives to households to seek formal healthcare services (demand side incentives) or to health workers to improve their performance (supply side incentives) are very effective. Using the above framework, we can identify the specific points in the causal chain which are preventing the system as a whole from realizing its potentials, and introduce innovations to address these. These innovations can then translate into large impacts on our final outcomes of interest.
Monday, December 10, 2012
8:00 – 9:00
Participants Check-in & Coffee Break
9:00 – 9:15
Welcome Remarks Asad Alam, World Bank Country Director for South Africa (tbc)
9:15 – 9:45
Opening Panel—Innovating in Design: Evidence for Impact in Health Chair: Arianna Legovini, Head, Development Impact Evaluation Initiative, World Bank Panelists: » Olusoji Adeyi, Sector Manager, Health, Nutrition and Population, Eastern and Southern Africa, World Bank Presentationby Olusoji Adeyi: English
9:45 – 10:15
Participant expectations Participants share their expectations for the workshop and ask questions to members of the opening panel
10:15 – 10:30
Get Healthy! A game of risks and choices - all workshop participants (main room) Facilitator meeting—all workshop facilitators that have specific team assignments (breakaway room)
10:30 – 10:45
10:45 – 11:45
Keynote Address: Scaling up Innovative Interventions in Development / Small Changes, Big Results: Design Features for High ImpactMichael Kremer, Harvard University Presentation: English
11:45 – 12:30
Impact Evaluation for Real Time Decision Making Arianna Legovini, Head, Development Impact Evaluation Initiative, World Bank Presentation:English
12:30 – 13:30
13:30 – 14:30
Methods session 1: Randomization English Session:David Evans, Senior Economist, World Bank (main hall) French Session:Adrien Bouguen, Researcher, Paris School of Economics (breakout room) Presentation: English | Français
14:30 – 15:30
Impact mechanisms 1: Provider incentives Harsha Thirumurthy, Assistant Professor of Health Economics, University of North Carolina at Chapel Hill Presentation: English | Français
15:30 – 15:45
15:45 – 17:45
Project clinic 1: Project overview and innovation opportunities Each project team is assigned a facilitator to provide technical support to the team throughout the length of the workshop (and beyond) and ensure that the project-specific goals are achieved. In this session, teams will discuss their projects and how they relate to the day's presentations. They will also discuss potential innovation opportunities and/or implementation challenges. Teams will fill in the results chain for their project.
19:00 till late
Welcome Dinner and Reception (pool deck, 1st floor)
Tuesday, December 11, 2012
9:00 – 10:00
Panel: The impact of research on policy Chair: David Evans, Senior Economist, World Bank Panalists: (TBC) » Dr Cheikh Sakho, Project Coordinator, Senegal Global Fund Health Systems Project » Priscilla Reddy, Human Science Research Council (HSRC), South Africa » Kamilla Gumede, Executive Director, J-PAL Africa Presentation by Dr Cheikh Sakho: English Presentationby Priscilla Reddy: English
10:00 – 10:15
10:15 – 11:15
Methods session 2: Non-experimental methods English Session:Dan Stein, Senior Economist, World Bank French Session:Adrien Bouguen, Researcher, Paris School of Economics Presentation: Français | Case Study: English|Français
11:15 – 12:15
"Get out the vaccine" activity Moderator: Marcus Holmlund, DIME Health IE Coordinator, World Bank & Project team facilitators
12:15 – 13:15
13:15 – 14:15
Impact mechanisms 2: Financial Incentives for Consumers Jed Friedman, Senior Economist, World Bank Presentation: English
14:15 – 14:30
14:30 – 14:45
14:45 – 15:45
Impact mechanisms 3: Information and Reminders Joshua Graff Zivin, Professor of Economics, University of California at San Diego Victor Orozco, DIME Health IE Coordinator, World Bank Presentation: English | Français
15:45 – 18:00
Project clinic 2: Identification strategy Each project team will discuss how to use IE methods to carry out an impact evaluation, or how to integrate material from the "impact mechanisms" sessions into their project design
wEDNESDAY, December 12, 2012
9:00 – 10:15
Panel: Economic Returns to Investments in Health Chair:Deon Filmer, Lead Economist, World Bank Panalists: » Furio Rosati, Understanding Children’s Work (UCW)—Investing in Health is Smart Economics » Harold Alderman, Human Development Network-Social Protection, World Bank—Returns to Early Investments in Child Nutrition and Health » Joshua Graff Zivin, Professor of Economics, University of California at San Diego—The long-run economic impacts of antiretroviral therapy Presentationby Furio Rosati: English | Français Presentationby Harold Alderman: English | Français Presentationby Joshua Graff Zivin: English | Français
10:15 – 10:30
10:30 – 11:30
Impact mechanisms 4: Community Monitoring and Accountability » Deon Filmer, Lead Economist, World Bank » Edit V. Velenyi, Health Economist, World Bank Presentation: English | Français
11:30 – 12:30
Impact mechanisms 5: Supply Chain Management and The Doctor-Patient Relation Prashant Yadav, Senior Research Fellow, William David Institute at the University of Michigan Presentation: English
12:00 – 13:00
13:00 – 15:00
Speed dating Project teams have the opportunity to meet with several presenters and IE practitioners for short periods (15 minutes each) to discuss their project and/or IE design
15:00 – 15:15
15:15 – 16:00
Targeting Health Programs and Intrahousehold Allocations Harold Alderman, Human Development Network-Social Protection, World Bank Presentation: English | Français
16:00 – 18:00
Project clinic 3: Mechanisms matter Project teams will continue to discuss how the mechanisms presented could be used in their project/IE design, and will begin to prepare a short presentation of their work to be given on Thursday
tHURSDAY, December 13, 2012
9:00 – 9:45
Implementing impact evaluations in practice » Marcus Holmlund, DIME Health IE Coordinator, World Bank » Elisa Rothenbuhler, Human Development Network, World Bank Presentationby Marcus Holmlund: English | Français Presentationby Elisa Rothenbuhler: English | Français
9:45 – 10:00
Get healthy! - results of the game » Victor Orozco, DIME IE Health Coordinator, World Bank Presentation
10:00 – 10:15
10:15 – 11:15
Project clinic 4: Presentations Project teams will finalize their presentations
11:15 – 12:30
Project team presentations Teams present their planned evaluations focusing on the research questions and the IE design to obtain feedback from the rest of the group