Click here for search results

Impact Evaluation

The following studies are evaluations of policies and interventions to assess whether human development programs are achieving their objectives.

The impact evaluation studies at the World Bank, particularly in areas related to human development, have in many cases been initiated by the Development Research Group. These studies apply a variety of evaluation strategies depending on the nature of the policy or program being evaluated and the stage when the researchers got involved.

While designing and implementing an evaluation before a policy or program begins make the evaluation a little easier, this is not often possible. The reality is that governments usually implement policy initiatives on a large scale straight from concept or introduce them in areas where they are more  likely to succeed, not where they can be tested fairly. To evaluate the most important of these initiatives requires a creative but rigorous research design and appropriate data. 

Articles & Briefs
new template - orange triangular bullet  Cushioning the Effects of Health Shocks on Households  
(
Brief 2006)
new template - orange triangular bulletOrphanhood, Poverty, & School Enrollment  
(
Digest Article 2006)
new template - orange triangular bullet  Fungibility and the "Flypaper Effect" of Aid  
(
Digest Article 2007)
new template - orange triangular bullet 

Evaluating Human Development Projects: What Works, What Doesn’t, Why?(Report 2008)  

new template - orange triangular bullet

More articles and briefs

 

Contents Researchers
 Related Web Sites 

 

 

 


 

Education

Health & nutrition & poverty


Conditional cash transfers for education and health (Ecuador)
Contact: Norbert Schady

The study will examine experimental and quasi-experimental evidence on the impact of conditional cash transfer programs. Ecuador is in the process of reforming an unconditional, poorly targeted cash transfer program (the Bono Solidario) into a better-targeted, conditional cash transfer program (the Bono de Desarrollo Humano). The conditions which households must fulfill are in education (if they have children aged 6 to 15), or in health (if they have no school-aged children, but have children aged 0 to 5). This program is by far the largest of discretional social expenditures, accounting for 0.75% of GDP in 2002. The evaluation of the education component is based on a combination of randomization and a regression-discontinuity design, while the evaluation of the health component is based entirely on randomization of districts into treatment and control groups. Baseline household and facility surveys in the education component are complete. Follow-up surveys should be completed by October 2004; the baseline household and facility surveys in health are currently in the field, with follow-up surveys expected in December 2004.

Findings

  • H. Oosterbeek, J. Ponce, and N. Schady. 2008. "The impact of cash transfers on school enrollment : evidence from Ecuador." World Bank Policy Research Working Paper  4645  (June).
  • C. Paxson, and N. Schady. 2007. "Does money matter ? The effects of cash transfers on child health and development in rural Ecuador." World Bank Policy Research Working Paper  4226, (May).
  • A. Maria Caridad, and N. Schady. 2006. "Cash Transfers, Conditions, School Enrollment, and Child Work : Evidence from a Randomized Experiment in Ecuador." World Bank Policy Research Working Paper  3930.
  • C. Paxson, and N. Schady. 2005. "Cognitive Development among Young Children in Ecuador: The Roles of Wealth, Health and Parenting." World Bank Policy Research Working Paper  3605.

Early childhood education (Philippines)
Contact: Elizabeth M. King

This evaluates a Bank-funded ECD pilot initiative of the government in three regions. The evaluation collected longitudinal data over four years on a cohort of 6,693 children age 0-4 years at baseline in two treatment regions and a control region that did not receive the intervention. The program includes a wide range of health, nutrition, early education, and social services programs.

  • Graeme, Armecin, Jere Behrman, Paulita Duazo, Sharon Ghuman, Socorro Gultiano, Elizabeth M. King, and Nanette Lee. 2006. "Early Childhood Development through an Integrated Program: Evidence from the Philippines. " World Bank Policy Research Working Paper  3922.

Education interventions (Cambodia and Indonesia)
Contacts:  Deon P. Filmer,
 Norbert Schady

This work uses retrospective and prospective impact evaluations to assess the potential for scholarships to promote education enrollment, grade attainment, and learning achievement. In Cambodia the work covers two programs. First, a retrospective evaluation of a past program that uses institutional data as well as purposefully collected data on outcomes for scholarship applicants. Second, a prospective evaluation of a World Bank supported scholarship program. The evaluation uses the design of the program to identify impact. In Indonesia the work consists of an ex-post evaluation of a large scholarship program instituted in response to the financial crisis of the late 1990s. Variation in program design allows the identification of program impacts.

Findings

  • D. Filmer and N. Schady. 2006. "Getting Girls into School: Evidence from a Scholarship Program in Cambodia." World Bank Policy Research Working Paper 3910.

Grassroot information campaigns and education outcomes (India)
Contact: Stuti Khemani

Measures the effectiveness of community campaigns to increase awareness through information, advocacy and public action campaigns.

Public-private partnerships for education (Punjab, Pakistan)
Contact:
 Jishnu Das  

This study is a long-term evaluation on the potential of public-private partnerships in the educational sector, initiated in 2003. This project is integral to the creation of a regulatory framework for education in Pakistan, as well as the Bank's Educational Sector Strategy in the country. This project has progressed exceptionally well with the first phase of data collection over and analytical work underway. In the second phase, randomized interventions have been put in place and an interim and final round of data collection will be completed by November 2005. There has been very strong regional support for this work. In addition, the response from the Punjab government has been extremely encouraging and close coordination between the team and the education department in Punjab is anticipated for a report on basic education in Punjab, due for completion by the end of 2004.

  • T. Andrabi, J. Das, A. Ijaz Khwaja, T. Vishwanath, and T.Zajonc. 2008. "  Learning and Educational Achievement in Punjab Schools." World Bank, Washington, DC.
  • T. Andrabi, J. Das, A. Ijaz Khwaja, T. Zajonc. 2005. "Religious School Enrollment in Pakistan: A Look at the Data." World Bank Policy Research Working Paper 3521.


Health & nutrition & poverty

Community-based nutrition programs (Burkina Faso,  Madagascar, and Senegal)
Contact:
 Harold Alderman 

The ongoing research seeks to assess the impact of investments in nutrition using a model of community-based interventions on the nutritional status of young children. The research is part of a larger research agenda that compares community-based interventions in Madagascar (Emanuela Galasso) and Senegal (Harold Alderman). The evaluation in Madagascar has two components. The first one is to rely on the extensive project monitoring data with process and outcome indicators: in a context of  a program that puts emphasis on behavioral change (rather than income growth), the aim is to assess whether the length of exposure to the program across communities has a differential impact on nutritional outcomes by improving the knowledge of nutritional practices.  The second component will assess the impact of the program using a baseline and a follow-up household anthropometric survey collected in both treated and non-treated communities. The objective is to estimate the impact of the program on nutritional status across different socio-economic groups. Moreover, we will explore how the results can be explained by different modes of service delivery (such as food supplementation by the WFP in selected areas, or key aspects of the NGO providers, such as their human resources and organizational structure).

Health insurance programs (China)
Contact:
 Adam Wagstaff  

The study examines the impacts of various health insurance programs in China on service utilization, nonmedical consumption and health outcomes. The study will also assess the extent to which uninsured households are successfully able to smooth their consumption, how the extent of self-insurance varies across rural and urban areas, the poor and nonpoor, and over time.

Findings

  • A. Wagstaff and M. Lindelow. 2005. "Health shocks in China: are the poor and uninsured less protected?" World Bank Policy Research Working Paper 3740.
  • A. Wagstaff and M. Lindelow. 2005. "Can insurance increase financial risk ? The curious case of health insurance in China." World Bank Policy Research Working Paper 3741. 
  • A. Wagstaff and S. Yu. 2005. "Do health sector reforms have their intended impacts? The World Bank's Health VIII project in Gansu province, China." World Bank Policy Research Working Paper 3743.
  • A. Wagstaff. 2007. " Extending health insurance to the rural population : an impact evaluation of China's new cooperative medical scheme." World Bank Policy Research Working Paper 4150.

Health insurance programs (Vietnam)
Contact: Adam Wagstaff

This study combines propensity score matching and double differencing on pre-intervention and post-intervention data to estimate the impacts of Vietnam’s health insurance (VHI) program on a variety of outcomes. Health service use focuses on the probability of any provider contact among the sick, the probability of contact with specific provider types including those whose services are not covered by VHI (private providers and traditional healers), and those whose services are covered (public hospitals and commune health centers). Also studied are out-of-pocket payments and consumption of budget items other than medical care (the possibility that insurance reduces precautionary saving and encourages consumption). The estimates are used to predict VHI’s impact on income poverty. Also studied are VHI’s impact on health outcomes using anthropometric measures of malnutrition.

Findings

  • A. Wagstaff. 2007. “The economic consequences of health shocks: Evidence from Vietnam.” Journal of Health Economics 26: 82-100.
  •   A. Wagstaff, and M. Pradhan. 2005. "Health Insurance Impacts on Health and Non-medical Consumption in a Developing Country (Vietnam)."  Policy Research Working Paper 3563, World Bank, Washington, DC.
  • A. Wagstaff. 2007. "Social health insurance re-examined." World Bank Policy Research Working Paper  4111.
  • A. Wagstaff. 2007. "Health insurance for the poor : initial impacts of Vietnam's health care fund for the poor." World Bank Policy Research Working Paper 4134.

HIV/AIDS interventions (Multicountry)
Contact:  Damien de Walque

Several proposals have been developed to study the economics of AIDS and evaluate the impact of AIDS prevention and treatment service delivery (including impact on orphans and vulnerable children). In Burkina Faso, Ghana, Kenya, Mozambique and South Africa we will evaluate the impact of anti-retroviral delivery, focusing on the socioeconomic benefits for the patients and their families and on the impact on the dynamics of HIV/AIDS transmission. In Burkina Faso, we are also evaluating a social protection mechanism for families affected by the HIV/AIDS epidemic. In Tanzania, we are planning to study the impact of education on HIV/AIDS prevention and in Uganda we are planning to study the impact of HIV/AIDS specific service delivery in the health sector.

More on HIV/AIDS research

Nutrition programs (Indonesia)
Contact:  Jed Friedman 

We provide new evidence on the effect of iron deficiency on economic and social prosperity of older adults drawing on data from a random assignment treatment-control design intervention. The Work and Iron Status Evaluation is an on-going study following over 17,000 individuals in Central Java, Indonesia. Half the respondents receive a treatment of 120 mg of iron every week for a year; the controls receive a placebo. Compliance is monitored carefully. Results from the first six months of the intervention show that for adults age 30 through 70 years, males who were iron deficient prior to the intervention and who are assigned to the treatment are better off in terms of physical health, psycho-social health and economic success (increased productivity). Benefits for women are in the same direction but the effects are more muted. The results provide unambiguous evidence in support of the hypothesis that health has a causal effect on economic prosperity of males during middle and older ages.

Findings

Rural Road Rehabilitation Project - Vietnam
Contact:  Dominique van de Walle

This study evaluates the impact of the World Bank-financed Viet Nam rural transport project I, a large-scale rural roads rehabilitation project that aims to reduce poverty. The analysis focuses on the welfare impacts of rural roads as an input to policy discussion about how best to allocate scarce public resources for poverty reduction. The key questions are: What are the impacts of rural roads on both the mean and distribution of living standards, broadly defined to include dimensions other than income, e.g. health and education outcomes, and what factors affect those impacts? How do the benefits and their distribution change over time? How much do benefits depend on other investments such as reflected in human capital endowments?

Findings

  • van de Walle, Dominique, and Ren Mu. 2007.  “Fungibility and the Flypaper Effect of Project Aid: Micro-evidence for Vietnam.” Journal of Development Economics 84(2): 667-684.
  • Mu, Ren and Dominique van de Walle. 2007.  “Rural Roads and Poor Area Development in Vietnam,” World Bank Policy Research Working Paper 4340.
  • Mu, Ren, Dominique Van De Walle. 2007. "Fungibility and the Flypaper Effect of Project Aid: Micro-evidence for Vietnam." World Bank Policy Research Working Paper 4133.
  • van de Walle, D., and D. Cratty. 2005. "Do Donors Get What They Paid For? Micro Evidence on the Fungibility of Development Project Aid." World Bank Policy Research Working Paper 3542
  • van de Walle, D. 2005. "Choosing Rural Road Investments to Help Reduce Poverty." World Bank Policy Research Working Paper 2458

 Completed


Child Nutrition Supplementation Program ( India)
Contact: 
 Monica Das Gupta  | Michael Lokshin

This study examines the impact of nutrition intervention programs (ICDS) on child health outcomes in India, a large program for child growth, a substantial component of which is to provide nutritional supplementation to young children.  We examine how progressively the program is distributed, and whether it has had a discernible impact on child nutritional status.


Findings

  • M. Das Gupta, M. Lokshin,M. Gragnolati, O. Ivaschenko. 2005. "Improving Child Nutrition Outcomes in India: Can the Integrated Child Development Services (ICDS) Be More Effective ?" World Bank Policy Research Working Paper 3647.  
  • Das Gupta, Monica, Michele Gragnolati, Caryn Bredenkamp, Yi-Kyoung Lee, and Meera Shekar. 2006.  "ICDS and Persistent Undernutrition: Strategies to Enhance the Impact. " Economic and Political Weekly 41(12): 1193-1201.


Investments in Early Child Development on Nutrition and Cognitive Development
(Bolivia, Philippines, and Uganda)
Contact:  Harold Alderman, Elizabeth King 

Impact evaluations of Bank-funded projects aimed at improving the nutrition and cognitive development of young children. In Uganda, there was no noticeable change on cognitive development. However, nutrition improved for younger children and knowledge and practices improved as well. There was also an appreciable increase of growth in those randomized assigned communities where de-worming medicine was provided.

Multi-country

  • H. Alderman, J. Behrman, H-P. Kohler, J.n A. Maluccio, and S. Cotts Watkins. 2000. "Attrition in Longitudinal Household Survey Data - Some Tests for Three Developing-Country Samples." World Bank Policy Research Working Paper 2447

Uganda

  • Kabatereine, N. B., E. Tukahebwa, S. Brooker, H. Alderman, and A. Hall, The Epidemiology of Intestinal Helminth Infections among Schoolchildren in 18 Districts of Southern Uganda,  East Afr Med J. 2001 78(June, 6):283-86, 2001. Abstract
  • Alderman Harold, Joseph Konde-Lule, Isaac Sebuliba, Donald Bundy, and Andrew Hall.  2006. "Increased Weight Gain in Preschool Children due to Mass Albendazole Treatment Given During 'Child Health Days' in Uganda: A Cluster Randomized Controlled Trial."  British Medical Journal 333: 122-126. 2006.
  • Alderman, Harold. 2007. "Improving Nutrition through Community Growth Promotion: Longitudinal Study of the Nutrition and Early Child Development Program in Uganda."  World Development35(8):1376-89.


Nutrition Impact of Milk Subsidies (Peru)
Contact:
 Harold Alderman

  • Stifel, David, Harold Alderman. 2003. "The "Glass of Milk" Subsidy Program and Malnutrition in Peru." World Bank Policy Research Working Paper 3089
  • Stifel, David, and Harold Alderman. 2006.  "The “Glass of Milk” Subsidy Program and Malnutrition in Peru." World Bank Economic Review 20(3): 421-448.
  • Stifel, David, and Harold Alderman. 2005. "Targeting At The Margin: The “Glass Of Milk” Subsidy Program In Peru."  Journal of Development Studies 41(5):839-864.


Examples of Natural Policy Experiments


Publications

A list of publications (by year) for this unit is available here.

Policy Research Working Papers on Human Development Imact Evaluation
(Please use the free Adobe Acrobat Reader to view PDF PDF files)
The following policy research working papers are drawn from the World Bank's institutional archives. Each link opens a page with an abstract of the document and several download options.


Last updated: 2012-03-05

Library

WPS5952Do informed citizens receive more...or pay more ? the impact of radio on the government distribution of public health benefitsKeefer, Philip; Khemani, Stuti2012/01
WPS5873The measurement of educational inequality : achievement and opportunityFerreira, Francisco H. G.; Gignoux, Jeremie2011/11
WPS5629School inputs, household substitution, and test scoresDas, Jishnu; Dercon, Stefan; Habyarimana, James; Krishnan, Pramila; Muralidharan, Karthik; Sundararaman, Venkatesh2011/04
WPS5601A practical comparison of the bivariate probit and linear IV estimatorsChiburis, Richard C.; Das, Jishnu; Lokshin, Michael2011/03
WPS5522Food crisis, household welfare and HIV/AIDS treatment : evidence from Mozambiquede Walque, Damien; Kazianga, Harounan; Over, Mead; Vaillant, Julia2011/01
More...