Thursday, July 28, 2005
12:30 - 2 pm
Room: MC-9 W150
Presenter: Mike Bamberger (Consultant)
The current debate on randomized evaluation designs has diverted attention from what, for most Bank staff is the more pressing question of when and how methodologically defensible estimates of project effects and impacts can be obtained when operating under real-world budget, time and data constraints. The Brown Bag Lunch (BBL) discussed two common scenarios. The first is when the evaluation begins at the start of the project but where there are budget, time, data, logistical and political constraints on the type of evaluation design that can be used. The second, and probably more frequent scenario is when the evaluation does not start until late in the project cycle or when the project is already completed. Under this second scenario there is normally no baseline study and the evaluator will often also have time and budget constraints. A set of design options was described for addressing the different constraints and a checklist was presented for assessing threats to validity and adequacy of the different design options. Guidelines, including the use of mixed-method approaches was also presented for strengthening all of the design options.
Lessons and challenges for the Bank were discussed, including how to decide when a methodologically acceptable impact can, and cannot be conducted. The BBL ended by introducing a proposed OED/Poverty Impact Analysis, Monitoring and Evaluation TG initiative to prepare a "how to" booklet (similar to "Influential Evaluations") providing guidelines, illustrated with brief case studies, on how to conduct impact evaluations under real-world constraints.
Mike Bamberger: Since retiring from the Bank in 2001 Michael Bamberger has been applying real-world evaluation approaches in consulting assignments and workshops with UNDP, UN-DESA, USAID, DFID, the Foundation for Advanced Studies in International Development and the Bank. A book is scheduled for publication in January 2006.
Back to Training Events and Materials