Click here for search results

Evaluation Tools

The findings of this evaluation rest on the following five sets of evidence. Click on an item for more details. For more information, download Appendix B. 

  1. Reviews of the effects of ESW and TA and the factors affecting these effects in 12 countries
  2. Electronic surveys of all Bank TTLs on specific ESW and TA
  3. An electronic survey of Bank loan TTLs
  4. Electronic surveys of in-country stakeholders
  5. Statistical analysis


1. Reviews of the effects of ESW and TA and the factors affecting these effects in 12 countries. The reviews entailed desk reviews and structured interviews (including in the field) with in-country stakeholders, Bank staff, and management working on those countries. The 12 countries are Bangladesh, the Democratic Republic of Congo, Guyana, Jordan, Lesotho, Malaysia, Mali, Mauritius, Peru, Romania, Serbia, and Vietnam. The countries were selected to be broadly representative of Bank clients with respect to Region, income, population, Country Policy and Institutional Assessment, size of lending (per capita), and share of ESW cost in the cost of country services. The country reviews sought the views (with quantitative ratings) of the stakeholders on various dimensions of specific ESW and TA tasks (quality, partnership, origination, dissemination, timeliness, and so on) and the extent to which these tasks met the various ESW and TA objectives.

A total of 360 in-country stakeholders were interviewed for the country reviews, of which 191 were public officials, 9 were members of legislatures, 63 were donor representatives, 39 were civil society representatives, 33 were from the private sector, and 18 were from academia or research institutions. A total of 129 ESW, 64 TA, and 88 loans were reviewed. The sectoral distributions of the ESW and TA reviewed were in line with the sectoral distributions of all such tasks delivered Bank-wide during fiscal 2000-06.


[return to top]

2. Electronic surveys of all Bank TTLs on specific ESW and TA they had led during fiscal 2000-06. The surveys sought the views of TTLs on various dimensions of ESW and TA and their perceptions (with quantitative ratings) on the extent to which the various ESW and TA objectives were met.

The net addressable sample (see Appendix B for definition) of TTLs was 1,143 for ESW and 419 for TA; of these, 696 ESW TTLs and 220 TA TTLs responded, representing response rates of 61 and 53 percent, respectively. Bias checks revealed no significant biases with respect to the respondents: specifically, those TTLs who responded did not generally have higher QAG ratings on their tasks than those who did not respond.


[return to top]

3. An electronic survey of Bank loan TTLs on their views on (and quantitative ratings for) specific ESW delivered during fiscal 2000-06 that may have informed their loans. Those ESW TTLs who were also loan TTLs were excluded from the survey.

There were 91 responses out of a net addressable sample of 186 loan TTLs, a 49 percent response rate. Bias checks indicated no Regional or sectoral biases in the responses.


[return to top]

4. Electronic surveys of in-country stakeholders for the purpose of validating findings from the country reviews; these are a survey on specific ESW tasks, a survey on specific TA tasks, and a survey on general views on ESW and TA tasks.

For the survey on specific ESW tasks, of a net addressable sample of 297, 70 responded on ESW for 41 countries (response rate of 24 percent). For the survey on specific TA tasks, of a net addressable sample of 144, 33 responded on TA for 17 countries (response rate of 23 percent). For the general ESW/TA survey, out of a net addressable sample of 2,418, 550 responded from 114 countries (response rate of 23 percent). Bias checks did not reveal significant biases in the sectoral or Regional distribution of respondents with respect to those who were sent the survey or with respect to the sectoral or Regional distribution of all Bank ESW and TA.


[return to top]

5. Statistical analysis of associations between different aspects of ESW and TA (numbers, average costs, quality, types, partnerships, origination, and so on) and indicators of achievement of objectives (loan quality, government policies, institutions, and so on). This component of the work was based on Bank administrative data, data collected from surveys, and other pertinent data on country conditions.

Only those findings that are supported by all five sources of information are reported, unless otherwise noted. All the quantitative ratings provided in the country reviews and all the surveys are on a scale of 1 - 6, with 1 being low and 6 being high, 4 being slightly above average, and 3 being slightly below average. The evaluation also considered the findings of other IEG evaluations where applicable.

Appendix D provides the evaluation's economic Analysis  of ESW and TA.

[return to top]




Permanent URL for this page: http://go.worldbank.org/ZKJDF265L0