The majority of ESW and TA met their objectives at least to an average extent during fiscal 2000-06. Between 65 and 80 percent of users of ESW and TA in client countries and between 74 and 87 percent of users within the Bank gave ratings of average and above when asked to what extent these products met their stated objectives. But there were substantial differences in ratings across countries and tasks.
The Extent to Which ESW and TA Met Their Stated Objectives
Most ESW and TA met their stated objectives to at least an average extent, although their effectiveness was greater in shaping Bank lending and strategy than in providing support directly to client countries. The indirect effects of ESW and TA on client countries- through Bank lending-were greater than the direct effects. Between 65 and 80 percent of users of Bank ESW and TA in client countries gave ratings of average and above on the extent to which ESW and TA met their stated objectives; between 74 and 87 percent of such users in the Bank (task team leaders for loans and strategies) gave such ratings. ESW had the largest effects on informing Bank strategy, which is not a stated objective.
The presence of relevant ESW was statistically associated with better loan design. These results corroborate the relatively high ratings, on average, from loan task team leaders on the extent to which ESW informed lending. The evaluation period was not long enough for ascertaining the effect of ESW on loan outcomes.
ESW in a range of sectors and report types, including tasks initiated by the Bank, had positive effects both within the Bank and in client countries. Country Economic Memoranda and Public Expenditure Reviews (PERs) were especially useful for informing Bank strategies and development policy loans. Within client countries, PERs, Financial Sector Assessment Programs, and Investment Climate Assessments were among the more prominently cited ESW products that had, variously, informed government policy, built capacity, stimulated public debate, and influenced the development community. Box ES.1 provides examples of particularly influential ESW and TA.¬†
ESW and TA had effects beyond the stated objectives in some cases. The more prominent ones included effects on countries other than those for which they were intended, private investment decisions by domestic and international investors, support for local research, and the creation of informal networks between Bank staff and policy makers.
Stakeholders in most of the countries reviewed indicated that ESW and TA had made a difference to the reforms in their countries. This view was expressed in countries with a wide range of income, government capacity, and policy orientation. In some countries, stakeholders indicated that the reforms in their countries would not have happened at all without Bank ESW and TA. Others indicated that without Bank ESW and TA, the reforms would have, variously, been of lower quality, taken longer, and been more piecemeal and less focused. Some indicated that Bank ESW and TA had helped them avoid or reduce the cost of policy mistakes.
Bank ESW and TA also provided a level of comfort or confidence for some countries to move forward. A few pointed to the credibility that Bank ESW and TA provided. The characteristics of Bank ESW and TA that clients valued were high technical quality, objectivity and neutrality, and provision of international perspectives. In 2 of the 12 countries reviewed, however, a majority of the incountry stakeholders indicated that Bank ESW and TA made no difference to the reforms in their countries.
The effectiveness of ESW and TA ranged from substantially above average to substantially below average across countries and tasks. The wide range of effectiveness is in part caused by country-specific factors, such as government capacity and government receptivity, and in part by factors that are more amenable to being addressed by the Bank.
Factors in ESW and TA Effectiveness
When government capacity was lower, ESW and TA were less effective. ESW and TA were less effective in postconflict and otherwise lowcapacity countries. Policy makers in such countries had limited capacity to absorb all the ESW produced. High turnover of government officials also negatively affected capacity and, in turn, the effectiveness of ESW and TA in some countries.
Government receptivity also influenced the effectiveness of ESW. There were instances where political constraints influenced government receptivity to certain ESW products, either rendering such ESW less effective or delaying its effectiveness until the political situation changed.
ESW and TA products of lower technical quality were less effective. In some instances the Bank did not have staff and consultants with the requisite skills for particular ESW and TA tasks, which undermined the Bank's credibility.
ESW products of higher quality cost more. Furthermore, the average cost of ESW tasks was lower in IDA than in IBRD countries. The Bank budget but not the trust fund component of cost was associated with the quality of ESW.
Close collaboration with clients, from the initiation of the task through the formulation of recommendations, was important for ESW and TA to be influential. This may or may not have entailed a client actually producing part of the task, such as writing sections of a report.
Where partnership with clients made a difference, a variety of approaches was equally effective, depending on country conditions. In countries with more limited government capacity, partnering in the production of narrowly focused tasks was more effective than partnering in broadly scoped tasks.
Whether the client specifically requested the ESW and TA did not matter for effectiveness, although client buy-in was still important. In-country stakeholders in both low- and middle-income countries indicated that the Bank has a useful role in proposing topics for ESW and TA, drawing on Bank knowledge and international experience. Nevertheless, the Bank needs to ensure that there is genuine client interest or needs to engender such interest. Tailoring tasks such as core diagnostics to clients' needs and making efforts to collaborate from the conception stage were both effective in engendering client interest.
Sustained follow-up beyond one-off dissemination was important for ESW and TA effectiveness. The ESW and TA with the greatest effects were those for which there was sustained follow-up on completion. The follow-up could be in the form of lending or nonlending activities, with the latter encompassing formal or informal TA.
The scope of dissemination for effective ESW and TA varied. In some cases targeted dissemination to the relevant counterparts resulted in important changes; in other cases wide dissemination stimulated the public debate that spurred changes. How wide dissemination needed to be for ESW and TA to be effective also varied within the same country for different topics.
Translation is important to client countries. Stakeholders in a majority of the countries reviewed for this evaluation indicated that better and more rapid translation is needed. Having Bank staff or consultants that speak the local language clearly enhanced the effectiveness of these tasks.
What Clients Want from the Bank
Stakeholders in IBRD countries indicate a strong preference for the Bank's nonlending services over its lending services. The preference is less strong in IDA countries. TA is preferred to ESW in both groups of countries. MICs value the Bank's advice, and some of them borrow from the Bank to obtain that advice. Stakeholders in some MICs expressed a strong view that Bank ESW should be delinked from its lending operations. IDA countries need the Bank's funds but also value the Bank's advice. TA is preferred to ESW in countries with high government capacity as well as in those with lower capacity.
Stakeholders found the Bank's reports and TA more useful than those provided by other institutions. This finding was stronger for TA than for ESW.
The Bank's clients have a strong desire for greater Bank presence in the country. Having ESW and TA undertaken from country offices facilitates the close collaboration on these tasks that is important for them to be effective. Followup is also best sustained from country offices, given the long-term close interactions typically entailed. Related to this is the need to further devolve decision-making powers to the country offices, as they are front-line recipients of government requests, especially for rapid-response TA.