|Management welcomes this Independent Evaluation Group (IEG) report on the World Bank’s economic and sector work (ESW) and technical assistance (TA). In the findings of the review, management sees much on which it can build, and it regards the review as providing useful inputs to the ongoing work on analytic and advisory activities (AAA). Management broadly concurs with the recommendations emerging from the evaluation.
The IEG evaluation underscores the importance of ESW and TA and provides a broad endorsement of the quality and relevance of these Bank products. Overall, the evaluation notes, the ESW reforms of fiscal 1999 were accompanied by an increase in the quality of ESW. It also notes that in-country stakeholders validated the Quality Assurance Group’s high ratings on internal quality, although stakeholders rated dissemination activities lower than technical quality.
The evaluation concludes that most ESW and TA tasks met their stated objectives and were effective in shaping lending. It also corroborates the link between the existence of relevant ESW and the quality of loan design.
From a client perspective, stakeholders in most countries indicated that ESW and TA tasks had made a difference to the reforms in their countries, although the effectiveness of these activities differed across countries. Management finds that IEG’s discussion of the several factors that influence ESW and TA effectiveness and the summary of the preferences expressed by clients regarding different Bank instruments and their delivery modes provide useful and welcome insights.
The recommendations include a call to reinvigorate the Bank’s mandate for a strong knowledge base; to focus ESW resources on fewer but betterfunded studies; to enhance the institutional arrangements for undertaking ESW and TA by ensuring substantive task team presence in the field, particularly in countries with low institutional capacity, and by formulating at the concept stage a dissemination and follow-up approach; and to build on client preferences, including feedback after completing the task. Last, the report underscores the need to strengthen the results tracking framework for ESW and TA, specifically the reporting mechanisms, and to improve its implementation. Management broadly agrees with these recommendations.
The IEG evaluation underscores the need to make ESW more responsive to clients (during planning, preparation, and feedback stages). Management considers this approach to be good practice. But there is an implicit assumption that the client is a specific entity that can express its needs and preferences at the time these activities are planned. This is not always necessarily the case: sometimes the intended client is a broader and more diffuse set of stakeholders. In some cases, too, ESW is used upstream to help inform the country’s overall process of developing a poverty strategy (poverty assessments, for example), a role the importance of which is recognized in the IEG evaluation.
Feedback and instrument choice
Management agrees with the importance of obtaining client feedback, but it must be done in a way that does not overburden the Bank’s counterparts. In preparing its review, management will analyze options for incorporating client feedback. Organizing such feedback needs to take into account the Country Assistance Strategy (CAS) context, particularly the CAS pillar(s) or themes to which they contribute. The report also mentions incentive issues regarding instrument choice. The choice between different knowledge and learning instruments and lending is a fundamental part of the CAS process and of the CAS country consultations that help address incentive biases.
Regional and global studies
The IEG evaluation, by design, focused on ESW aimed at specific countries and reviewed the relative usefulness of regional and global studies in this regard. Management notes that these latter studies do have a wider reach that fell outside the scope of the IEG evaluation. Country users can be expected to rate country ESW higher than regional or global work, because country ESW is customized to their needs. But global and regional products often deepen methodological tools and make them more accessible to teams, while providing a cross-country analytical context that can serve as a very valuable background to countryspecific ESW.
Although agreeing with the broad thrust of the findings, management would like to point out that the evaluation contains a few examples of strong statements related to a single individual or a limited sample. These statements sometimes appear to be going beyond the main points being made.
Overall, management agrees with most of IEG’s findings and accepts its recommendations. Management’s specific responses to IEG recommendations are given in the Management Action Record.
MANAGEMENT ACTION RECORD
|Recommendation||Management Response |
|Reinvigorate the mandate—which underpinned the fiscal 1999 ESW reforms—for country teams to maintain a strong knowledge base on countries and sectors where the Bank is providing or planning to provide funds. Bank country strategies and lending activities need to continue to be supported by requisite analysis, although a return to strictly defined “core diagnostic” ESW is unnecessary. ||Agreed/Ongoing. Management agrees with the recommendation of maintaining a strong knowledge base in countries where the Bank is providing or planning to provide funds. This recommendation, which is supported by the empirical association between the existence of upstream ESW and the quality of loan design, is particularly relevant as the Bank moves into thematic lending areas such as governance and climate change mitigation. In the context of the review of AAA starting this fiscal year, and particularly for International Development Association (IDA) countries, management will take stock of the existing body of ESW and outline steps to address identified issues. |
|Ensure ESW tasks in IDA countries are adequately resourced, even if it means fewer ESW in some countries. This will help to address the lower level of resources for individual ESW tasks in IDA countries than those in International Bank for Reconstruction and Development countries and is supported by the finding that cost matters for quality and quality matters for effectiveness. Greater selectivity will also help to reduce the burden on limited government capacity in some IDA countries and will free resources for more collaboration and follow-up needed to enhance the effectiveness of ESW. Selectivity could be enhanced by giving priority to ESW that informs Bank lending and strategy or that is clearly desired and needed by the client. ||Agreed/Ongoing. Management agrees with the importance of selectivity in allocating ESW resources. In the context of ESW and TA, greater selectivity may lead to reducing the number of tasks and to increasing the budget assigned to each of the tasks. But the issue should not be seen just in terms of numbers of tasks, but also in terms of consolidating tasks within a programmatic, multiyear, results-based approach so as to enhance their overall impact. The planning of the AAA program, including ESW and TA activities, takes place in the context of preparation of the CAS, and selectivity is a major consideration in preparing a CAS program that is tailored to country needs. As part of the consultation process, as well as ongoing country policy dialogue, specific knowledge gaps are identified. How and when to address them is planned, taking into account country needs, the work programs of other partners, and country circumstances. For IDA countries, issues with data quality and availability of analytic work prepared by the government or other sources may entail a larger effort to fill gaps over time. In countries where what is needed is knowledge on implementation (the engineering of reform) rather than traditional ESW, it is possible that, given weaker country capacity, smaller and more narrowly focused ESW or TA is more effective than larger efforts. In its AAA review, management will analyze resource allocation issues surrounding ESW and TA tasks and will propose steps to strengthen collaboration with in-country institutions and other stakeholders to ensure an effective ESW/TA program. |
|Enhance the institutional arrangements for undertaking ESW and TA. To the extent allowed by budget, ensure substantive task team presence in country offices, particularly in countries with low institutional capacity. This will facilitate closer collaboration with clients from task initiation through follow-up. In addition, formulate a dissemination and implementation strategy for ESW and TA at the concept paper stage. Such a strategy should identify the target audience, the mode of dissemination, and the follow-up arrangements after dissemination, all of which should be explicitly budgeted for as integral to the task. ||Agreed/Ongoing. Management agrees that task team presence in field offices, particularly in countries with low institutional capacity, will facilitate close collaboration with clients. The delegation of task management responsibilities to the field has been an ongoing process—for example, in the Sub-Saharan Africa Region and fragile states—and further steps are being considered. There are associated trade-offs that transcend budget constraints. Ensuring that ESW/TA is of top quality often requires that task management be carried out by thematic experts who cannot be decentralized, given their global role. In many countries, the size of the ESW program and budgetary constraints limit decentralization of specialized staff. In any country, the balance depends on the country context and the specific issues. Management is reviewing these considerations in the context of the ongoing review of decentralization. Management fully agrees with the recommendation to clearly formulate and adequately fund the dissemination and follow-up strategy for ESW and TA at the concept stage of the task. The AAA review will provide further guidance in this area. |
|Recognize and build on client preferences, whether for nonlending versus lending services or for TA versus ESW. Institute a mechanism to obtain client feedback on a periodic basis on delivered ESW and TA products. Such feedback should include clients’ views on collaboration, follow-up, and usefulness of the tasks (including specifics of how the tasks were used). The client feedback should be requested at a set period (for instance, around 1 year) after the delivery of the task to the client to allow time for follow-up, and it should be the last milestone for ESW and TA. Obtaining client feedback would encourage a stronger results focus for ESW and TA and would help counterbalance current Bank incentives for lending over nonlending and for ESW over TA. ||Agreed. Management agrees that client feedback is a key input to assess how clients view the different dimensions of ESW/TA in particular and the design of the Bank’s program in general. Feedback is already sought and used in the context of client surveys, CAS preparation, policy dialogue, lending preparation, and project implementation. Feedback focused on ESW and TA should provide further insights on client views as well as lessons for the design and implementation of the ESW and TA program. Overall, the focus should be more on the development results that might follow from changes in client approach and policies, rather than short-term client satisfaction with the production process. In the context of the AAA review, management will review options regarding the periodicity of requests for feedback to minimize the burden on country counterparts. Management will also analyze whether the feedback should focus on independent pieces of work rather than on a body of work with a common thematic focus. |
|Take the results tracking framework seriously, including by incorporating systematic client feedback, as noted in the above recommendation. ||Agreed. Management agrees with the recommendation to ensure greater attention to results, for example, by incorporating client feedback. Although use of feedback is an important component of a results framework, other elements could also be quite relevant. Attribution is certainly an issue: it is difficult to ascribe results to a single piece of ESW. In some cases a credible results framework can be established around a thematic program of activities that will help monitor results within an adequate time frame. In the context of the AAA review that is now starting, management will examine the results framework for ESW and TA and outline steps to strengthen it.|
Management will consider all these agreed actions completed with the conclusion of the AAA review and management decision based on the results of the review. Management will report to the Board on the outcome of that process in fiscal 2010.