Management welcomes this Independent Evaluation Group (IEG) review of the World Bank/International Finance Corporation (WB/IFC) Doing Business (DB) indicators. It notes the finding that the DB exercise has been effective in motivating interest, spurring dialogue on reforms, and stimulating action. Suggestions and recommendations in the review will be used to strengthen the DB process going forward. That said, management has a set of observations it would like to make on the analysis. Specific responses to IEG’s recommendations are given in the attached Management Action Record table.
Concurrence with the Broad Thrust of the Analysis and Recommendations The evaluation contains a number of important conclusions that management finds most helpful. Specifically, these include: - An acknowledgement that the DB exercise has been highly effective in spurring dialogue on reforms and motivating interest and action. It has also informed a large academic literature on regulatory reform and the impact of regulation on economic and social outcomes (nearly 800 academic articles as of June 2007). Such research can aid policy makers, particularly in developing countries, in the search for the optimal kind and level of regulation to ensure that the majority of the population can participate in economic activity and benefit from legal certainty and social protection.
- The recommendation to apply similar benchmarking to other development issues and to encourage the development of actionable cross-country indicators that can track improvements over time.
- The finding that while effective in catalyzing reforms debates and dialogue, the DB indicators have not distorted policy priorities or encouraged policy makers to make superficial changes to improve rankings.
- The conclusion that the DB employing workers index complies with the core labor standards and all other relevant conventions of the International Labor Organization.
- The conclusion that the legal origin, whether civil or common law, does not determine a country’s score in the DB indicators. A hypothetical civil law economy based on best practices would rank third in the global ease of doing business.
- Concrete suggestions on improving the transparency of the data collection and analysis and the respondent selection process.
- A recommendation to use other World Bank analyses, most importantly the Enterprise Surveys and Investment Climate Assessments, to inform the choice of topics in DB and enrich the analysis in future reports.
- A concrete proposal on piloting methodologies on new indicators before including them in the aggregate ranking on the ease of doing business.
Management Observations Management has four issues that it would like to raise with regard to the analysis and recommendations in this review.
Paying Taxes Indicator. The IEG review recommends revising the paying taxes indicator to include only measures of administrative burden. In management’s view, this recommendation is not consistent with another important recommendation, on the use of the other World Bank analyses to determine the priorities for regulatory reform. In the World Bank Enterprise Surveys, for example, tax rates are considered a top obstacle in twice as many countries as tax administration. In the Enterprise Surveys done in fiscal 2007, 17 of 40 find the tax rate to be among the top 3 obstacles, and 33 of 40 find it to be a bigger obstacle than tax administration. More generally, taxation is a regulatory tool and there is a trade-off between regulation and taxation. It is important to note that DB measures business taxes only, and therefore does not reflect the country’s overall fiscal policy and revenue collection.
Making Available Previously Published Data sets. The IEG review recommends making available all previously published data sets, not corrected for errors and methodology changes. This practice is unorthodox and is not followed by other major primary data providers. Instead, DB follows the practice of other data providers and makes available back-calculated data series, corrected for errors and methodology changes. These data are made available on the "Get Full Data" page of the DB Web site. In addition, the data used in the background research for DB are published on the "Research" page of the DB Web site. These two data sources have been widely used by researchers, with more than 800 academic papers utilizing the DB data. Management agrees to make more information available on reasons for data changes to facilitate the distinction between methodological changes, systematic changes in coding rules, and errors. All methodology changes are described in detail on the Web site at http://www.doingbusiness.org/MethodologySurveys/. All revisions that affect the data published in the DB 2007 report integrated as of the time of publication of the DB 2008 report are being made available at the "Get Full Data" page of the Web site.
Increasing the Number of Respondents. The IEG review recommends increasing the number of DB respondents. Management notes that the DB methodology fundamentally depends on reading the text of laws and regulations. DB respondents provide references to the relevant texts of the laws and regulations. This is unlike the methodology of perceptions-based surveys, which depend on having large samples of representative respondents. To ensure accurate interpretation of regulations and time estimates, DB works with local experts who routinely administer or advise on legal and regulatory requirements. Since 2004, 10,270 local experts have contributed. Management agrees to further increase the respondent pool, and has taken action, including through visits to 151 countries. In addition, management has hired a respondents’ manager as a member of the DB team to select and increase the number of respondents, focusing in particular on the poorest countries and other economies with the fewest number of respondents. Further, management commits to increase the involvement of Bank Group staff in recruiting respondents; to conduct annual data collection visits to the 50 economies with the fewest number of respondents; and to expand the piloted practice of giving out awards to the respondents who have contributed high-quality data over a sustained period of time.
Level of Regulation. The IEG review states that DB presumes less regulation is always better. This is incorrect. Six of the 10 indicators reward countries for having more regulation or a simplified way of implementing existing regulation. Top reformers in DB 2007 implemented stricter regulations (for example, China, Mexico, and Tanzania) or simplified their implementation (for example, Croatia, Guatemala, and Romania). The top 10 countries in the ease of doing business are Singapore, New Zealand, the United States, Hong Kong, Denmark, the United Kingdom, Canada, Ireland, Australia, and Iceland. Countries with no regulation receive a "no practice" score in the relevant area and the lowest ranking.
Conclusion Overall, management welcomes this evaluation from IEG. Management generally accepts IEG’s recommendations, with some caveats. Detailed responses to the recommendations are outlined in the attached Management Action Record.
Management Action Record | Recommendation | Management Response | To improve the credibility and quality of the rankings, the DB team should:
(a) Take a strategic approach to selecting and increasing the number of informants: - Establish and disclose selection criteria for informants.
- Focus on the indicators with fewest informants and countries with the least reliable information.
- Formalize the contributions of the supplemental informants by having them fill out the questionnaire.
- Involve Bank Group staff more actively to help identify informants.
(b) Be more transparent on the following issues of process: - Informant base: Disclose the number of informants for each indicator at the country level, differentiating between those who complete questionnaires and those who provide "supplemental" information.
- Changes in data: Disclose all data corrections and changes as they are made. Explain their effect on the rankings, and, to facilitate research, make available all previously published data sets.
- Use of the indicators: Be clear about the limitations in the use of the indicators for a broader policy dialogue on a country’s development priorities.
(c) Revise the paying taxes indicator to include only measures of administrative burden. Since the tax rate is an important part of the business climate, DB should continue to collect and present simple information on corporate tax rates, but exclude it from the rankings (as it does for information on nonwage labor costs in the employing workers indicator). A wider range of informants should also be engaged for the paying taxes indicator.
| Mostly Agreed. Bank Group management mostly agrees with this recommendation, noting that it primarily points to the importance of intensifying the rigor of recruiting and maintaining a large pool of expert respondents. - To implement the first part (point a) of this recommendation, management has hired a respondents’ manager on the DB team. The task of the respondents’ manager is to select and increase the number of respondents, focusing in particular on the poorest countries and other economies with the fewest number of respondents. In addition, management commits to increase the involvement of Bank Group staff in recruiting respondents and to conduct annual data collection visits to the 50 economies with the fewest number of respondents. Thirdly, management will expand the piloted practice of giving out awards to the respondents who have contributed high-quality data over a sustained period of time. Such awards serve to express gratitude for the respondents’ efforts and to maintain the pool of respondents.
- To implement the second part (point b) of this recommendation, management commits to disclosing the number of respondents for each indicator at the country level, starting with the launch of Doing Business 2009Â . Management is also making available details on data corrections/changes and methodology changes that have been made in the year following the launch of the previous report. Lastly, management commits to expanding the discussion on the limitations in the use of the DB indicators in the "Methodology" section of the report and on the Web site. However, management disagrees with the recommendation to make available all previously published data sets, not corrected for errors and methodology changes. This practice is unorthodox and is not followed by other major primary data providers. The data used in the background research for DB are already published on the "Research" page of the DB Web site. The full time series of DB data, corrected for errors and methodology changes, is also available at the "Get Full Data" page of the DB Web site. These two data sources have been widely used by researchers, with more than 800 academic papers utilizing the DB data.
- Management mostly disagrees with the last point (point c) of the recommendation. The tax rate is often identified as a major constraint to business activity in the World Bank Enterprise Surveys. Including a measure of overall tax burden in the DB indicators provides a complete treatment for the topic of paying taxes. Focusing only on the administrative burden of paying taxes will take the DB methodology away from covering a broader spectrum of areas relevant to small domestic businesses. However, management commits to expand the range of respondents on the paying taxes’ survey by recruiting a larger set of accounting and tax experts. Â
| To make its reform analysis more meaningful, the DB team should: (a) Make clear that DB measures improvements to regulatory costs and burdens, which is only one dimension of any overall reform of the investment climate. (b) Trace the impact of DB reforms at the country level. The DB team should work with country units to analyze the effects of implementing the reforms measured by the DB indicators (such as revised legislation or streamlined process) on: (i) firm performance, (ii) perceptions of businessmen on related regulatory burdens, and (iii) the efficiency of the regulatory environment in the country.  | Agreed. Bank Group management agrees with this recommendation and will strive to make it even clearer in future DB reports and presentations that DB covers only some dimensions of the overall reform of the investment climate. Management also commits to a measurement and evaluation agenda, in partnership with WB country units and IFC regional facilities, to document the effect of DB reforms on a set of economic and social indicators. The World Bank Enterprise Surveys in particular will be used for this work.  | To plan future additions to or modifications of the indicators, the DB team should: (a) Use Bank analyses to drive the choice of DB indicators. Business Enterprise Surveys, Investment Climate Assessments, and other work can help determine stakeholders’ priorities for domestic private sector growth. The DB team should use such analyses to determine the choice of new indicators, and periodically assess its current set of indicators. (b) Pilot and stabilize the methodology before including new indicators in rankings. Frequent changes in methodology make comparison across time less meaningful. New indicators should be piloted (that is, data collected and published for comment, but not factored into the rankings) until the methodology is validated and stabilized.  | Agreed. Bank Group management agrees with this recommendation and will direct the DB team toward using other Bank Group analyses, and in particular the Enterprise Surveys and Investment Climate Assessments, for both determining the choice of new indicators and periodically assessing the existing set of DB indicators. Management also commits to publishing new sets of indicators in future DB reports for comment, while not factoring those in the rankings until their methodology is validated by academic research.  |
|