Click here for search results

Governance Matters III: Frequently Asked Questions

spacer_20x3Homespacer_10x2Dataspacer_10x2Papersspacer_10x2Programsspacer_10x2Diagnosticsspacer_10x2In the News

This page provides answers to the most relevant and most commonly raised issues about our Governance Indicators contained in the paper: Governance Matters III: Governance Indicators for 1996-2002.

QUESTIONS

GENERIC
What is meant by Governance?
What are the 6 dimensions of Governance?
How frequently are the Governance Indicators updated?
Will you go further back in time to compile Governance Indicators for years previous to 1996?
How many countries are covered by the Governance Indicators?
What options do I have to access the data?
What are the underlying sources for the Governance Indicators?
What is the criteria used to assign different colors to countries in the interactive charts and worldmaps?
It may be useful that governance can be measured, but does it really matter?

MORE ADVANCED
Why are margins of errors important?
Why do you use subjective measures as opposed to objective indicators?
How do I interpret changes in countries' estimates over time?
In simple terms, how is the aggregation methodology carried out to produce Governance estimates?
Can we infer any global trend over time from the Governance Indicators?
How confident can we be that over time changes are indeed significant?
Why do you distinguish between representative and non-representative sources?
How can I see what variables were actually used to compile the Governance Indicators?
What implications can we draw in regards to the Millennium Challenge Account (MCA)?
How have potential ideological biases in poll agencies' ratings been addressed?
How confident can we be that rankings drawn from point estimates are accurate?
How confident can you be that sources are independent from each other, as assumed in your aggregation process?
What is the best use I can make of these Indicators?
Why are there countries with ratings above 2.5 or below -2.5?

What is meant by Governance?

Governance can be broadly defined as the set of traditions and institutions by which authority in a country is exercised. This includes (1) the process by which governments are selected, monitored and replaced, (2) the capacity of the government to effectively formulate and implement sound policies, and (3) the respect of citizens and the state for the institutions that govern economic and social interactions among them.

For more information, consult the paper (pages 3-5).

Back to top 520

What are the 6 dimensions of Governance?

The six dimensions of Governance are: Voice and Accountability; Political Stability and Absence of Violence; Government Effectiveness; Regulatory Quality; Rule of Law; and Control of Corruption.

Voice and Accountability includes in it a number of indicators measuring various aspects of the political process, civil liberties and political rights, measuring the extent to which citizens of a country are able to participate in the selection of governments.

Political Stability and Absence of Violence combines several indicators which measure perceptions of the likelihood that the government in power will be destabilized or overthrown by possibly unconstitutional and/or violent means, including domestic violence and terrorism.

Government Effectiveness combines responses on the quality of public service provision, the quality of the bureaucracy, the competence of civil servants, the independence of the civil service from political pressures, and the credibility of the government's commitment to policies.

Regulatory Quality instead focuses more on the policies themselves, including measures of the incidence of market-unfriendly policies such as price controls or inadequate bank supervision, as well as perceptions of the burdens imposed by excessive regulation in areas such as foreign trade and business development.

Rule of Law includes several indicators which measure the extent to which agents have confidence in and abide by the rules of society. These include perceptions of the incidence of crime, the effectiveness and predictability of the judiciary, and the enforceability of contracts.

Finally, Control of Corruption measures perceptions of corruption, conventionally defined as the exercise of public power for private gain.

For further details, consult the paper (pages 4-5).

How frequently are the Governance Indicators updated?

The Governance Indicators are updated every two years. All relevant information (including data, methodological papers, interactive charts, and world maps) for the last round of updates (2002) is already posted on the web at: http://www.worldbank.org/wbi/governance/. The next round of Governance Indicators will be posted in early 2005.

Will you go further back in time to compile Governance Indicators for years previous to 1996?

No, 1996 will remain our starting year. As we go back in time, we would have to drop several of our sources as they became available only in recent years. Dropping sources would decrease both the precision of our estimates (i.e. higher standard error) and the interpretation of changes over time (as a country relative position could be affected by the subtraction of sources rather than an actual change in its performance).

Why are margins of errors important?

Inherent to all Governance Indicators is a margin of error, which might vary from country to country, normally attributable to two factors: (i) cross-country differences in the number of sources in which a country appears, and (ii) differences in the precision of the sources in which each country appears.

In spite of the considerable number of individual sources used (which tends to decrease the extent of measurement error), there are still substantial margins of error associated with governance estimates. This implies among other things that it is difficult to assign many countries to a definitive performance category according to their estimated level of governance, and even more difficult to compile precise rankings.

For a more thorough discussion, consult the paper (pages 12-15).

Why do you use subjective measures as opposed to objective indicators?

The primary reason for this choice is that for many of the key dimensions of governance, such as corruption or the confidence that property rights are protected, objective data are almost by definition impossible to obtain, and so there are few alternatives to the subjective data on which we rely.

For a more thorough discussion, consult the paper (pages 22-24).

How do I interpret changes in countries' estimates over time?

A change over time could be attributed to 4 factors. First of all, it could come from a change in the score assigned to a country by the same source over time. This is the most common and relevant factor, directly reflecting changes in perceptions of the country's performance. A second factor is the addition of new sources whose ratings might be different from the average ratings from pre-existing sources. Changes over time in relative performance may also reflect the addition of new countries to the aggregate indicator. If for example we add a country with a governance rating that is high relative to those countries already in the index, then by construction all the countries which rank lower than this country will receive lower scores. Finally, a change in a country's performance could derive from a change in the weights in the aggregation procedure. Overall, however, these last two factors typically have only very small effects on changes.

For more details, consult the paper (pages 16-19).

What are the underlying sources for the Governance Indicators?

We use 25 sources from 18 different organizations.

For more details on these organizations, click here.

In simple terms, how is the aggregation methodology carried out to produce Governance estimates?

We use an Unobserved Component Model (UCM) to aggregate the various response in the broad 6 clusters. . This model treats the "true" level of governance in each country as unobserved, and assumes that each of the available sources for a country provide noisy "signals" of the level of governance. The UCM then constructs a weighted average of the sources for each country as the best estimate of governance for that country. The weights are proportional to the reliability of each source. The resulting estimates of governance have an expected value (across countries) of zero, and a standard deviation (across countries) of one. This implies that virtually all scores lie between -2.5 and 2.5, with higher scores corresponding to better outcomes.

For technical details, consult the paper (pages 8-12).

How many countries are covered by the Governance Indicators?

Coverage varies depending on the indicator and the year. In 2002, Voice and Accountability has the largest coverage (199 countries), Political Stability has the smallest (186) while all other dimensions have same coverage (195).

For a complete list of countries for each Governance Indicator, consult Appendix C of the paper (pages 89-106).

Can we infer any global trend over time from the Governance Indicators?

Our indicators measure governance in units where the average score for the world as a whole is zero in every period. Therefore, the Governance Indicators are only meant to capture countries' relative position vis-a-vis the others. We can however use our indicators jointly with our underlying sources to draw conclusions on broad global trends. For instance, since many of our individual sources show a deterioration over time in worldwide averages, then we can safely infer that a country's deterioration in its relative position cannot be attributed to an overall improvement in other countries, but rather is likely to reflect a poorer performance by the country.

For a more detailed discussion, consult the paper (pages 8-12).

How confident can we be that over time changes are indeed significant?

The margins of error associated with levels of governance are substantial. Since changes over time are in most cases small relative to levels of governance, it is safe to assume that most of the observed changes over time are neither statistically nor practically significant. However, there are some cases where the changes over time are large enough that the 90% confidence intervals in the two periods do not overlap. This rule of thumb helps to identify cases of changes over time that are likely to be of practical significance.

A more detailed discussion of confidence intervals, standard errors and changes over time can be found in the paper (pages 14-19, see also figure 3 on pages 49-51).

Why do you distinguish between representative and non-representative sources?

This distinction allows for minimization of the imprecision of point estimates due to measurement errors in underlying sources. First of all, non-representative sources are more likely to be subject to higher measurement error given their more limited scope (for instance, a source rating only rich countries might give them lower ratings than other sources covering a more balanced panel of low and high-income countries). On a more technical note, the distribution of unobserved governance in the subset of countries covered by these smaller-scope sources is not the same as that in the world as a whole. As a result, for these sources we cannot make the assumption that unobserved governance in the countries covered by these surveys follows a standard normal distribution, as is required by the maximum likelihood procedure.

For a more detailed discussion, consult the paper (pages 8-12).

How can I see what variables were actually used to compile the Governance Indicators?

Appendix A of the paper (pages 54-81) lists all the sources that were used, along with a brief description and weblink to the respective homepages.

Appendix B of the paper (pages 82-88) instead provides details on how we have assigned individual questions from each of these sources to our six governance clusters.

What options do I have to access the data?

Appendix C of the paper (pages 89-106) provides a printout of all the data. For each indicator, it shows for each country the estimate level, the standard error and the number of sources used in each year.

Alternatively, you can download the complete dataset in excel format directly from this website. To do so, click here.

To download only specific data tailored to your needs, try this new user-friendly feature of the interactive website.

What implications can we draw in regards to the Millennium Challenge Account (MCA)?

The proposed MCA allocation rule is designed to ensure that MCA funds will be allocated to low-income countries with relatively sound policies and institutions. A group of 74 countries that are eligible for concessional IDA lending from the World Bank, and which have per capita incomes less than $1435 in 2001, will potentially be eligible for MCA funds in its first year. According to the MCA eligibility rules, this set of countries will be rated according to 16 performance criteria covering three dimensions of performance: "governing justly" (6 criteria), "investing in people" (4 criteria), and "promoting economic freedom" (6 criteria). Four of the Governance Indicators we have constructed (voice and accountability, government effectiveness, rule of law, and corruption) have been proposed as performance indicators under the MCA's "governing justly" performance dimension, with the remaining two for this dimension being measures of civil liberties and political rights constructed by Freedom House. In addition, a fifth governance indicator, Regulatory Quality, is included under "promoting economic freedom". In order to qualify for MCA assistance, countries must (a) be in the top half of all potentially eligible countries according to the corruption rating from the governance indicators, and (b) must be in the top half of all potentially eligible countries on at least half of each of the performance criteria under each of the three dimensions of performance. This rule is designed to ensure that resources are channeled towards countries that are performing well in a variety of dimensions of governance, and in which corruption especially is relatively low.

However, it is important to note that the substantial margins of error associated with governance estimates mean that it is difficult to assign many countries to a definitive performance category according to their estimated level of governance. This point applies to any of the MCA criteria. Given the presence of non-trivial margins of errors, some countries ranked under the median might in fact belong to the top half of the distribution. Classifications based on individual indicators, or even on a single aggregate indicator, inevitably run the risk of mis-classifying countries due to the margins of error inherent in all indicators.

This underscores the need for a certain degree of flexibility in the MCA allocation rule, in light of the importance of caution when using governance indicators to classify countries into groups. To reduce the risk of misclassification, it is important to look at a variety of indicators and additional sources of data, especially for borderline cases.

For a more detailed discussion, consult the paper (pages 26-29).

How have potential ideological biases in poll agencies' ratings been addressed?

We address this issue as follows. Our identifying assumption is that surveys of firms or individuals are not tainted by ideology, since they reflect the views of a large number of respondents in each country. In contrast, it is possible that the views of a smaller number of raters affiliated with a particular institution may reflect the ideology of that group. We can therefore identify the effects of ideology by looking at the correlation across countries between the ideology of the government in power, and the difference in the percentile ranks assigned to countries by a poll of experts and a survey of individuals and firms. We implement this idea using the World Bank's Business Environment Survey for 2000, and an indicator variable that takes on the value 1 if the government in power is left-of-center, 2 if it is center, and 3 if it is right-of-center, taken from the database of political institutions constructed by Beck et. al. (2001). The coefficient on the ideology variable will therefore capture the extent to which a given poll of experts rates countries countries with left- or right-wing governments systematically differently from a survey (a positive coefficient indicates that the poll in question tends to rate right-of-center governments more highly relative to a survey). Our results showed that we find only one source which appears to have a consistent ideological bias, with the Heritage Foundation assigning relatively higher scores to countries with right-of-center governments than the corresponding survey and this "ideology bias" is fairly modest in magnitude.

For a more detailed discussion, consult the paper (pages 24-25).

What is the criteria to assign different colors to countries in the interactive charts and worldmaps?

Each country color pattern follows a simple quartile distribution (for illustrative purposes): the best quartile (over 75th percentile) is in green (with top 10% colored in darker green), the second best quartile (over 50th) is in yellow, the third (over 25th) is in orange, and the fourth is in red (with bottom 10th in darker red). Please note that this simple color coding does not account for the size of the confidence intervals; the color mapping is based on the point estimates.

To access interactive charts and/or maps, visit our interactive webtool.

 

How confident can we be that rankings drawn from point estimates are accurate?

Because of margins of errors, we cannot make precise rankings. However, we can still make inferences based on confidence intervals. If we for instance divide the distribution of countries' estimates into two categories (low-high governance rating), we can calculate the probability that any given country might indeed belong to the opposite side of the distribution.

A more detailed discussion of confidence intervals, standard errors and rankings can be found in the paper (pages 11-14, see also figure 1 on pages 45-47).

 

How confident can you be that sources are independent from each other, as assumed in your aggregation process?

It is true that an important assumption of our Unobserved Component Model is that the errors are independent across sources. This assumption in particular imposes the identifying assumption that the only reason why two sources might be correlated with each other is because they are both measuring the same underlying unobserved governance dimension.

We have taken several steps to ensure that this assumption would hold. In the first place, we have avoided including sources which were themselves constructed upon other indicators used in the aggregation procedure. For instance, we did not include the Corruption Perceptions Index (CPI) by Transparency International because the CPI is itself an aggregate of a number of individual sources, all of which were included separately in our corruption indicator.

Secondly, we were very cautious in flagging risk rating agencies who would base their own assessments on the assessments of other agencies included in our sample. We have to the best of our knowledge excluded any source of governance data where we found it was explicitly based on another one of our sources.

If errors are positively correlated across sources despite the precautions we have taken, this would likely have little effect on the governance estimates we construct. However, it would imply that the margins of error we have constructed are quite conservative, and that the true level of imprecision of the indicators would be even large than we have estimated.

For more details, consult the data section in the paper (pages 4-8).

 

What is the best use I can make of these indicators?

Notwithstanding the substantial increase in data collection for this round, which has both expanded country coverage and improved the precision of the aggregate indicators, margins of error remain. We hope that in the future the availability of additional data will enable further improvements in precision. However, the presence of margins of errors imply that we cannot make precise rankings of the countries solely based on the point estimates.

The Governance Indicators however can serve the purpose of providing individual countries with a set of monitorable indicators of governance they can use to benchmark themselves against other countries and over time. We recognize there are limitations to what can be achieved with this kind of cross-country, highly-aggregated data. Therefore, this type of data cannot substitute for in-depth, country-specific governance diagnostics as a basis for policy advice to improve governance in a particular country, but should rather be viewed as a complement tool.

A more detailed discussion of confidence intervals, standard errors and rankings can be found in the paper (pages 11-14 and pages 45-47).

 

It may be useful that governance can be measured, but does it really matter?

It matters enormously. We find that a country improving its quality of governance from a low level to an average level can in the long term quadruple the income per capita of its population, and similarly reduce infant mortality and illiteracy. And the direction of causality is clear: it goes from better governance to higher incomes, and not vice versa In other words, governance is not a 'luxury' good that only wealthier countries can afford; is not the automatic result of development. To the contrary, it requires continuous political will and commitment, and difficult work.

For a more detailed discussion, consult the papers Governance Matters I and Growth Without Governance.

 

Why are there countries with ratings above 2.5 or below -2.5?

Given our assumption about governance being normally distributed, there is a 99% chance that a country's rating would fall between -2.5 and 2.5. However, under very specific circumstances, a country's rating might exceed these thresholds. This simply means that the country has an extremely poor record (below -2.5) or extremely good record (above 2.5) in that specific governance indicator.




Permanent URL for this page: http://go.worldbank.org/9C0O2YRTE0