Indicators of the Strength of Public Sector Management Systems (ISPMS) - A Multi-Donor Initiative to Measure Public Sector and Governance Institutions
The Indicators of the Strength of Public Management Systems (ISPMS) initiative is a multi-stakeholder effort coordinated by the World Bank to identify and develop international consensus around a comprehensive and comprehensible set of cross-national performance indicators for governance and public management systems that will provide a “health check” and help to prioritize and target reform efforts.
The German Ministry for Economic Cooperation and Development (BMZ) provided support to launch the initiative by financing the establishment of a Secretariat (located in the World Bank) and the first phase of technical work.
Why are ISPMS needed?
The institutional arrangements for managing staff and resources within the public sector, as well as the broader governance environment within which the public sector operates, matter fundamentally for development outcomes. However, the international community lacks the data it needs to track whether and how we are making progress. At the Fourth High Level Forum on Aid Effectiveness, held in Busan in late 2011, it was widely recognized that measuring public sector and governance institutions is a key development challenge. The Effective Institutions Platform that formed out of Busan has underscored the need for high quality "indicators of success", making it a priority to collect data on strengthening institutions in a more coherent manner and promote partner-led and joint assessments on the strength of public sector institutions. Likewise, a key finding of the report of the High Level Panel on the Post-2015 Development Agenda is that "better data and statistics will help governments track progress and make sure their decisions are evidence-based; they can also strengthen accountability."
Initially, the initiative has focused on identifying indicators in the areas of public financial management, procurement, public administration and civil service, tax administration and public information. Transparency, accountability and participation are key themes that cut across these systems.
Who are the users of the ISPMS?
The main users of these indicators are expected to be governments, development agencies/donors, citizens and civil society organizations, and researchers. Governments can use the "health check" to prioritize and improve the design and implementation of their public sector reform initiatives. Donors and development agencies can use the indicators to develop strategies (including whether or not to use country systems), plan programs, set targets and track results. Citizens and civil society organizations can use the indicators to hold government to account and advocate for change. Researchers can use these indicators to conduct empirical tests of which reforms matter for development outcomes and service delivery in which contexts.
The Steering Group
A Steering Group and several Technical Expert Groups were also established to guide the work at the strategic and technical levels. The Steering Group is composed of members from the World Bank, the UK Department for International Development, BMZ, the Asian Development Bank, the Organization for Economic Cooperation and Development, the Australian Government Overseas Aid Programme, the Canadian Department of Foreign Affairs and Trade, the Swedish International Development Cooperation Agency, the European Commission, Global Integrity, the International Budget Partnership, Transparency International, and the Government of Bangladesh.
Progress in Phase I
The Secretariat developed a set of criteria aimed to address two key problems with prior efforts to develop indicators: (1) an inability to demonstrate a clear connection between reform, longer-term institutions building, and other development outcomes and (2) complicated and expensive data collection efforts. Based on the set of agreed utility and feasibility criteria (Table 1), the Technical Expert Groups have taken stock of existing indicators and databases. Of over 750 existing indicators of governance and public management systems from 25 databases reviewed, about 100 were identified as meeting the criteria.
Table 1. ISPMS criteria
1 Existing indicators were determined to be feasible if they were available for at least 20 countries and had been collected at least twice, but not necessarily regularly (e.g. biannually).
|ISPMS Criterion ||Definition |
|Utility Criteria |
|action-worthy ||Indicator measures an aspect of PSM performance which makes a difference for development outcomes and/or is intrinsically valuable in itself |
|actionable ||Indicator is specific enough to point to policy actions that can be taken to make improvements (e.g. not a composite indicator) |
|behavioral ||Indicator captures the functioning or performance of institutions, rather than specific legal, organizational or institutional forms |
|replicable ||Indicator is generated transparently and if the measurement was undertaken by someone else, they would reach similar conclusions |
|Feasibility Criterion |
|feasible ||Indicator should be sustainable to collect1 |
This first phase of work revealed that there are three critical gaps in the coverage of existing indicators:
- The indicators lack a 'whole of government' perspective. When added together, the currently available indicators do not provide a comprehensive picture of how the machinery of government in a country works.
- Coverage across systems, countries and time is uneven. In some key public management systems, such as public administration and civil service, tax administration, and procurement, critical areas of performance remain uncovered. For example, there are currently no indicators of staff qualifications (including training received) in public administration and civil service. In addition, time series data is rarely available, limiting indicators’ usefulness for decision making. For example, PEFA indicators encompass the budget cycle, providing good coverage of public financial management topics, they are not collected regularly for countries over time.
- There is a need for more objective assessments of performance. For example, indicators on public sector remuneration levels are currently collected via expert surveys but could be replaced with data from national records. For some of the PEFA indicators, actual performance figures could replace the assessment grades.
The Steering Group is currently planning Phase II of the project, which will look at ways to fill these gaps in coverage by (i) sourcing new indicators and improving coverage through a "marketplace"; (ii) incubating indicators in areas that are conceptually difficult to measure; and (iii) identifying and exploring administrative data sources where these are feasible.