The third Conference on the Institutionalization of Monitoring and Evaluation systems in the LAC region, organized jointly with the IADB, took place in Lima, Peru on July 23 and 24, 2007.The conference attracted more than 80 regional and international professionals from over 12 countries working in the area of M&E. This year’s conference agenda reflected the evolution of interest in the region offering presentations by experienced professionals on a number of ‘hot topics’ in M&E such as, impact evaluations, the role of congress and sub national M&E systems.
Latin American and Caribbean Network for M&E professionals
Launched in 2005 by the IADB and the World Bank, the informal network of M&E professionals in LAC has the objective of creating a forum for people to share their experiences and knowledge in the area of M&E. Its creation was a response to the growing interest of governments in LAC to orient their public administrations towards results through the usage of M&E tools, and the lack of good information to support this need. Three years into the initiative we can say with confidence that the network has contributed to building a knowledge base for M&E practitioners and has facilitated the exchange of country experiences in the region. What appeared as a fairly scattered collection of concepts at the 2005 network conference had evolved into a more integrated view of M&E by the most recent 2007 conference. The M&E network has not been alone in contributing to the advance in M&E knowledge indeed one can speak of a mobilization around M&E by governments, NGOs and aid agencies alike. With this energy new challenges arise such as ensuring quality of information and keeping focused on key objectives. We must be aware of these challenges whilst continuing to move forward sharing experiences and reducing the knowledge gap that still exists on many fronts.
M&E systems generate information which enables better governance through evidenced based policy making, improved policy design, better management, and strengthened accountability (Mackay 2007). A useful way to conceptualize a well functioning M&E system is one which facilitates equilibrium in the supply of, and demand for, high quality information, monitoring tools and evalaution techniques. What does a government do to achieve this? On the supply side a government may emphasize the need to strengthen registry systems, statistical agencies which collect key data and standardized methodologies for program evaluation and design. On the demand side, there is the key issue of incentives for the public administration to use the information which is being supplied to them, indeed this goes both ways, producers of information also need the right incentives to generate quality information. Dissemination strategies are also important for bolstering demand from the government and civil society, the latter of which are vital to ensuring the M&E system serves the purpose of enhancing social accountability.
These presentations from Chile, Brazil, Peru and Mexicoserve as case studies of national M&E systems in the LAC region
A key challenge which governments face when seeking to strengthen their M&E arrangements is finding the right equilibrium between national and sub national M&E systems. This can become a significant issue particularly as some countries choose to move towards greater decentralization. Sub national systems traditionally play a vital role in the collection of data as most monitoring information is collected at the facility level. This role is not restricted however; Sub-national M&E systems can serve very much the same functions as national systems by improving accountability, planning and governance in general. Indeed there are numerous examples of strong sub-national level and city level M&E systems in the region such as those in Pasto and Medellinwhich perform a range of functions. M&E systems at the sub-national level critically also give the opportunity to tailor an M&E system to local objectives and capabilities.
These presentations from Argentina, Mexico, Brazil and Colombiaoffer interesting case studies of sub national M&E systems.
Experience with government monitoring and evaluation (M&E) systems shows that well functioning systems must have both a supply of good quality information. Equally important is the demand for the information the system produces coming from stakeholders. The latter component is vital for a number of reasons, including social and intergovernmental accountability; results based public management and sustainability of the system. Simply put, an M&E system is a set of tools which only gain real value in terms of improved governance if stakeholders use them. Congress plays an important role as a check on government, a representative of the people and most tangibly perhaps, congress needs to pass the annual budget before it is implemented. For all of these reasons and more congress is an extremely important consumer of M&E information. The role of congress in M&E system has until now not been very large in LAC, nonetheless steps forward are being made in a number of countries. Professionals of M&E are becoming increasingly aware of the importance of congress and seeking to understand and address the challenges these representatives face in absorbing effectively the information M&E systems produce as well as their potential role as drivers of improved M&E systems.
These presentations from Mexico, Peru, Chile and Argentinaoffer important case studies of the role of congress in M&E:
Islands of best Practice: Monitoring and Evaluation Systems in Agencies and Ministries
Mackay (World Bank 2007) describes sector systems for M&E a ‘frontier’ issue for governments trying to achieve a successful M&E system. There exists little doubt among practitioners that sector systems are very important both in terms of the information they supply to the central M&E system and as users of their own information for program management. However, there has traditionally been in the literature an emphasis on the national level and not on sectoral experiences. Sectoral experiences can offer important lessons for other ministries but also serve as a microcosm of the challenges faced by government wishing to set up a national system. An understanding of the sectoral dimension is also extremely important for governments wishing to move forward in building a national M&E system because it is rare that different ministries have exactly the same capacity as each other in their M&E system. This heterogeneity highlights the importance of good diagnostics and the creed of M&E practitioners that one size does not necessarily fit all.
These presentations from Peru, Mexico, and Colombiaprovide insights into the experiences of transport, agricultural, social development and social protection ministries:
One of the most powerful tools in M&E systems is impact evaluation which using specific designs and statistical techniques can yield very precise and reliable results of government programs. They are thus a key tool for evidence based policy making. One of the key features of impact evaluations is that they should be carefully tailored to specific program needs and the data and budget available to evaluators. There are also key steps which should be common to all good impact evaluations such as bringing programs through the logic framework, finding a good control group and choosing indicators which are clearly linked to the program objectives. Impact evaluations are not a panacea however; they can be lengthy and costly for a government and sometimes not the right M&E tool for the specific needs at hand. Nonetheless results from impact evaluations have lead to real improvements in program design and results, and in many cases have been extremely cost effective. Impact evaluations have become key in the transfer of knowledge throughout the region and the world, a prime example being the impact evaluations of Oportunidades in Mexicowhich have contributed to the expansion of Conditional Cash Transfer as a tool for development.
International best practice countries garner much interest from policy makers seeking to benchmark their own progress and find new strategies to move forward in their own M&E systems. For example, performance based budgeting is practiced in some way by most OECD countries, a successful linkage of the budgetary decision making process with M&E results is perhaps the biggest challenge of an M&E system and much can be learned from countries which have been able to advance on this front. Nonetheless it is important to keep in mind, as demonstrated in these presentations and the wider literature that a countries vision for its M&E system depends its priorities at any give time, for some it is the effective monitoring of poverty reduction strategies, for others it is tightening of fiscal expenditures. Related to this is also the reality that M&E systems, like the governments which establish, sustain and sometimes hurt them, are subject to trends which go both up and down. Lessons which can be drawn from this insight is the need to focus on the institutionalization of M&E systems and achieving wide spread buy in from different stakeholders, both of which are key to ensuring the sustainability of the system beyond the life of specific administrations and public personalities.
These presentations from Canada, the USA and Australiapresent three case studies of M&E systems in OECD countries: