The Independent Commission for Aid Impact (ICAI), UK’s aid watch dog, today, released its review of DFID’s programming and results in water sanitation and hygiene (WASH). In this impact review, they take a close look at the results DFID reported in its 2015 Annual Report; results that cost £ 713 million between 2010 – 2014.
Do read the full report here.
Some thoughts on the areas of concern in the report:
- The focus on ‘leaving no one behind’ is spot on. It is easy to stack up impressive WASH numbers if one ignores the poorest and the most vulnerable in communities. Safe sanitation and hygiene need to be universal for health benefits to accrue to communities. Within WASH, sanitation is specifically complex, sometimes also called a ‘wicked problem’ – a challenge foremost, of inducing lasting behaviour change. The very nature of careful social engineering required to bring about this behaviour change seems to run contrary to some of the factors that make an intervention scalable – an ability to standardise inputs and break programme components down to easily replicable bits.
- Within the broad basket of ‘service delivery’ interventions, WASH is one of the trickier sectors when it comes to measuring sustained impact, especially at scale. Naturally then, ICAI find that while DFID’s claims of having reached 62.9 million people are broadly correct, it is very hard to establish if the benefits are sustained. Therefore, the results reported remain at the ‘output’ level and that is what ICAI ends up assessing, even though what they set out to do is an ‘impact’ review. While the report speculates on sustaining benefits beyond the 2011-15 period, I wonder whether those that accessed the programme in 2011-12 continued to experience any benefits in 2015.
- The link with government systems, in terms of implementation, monitoring and sustenance remains unclear: another typical WASH issue. Barring say, India, (and this is true especially in sub-Saharan countries, government WASH budgets are highly inadequate. A lot of the work that happens is funded by donors and this implies that monitoring and maintenance happens outside the official system. Achieving local ownership in such a context is a challenge.
- ICAI finds it difficult to assess value for money (VfM) in DFID’s WASH programmes. On one hand, it finds that there isn’t enough competitive procurement, but also there is a lack of established metrics and benchmarks to analyse VfM. Following DFID’s own 3Es framework, an Economy and Efficiency analysis should be possile across the portfolio, and as far as I can tell, is rapidly being developed in the sector, and within DFID. However, partly as a consequence of the lack of ‘outcome/impact’ data, cost-effectiveness studies are likely to remain a challenge. This work by an OPM-led consortium should be particularly relevant in improving VfM analysis across the sector.