Tuesday, 24 April 2012

Systems Failure - Paul Byars


Our research team, from Edinburgh University, have been working since February on analysing failure in water supply systems in Sierra Leone. Before we arrived in Sierra Leone we were already aware that this country had significant problems. Various NGOs and research groups had already indicated that the failures of the water supply systems in Sierra Leone were as high as 65%. Providing water supplies, in particular handpump wells, is a typical response of many Non-Governmental Organisations (NGOs), when attempting to reduce widespread problems related to health. In Sierra Leone’s neighbouring country, Liberia, a recent report by ‘the Liberian WASH Consortium’ suggested that 85% of the wells were provided in the immediate aftermath of its civil war. In the account of Liberia, after only 8 year of peace, there is a reported 50% failure of their systems. Sierra Leone has had a longer period of peace, and with a larger population and more water supply interventions. Therefore it is possible to suggest that the rates of failure may also be higher than their neighbour.

In Sierra Leone the water supply interventions have ranged from providing hand-pump well systems, to pulley systems to boreholes. We are primarily interested in the water supplies provided to the rural communities in the aftermath of the war. More recent interventions, such as reservoirs and piped water systems (few as they are) have not been included in our research. We are also aware that much of the rural population has yet to be served with any system. Studying this, though a significant problem in itself, would not help us assess the weaknesses of the current water supply systems.

Unlike the research carried out by the Rural Water Supply Network, InterAide and more recently by the World Bank using FLOW technology, our research is not focussed on targeting as many villages as possible, as this would only result in a binary (working/not working) analysis of the systems. Instead our focus is on a much more detailed, in-depth and qualitative understanding of failure. Our work is intended to give a reliable case study on what may be happening on the ground, on more than a single level, of the socio-technical system.

Our methodology is split into several parts. We start by engaging directly with the community as a group, meeting with chiefs, elders, community leaders and community spoke-persons (and anyone else that wants to attend, all are welcome). From this group we can better understand the village formation, the interactions with the NGOs and local government, and also the type of systems that they have been provided with. From this meeting we receive basic information about the functionality of their systems, and how the private and public sectors respond to their needs. The community also provide us with information on their social infrastructure that is designed to support the wells; these include their user committees, their local leaders, their well technicians, the local government support they have received as well as the community’s access to spare parts and tools.

The community surveys are followed by household Knowledge Attitude and Practice (KAP) surveys. These address the household aspects of water supply. These questions are dramatically different from the community questions. The KAP surveys are focused on individual household water demands, usage, supply, sources, treatment and storage. The questions are asked in confidence and offer an excellent insight into how individual household respond to their water needs.

The final part of our assessment makes use of the teams technical (engineering) skills. In this part we carry out a full technical assessment of the communities improved water points. This covers all aspects of the well, including the type of well, who built them, how long they have been working, the condition of each individual part, the water quality etc. We accompany this by visiting any of the village’s unimproved sources. These sources are normally used in times of community water shortages caused by seasonal failure and systems malfunctioning.

Though this initial assessment gives a good indication of some of the problems, even spending a day is insufficient time to fully understanding the links between the communities and their sources of water. Our many survey questions have to be structured (or in some cases semi-structured) so that we can complete our research in a single day, and have the ability to analyse the data. This means that there are still unresolved questions from the communities regarding water. To provide more qualitative responses some of the villages are targeted for a second day of focus group discussions. In this we speak to the Water User Committees (those responsible for ensuring access and upkeep of the wells and tariff systems), the technicians (those who have been trained to fix the wells) and the community well users (a random selection of community people that drink the water from these wells regularly). Each of these groups is separated by gender, to indicate if there is a gender imbalance with the responses. These focus group discussions each take a day in themselves to complete.

So far a total of 32 communities have been assessed, over 120 individual wells, over 96 household surveys and 25 focus group discussions. The culmination and preliminary analysis of this data has already indicated severe problems with water supply systems. Although the failure rates corroborate with the rates suggested in previous reports, our current research has indicated that not all failure happens in the same way or for the same reasons. Additionally, not all failure offers the same degrees of risk, impact or cost of recovery. Over simplifying the type of failure of the wells by the classification of them being ‘functional’ or ‘non-functional’ either trivialises or overly dramatises the actual situation.

In the context of an emergency, such as in this case study of the immediate aftermath of the civil war in Sierra Leone, there will have been reasons why the quality of work on the water supplies will not have been up to the standard expected in international development. The availability of materials and resources, as well as the technical assistance that is locally available, would have all contributed to limit the final quality of the projects. Therefore, out of context, it is difficult to assess if these mistakes were avoidable.

This project is focussed on better understanding the type of systems which remain at the end of a post conflict scenario. There currently exists an inadequate methodology for understanding the condition of the entire water supplies systems. Furthermore it is difficult to assess what questions are important to ask, which will contribute to developing an understanding of the condition of the system as a whole.

Given the time intensive nature of our current practice, it would be unfeasible for NGOs to contribute this amount of resources, which we are currently using, towards understanding the problem. What we hope to achieve is to identify key areas and trends in the well failures, isolating them and then put them into a Case-Based Reasoning model. Case based reasoning offers the ability to quantitatively determine the significance of individual failure trends of wells.

The next blog will outline in layman’s terms the methodology behind case-based reasoning and the reasons it was selected for this research.