Yoshi Shimizui/International Federation, Sierra Leone

Chapter 7 - summary
Measuring disasters: challenges, opportunities and ethics

How many people are killed or affected by disasters globally every year? Where and when do disasters occur? What causes the casualties? These questions appear simple, yet the answers are vitally important for informed decision-making. Humanitarian aid tends to follow in the wake of high-profile conflicts. Less reported or less strategically significant crises attract less aid. High-quality data on all disasters – especially war and famine – are lacking. Without it, thousands of victims die before humanitarian organizations even register their need. Inaccurate data can result in flawed decision-making that may cost lives or squander valuable resources. And without accurate information on global needs, no one can judge whether humanitarian spending is really impartial.

Several global databases exist. The emergencies database (EM-DAT) of the Centre for Research on the Epidemiology of Disasters, based in Belgium, has collected and analysed disaster data since 1988. Other databases are operated by reinsurance companies Swiss Re and Munich Re, regional groups (e.g. DesInventar in Latin America) and academic centres. But they are not inter-connected and comparisons are difficult. Linking global and local disaster information systems has proved difficult, as they define, collect and use data in different ways.

Some domestic databases include any event, however small, that results in death or damage. EM-DAT classifies an event as a disaster if at least “ten people are killed and/or 100 or more are affected and/or an appeal for international assistance is made or a state of emergency declared”. This definition catches significant events while avoiding information overload. Reinsurance databases concentrate on insurable events and economic damage and are biased towards natural disasters rather than complex emergencies, so their loss estimates often bear little relation to levels of humanitarian need. However, most databases, including EM-DAT, miss significant human suffering due to conflict, famine and disease.

Widely differing assessment methodologies are also in use. There is no agreement over how to define who is 'affected' or how many households need assessing to gain a reliable overview. Agencies’ results cannot be compared because methods and definitions are not standardized. And since methods are often not evaluated, it is hard to judge the quality of the data generated. Problems occur in the interpretation of data too, especially in chaotic and highly politicized contexts, when pre-disaster baseline data is lacking.

EM-DAT uses data from various sources. Following a disaster, needs assessments conducted by governmental or humanitarian teams provide primary data on specific problems. This data is often collated by the UN or Red Cross Red Crescent into consolidated country or regional reports (secondary data), which is prioritized by EM-DAT as it provides an independent overview. Sometimes tertiary data (e.g. media reports) is used if there is no other source.

The key to good data gathering lies in gaining access to those in need. But wars and disaster zones are often too difficult or dangerous to visit. And unpredictable population movements make it even harder to get accurate data. Most victims die away from relief centres and go unreported – even in refugee camps. In Bangladesh in the 1990s, deaths in refugee camps were recorded using both 'passive' surveillance (deaths reported to camp staff) and 'active' surveillance (counting graves, interviewing relatives). Just before a major health crisis, passive surveillance data suggested that mortality was decreasing, while active surveillance showed it was actually three times higher and increasing. If decisions had been based only on passive surveillance, the situation would have been catastrophically underestimated. Active surveillance prompted urgent action and proved that accurate and timely data can save lives.

For complex emergencies, estimates of death rates are often based on random retrospective mortality surveys, or on active surveillance in selected locations. These localized rates are then multiplied up to estimate deaths across a wider area. One series of mortality studies estimated that 3.3 million people had died due to war in the Democratic Republic of the Congo (DRC), from 1998-2002. Of these deaths, 86 per cent were caused by communicable diseases and malnutrition.

Such calculations are often controversial, as they are based on assumptions that similar mortality conditions to the survey site apply across wider areas, or that pre-disaster population and mortality statistics are accurate. Yet such baseline data is often non-existent during conflicts. Nevertheless, it is unethical to ignore data from complex emergencies and focus only on data from safer, more accessible areas. Equally, it is wrong to discount data just because it brings unwelcome news. There is a moral imperative for humanitarian agencies to investigate precisely those areas where data is incomplete but points to a major, hidden catastrophe.

The DRC mortality surveys suggest that deaths from war-related disease and malnutrition in that country alone far exceed the total of all deaths from 'natural' disasters during the past decade. According to the World Health Organization, communicable diseases claimed 13.3 million lives worldwide in 1998. So should malnutrition and disease be classified as disasters? The HIV/AIDS epidemic is certainly a disaster. In Kenya, AIDS deaths are equivalent to two 747 jets crashing every day. But disaster databases rarely include HIV/AIDS data. For the global humanitarian system to respond to all suffering according to need alone, reliable mortality data of all types is essential. Databases should develop a new category – complex emergency – which would combine mortality data from war, violence, hunger and disease.

Complex emergencies raise particular problems, such as how to define who is 'affected' and by what. Malawi, caught in a cycle of floods, drought, food insecurity and epidemics since 1994, is one example. If we record only the primary events (flood and drought), we risk underestimating the total impact. While very few died from the floods, we don’t know how many suffered or died from secondary impacts (hunger, malnutrition and epidemics). These 'casualties' are either not accurately reported or completely missed. It is often easier to extract data on single events like earthquakes. In complex emergencies, however, attributing numbers of killed or affected to particular causes becomes virtually impossible.

Collecting and using disaster data poses serious ethical challenges. At the height of a disaster – when humanitarian needs are urgent – should precious time and resources be spent gathering data or saving lives? Some argue it is unethical to delay life-saving responses until data has been gathered. Others argue that aid should be based on objective assessments of need

Some disasters – notably in Africa – are too dangerous or remote to raise sufficient international interest. Very little aid means that few relief workers are active in the region. That means data is patchy or non-existent. Without reliable data, appeals cannot be launched, awareness cannot be raised and aid does not arrive. This creates a vicious spiral of suffering which may go unnoticed.

Gathering data for advocacy leads to dilemmas if those in power do not like the message. In Uganda in the 1980s and Bangladesh in the 1990s, data collectors were arrested, jailed and beaten for uncovering unwanted news – e.g. data on atrocities or high death rates. In such situations, aid agencies have to judge whether to use their data to speak out and risk expulsion, or remain silent and be accused of colluding with the perpetrators.

In conflict zones, aid agencies may be working alongside military forces, raising issues of impartiality and neutrality. Any suspicion that information, provided by aid agencies, was being used for military purposes would jeopardize humanitarians' security and credibility.

Data collection – especially in wars and famines – needs much more research and investment if it is to improve. Recommendations include:

• standardize definitions and collection systems, to enable direct comparisons;
• improve proactive data collection in 'forgotten disasters'; and
• create new data categories (e.g. 'complex emergency') to capture the combined effects of war, malnutrition and disease

Another major challenge is to prevent data being covertly manipulated for political, military or commercial purposes. This could be addressed by developing an international code of ethical data collection and use, with detailed standards, guidelines and tools along the Sphere model.

High-quality information gathering is the nervous system of the humanitarian enterprise. Without it, any form of principled action - whether now or in the future - is paralysed.

Patricia Diskett, director of the Centre for Public Health in Humanitarian Assistance, Uppsala University, Sweden, contributed this chapter and box