The performance quantification issue is often overlooked, both by system managers and by researchers dealing with warning models for landslide early warning systems (LEWSs). For instance, the main focus of researchers dealing with warning systems for rainfall-induced landslides at regional scale, which are typically based on empirical rainfall thresholds (Guzzetti et al., 2007 and references therein), is on improving the correlation between rainfall indicators and landslides. However, literature studies rarely back analyze the relationship between warnings, which would have been issued adopting those correlations, and landslides. Especially for LEWSs operating at regional scale (ReLEWSs), empirical evaluations are often carried out by simply analyzing the time frames during which significant high-consequence landslides occurred in the test area (Keefer et al., 1987; Baum and Godt, 2010; Capparelli and Tiranti, 2010; Aleotti, 2004). Alternatively, the performance evaluation is based on 2 by 2 contingency tables computed for the joint frequency distribution of landslides and alerts, both considered as dichotomous variables (Yu et al., 2003; Cheung et al., 2006; Godt et al., 2006; Restrepo et al., 2008; Tiranti and Rabuffetti, 2010; Kirschbaum et al., 2012; Martelloni et al., 2012; Peres and Cancelliere, 2012; Staley et al., 2013; Lagomarsino et al., 2013, 2015; Greco et al., 2013; Segoni et al., 2014; Gariano et al., 2015; Stähli et al., 2015). The four elements of these tables – i.e., correct alerts (CAs) or true positives; missed alerts, false negatives or type II errors; false alerts, false positives or type I errors; true negatives (TNs) – are then used to assess the weight of the correct predictions in relation to the model errors by means of a series of statistical indicators of the model performance. In all these cases, however, model performance is assessed, neglecting some important aspects that are peculiar to ReLEWSs, such as the possible occurrence of multiple landslides in the warning zone, the duration of the warnings in relation to the time of occurrence of the landslides, the level of the issued warning in relation to the landslide spatial density in the warning zone and the relative importance system managers attribute to different types of errors.
Excerpt from
Calvello M, Piciullo L (2016).
Assessing the performance of regional landslide early warning models: the EDuMaP method. Natural Hazards and Earth System Science, 16:103–122.
http://www.nat-hazards-earth-syst-sci.net/16/103/2016/
References
Aleotti P (2004). A warning system for rainfall-induced shallow failures, Eng. Geol., 73, 247–265.
Baum RL, Godt JW (2010). Early warning of rainfall-induced shallow landslides and debris flows in the USA, Landslides, 7, 259–272.
Capparelli G, Tiranti D (2010). Application of the MoniFLaIR early warning system for rainfall induced landslides in Piedmont region (Italy), Landslides, 7, 401–410.
Cheung PY, Wong MC, Yeung HY (2006). Application of rainstorm nowcast to real-time warning of landslide hazards in Hong Kong, in: WMO PWS, Workshop on Warnings of Real-Time Hazards by Using Nowcasting Technology, 9–13 October 2006, Sydney, Australia.
Gariano SL, Brunetti MT, Iovine G, Melillo M, Peruccacci S, Terranova O, Vennari C, Guzzetti F (2015). Calibration and validation of rainfall thresholds for shallow landslide forecasting in Sicily, southern Italy, Geomorphology, 228, 653–665.
Godt JW, Baum RL, Chleborad AF (2006). Rainfall characteristics for shallow landsliding in Seattle,Washington, USA, Earth Surf. Proc. Land., 31, 97–110.
Greco R, Giorgio M, Capparelli G, Versace P (2013). Early warning of rainfall-induced landslides based on empirical mobility function predictor, Eng. Geol., 153, 68–79.
Guzzetti F, Peruccacci S, Rossi M, Stark CP (2007). Rainfall thresholds for the initiation of landslides in central and southern Europe, Meteorol. Atmos. Phys., 98, 239–67.
Keefer DK, Wilson RC, Mark RK, Brabb EE, Brown III WM, Ellen SD, Harp EL, Wieczoreck GF, Alger CS, Zatkin RS (1987). Real-time landslide warning during heavy rainfall, Science, 238, 921–926.
Kirschbaum DB, Adler R, Hong Y, Kumar S, Peters-Lidard C, Lerner-Lam A (2012). Advances in landslide nowcasting: evaluation of a global and regional modeling approach, Environ. Earth Sci., 66, 1683–1696.
Lagomarsino D, Segoni S, Fanti R, Catani F (2013). Updating and tuning a regional-scale landslide early warning system, Landslides, 10, 91–97.
Lagomarsino D, Segoni S, Rosi A, Rossi G, Battistini A, Catani F, Casagli N (2015). Quantitative comparison between two different methodologies to define rainfall thresholds for landslide forecasting, Nat. Hazards Earth Syst. Sci., 15, 2413–2423.
Martelloni G, Segoni S, Fanti R, Catani F (2012). Rainfall thresholds for the forecasting of landslide occurrence at regional scale, Landslides, 9, 485–495.
Peres DJ, Cancelliere A (2012). Development of rainfall thresholds for landslide early warning in Sicily: a method based on the use of rainfall annual maxima series, Rend. Online Soc. Geol. It., 21, 580–582.
Restrepo P, Jorgensen DP, Cannon SH, Costa J, Laber J, Major J, Martner B, Purpura J, Werner K (2008). Joint NOAA/NWS/USGS prototype debris flow warning system for recently burned areas in southern California, B. Am. Meteorol. Soc., 89, 1845.
Segoni S, Rossi G, Rosi A, Catani F (2014). Landslides triggered by rainfall: a semiautomated procedure to define consistent intensity-duration thresholds, Comput. Geosci., 3063, 123–131.
Stähli M, Sättele M, Huggel C, McArdell BW, Lehmann P, Van Herwijnen A, Berne A, Schleiss M, Ferrari A, Kos A, Or D, Springman SM (2015). Monitoring and prediction in early warning systems for rapid mass movements, Nat. Hazards Earth Syst. Sci., 15, 905–917.
Staley DM, Kean JW, Cannon SH, Schmidt KM, Laber JL (2013). Objective definition of rainfall intensity–duration thresholds for the initiation of post-fire debris flows in southern California, Landslides, 10, 547–562.
Tiranti D, Rabuffetti D (2010). Estimation of rainfall thresholds triggering shallow landslides for an operational warning system implementation, Landslides, 7, 471–481.
Yu YF, Lam JS, Siu CK (2003). Interim Report on Review of Landslip Warning Criteria, Special Projects Division, Special Project Report 4/2003, Geotechnical Engineering Office, Civil Engineering Department, SAR Government, Hong Kong, 20 pp.
Landslide Risk 