A Review of External Post-clearance Inspection: How Cost-effective is it?

by Åsa Gilbert and Aron Larsson [ Geneva International Centre for Humanitarian Demining ] - view PDF

The Geneva International Centre for Humanitarian Demining conducted a study in early 2012 to consider the effectiveness of performing external post-clearance inspections. GICHD took into account the practices of the International Mine Action Standards, the International Organization for Standardization and the numerous mine action programs worldwide. It also worked with the Swedish company Preference Consulting to determine the mathematical probability of finding a mine/explosive remnant of war during external post-clearance inspections. Lastly, GICHD examined the normative effect of external post-clearance inspections as well as the financial cost associated with their execution.

Figure 1. Flow chart outlining the relationship between components of Quality Management in accordance with IMAS.
Figure 1. Flow chart outlining the relationship between components of Quality Management in accordance with IMAS.
All graphics courtesy of GICHD.

Substantial time, money and effort has been spent on external post-clearance inspection since it was first included in the International Mine Action Standards in 2000.1 However, a recent review of the practical implementation and findings of external post-clearance inspections illustrates that nonconformity levels are minimal, which provides cause to reflect on the activity and approach itself.

In early 2012, the Geneva International Centre for Humanitarian Demining completed a study that examined the cost and benefits of this process. Through external post-clearance inspections, the study also sought to determine the level of statistical confidence in the land being free from explosive hazards after clearance.

Quality Management in Mine Action According to IMAS

The IMAS definition of quality management is somewhat narrow when compared to the International Organization for Standardization 9000 series standard. For the purpose of the discussion, this article uses the IMAS terminology and definitions.1,2

IMAS 09.20, “The inspection of cleared land: guidelines for the use of sampling procedures,” states the following:

“The aim of demining Quality Management is to provide confidence (to the beneficiary, the demining organization and the national mine action authority) that clearance and quality requirements have been met and that cleared land is indeed safe for use. Quality Management for demining comprises three complementary components.”3

Internal QC Internal sampling of cleared areas on a daily basis by a supervisor during operator break times.
External QC Sampling conducted during an external QA visit by an external QA officer. Normally not recorded.
Post-clearance inspection by IMAS 09.20 (sampling by an external body once a site has been completed).
Table 1. Terminology related to quality control.

The three components are accreditation, monitoring and post-clearance inspection. According to IMAS 09.20, accreditation and monitoring are parts of quality assurance, while post-clearance inspection is a part of quality control. In humanitarian demining, QA confirms “that management practices and operational procedures for demining are appropriate, are applied correctly and will achieve the stated requirement in a safe, effective and efficient manner.” 2 QC relates to the inspection of a finished product; “in the case of [humanitarian] demining, the ‘product’ is [safe] cleared land.”3

The essential difference between the two is that QA ensures that the processes for demining are appropriately applied, while QC ensures that the product, i.e., the cleared land, is indeed free from mines and explosive remnants of war hazards to a specified depth. QA takes place prior to and during survey and clearance operations, while external QC generally takes place once an operator completes an agreed clearance task and leaves the site. IMAS uses the term post-clearance inspection interchangeably with external QC.

Figure 2. Example of a sampling plan for external postclearance inspection. Each orange box represents a sample where deminers manually re-cleared the ground
Figure 2. Example of a sampling plan for external post-clearance inspection. Each orange box represents a sample where deminers manually re-cleared the ground.

Usually independent monitoring organizations, tasked by national mine action authorities, implement external post-clearance inspections. However, these inspections are sometimes conducted through a contractual arrangement or by the national mine action authority itself. The inspection involves re-clearance (also known as sampling) of a certain percentage of an area that an operator already cleared. Deminers typically carry this out manually. The actual sampling occurs once an area is completed but before the land is officially handed over to the end user.

External post-clearance inspection intends to ensure that the quality of work reaches an agreed standard. In mine action, the quality standard is that the area in question is free from mines and ERW to a pre-determined depth.

Internal QC is another vital component of any organization’s internal quality management system and involves post-clearance inspection performed by section/ team leaders or supervisors during a break or at the end of a working day. However, this article does not cover these internal QC activities.

Implementation of External Post-clearance Inspection

The International Campaign to Ban Landmines reports that there are approximately 50 active mine action programs throughout the world. Research has shown that 14 programs implement post-clearance inspection, more than one-quarter of all active mine action programs. The majority of countries with humanitarian mine action programs do not implement external QC processes; instead, they employ a stringent QA approach that includes accreditation of operators prior to deployment and regular inspections of survey and clearance teams during operations.

History of QA/QC in Mine Action

Prior to 2000, no globally accepted standards were in place to measure the quality of land considered safe through survey and clearance. Nor were any agreed approaches in place to measure the appropriateness, efficiency or effectiveness of employed survey and clearance methodologies. In 2000, steps were taken to incorporate a number of the standards that focused on quality within ISO; these were later applied to the mine action sector.

At the time, ISO standards were evolving as production industries moved from a focus on QC to QA. By 2000, lessons learned from the production industries indicated that a more balanced approach to quality management was required and that QA and QC should be given a more equal status.

In 2003, IMAS 09.20 was incorporated into IMAS. IMAS 09.20 is based on ISO 2859, the standard developed for production-line processes, such as the manufacturing of car parts.4 The application of ISO 2859 assumes that the product (in the case of demining, cleared land) is homogeneous; i.e., the product is uniform and has the same composition throughout.

Theoretical Review of IMAS 09.20

In the GICHD study, the analysis of the mathematical probability of finding a mine/ERW that was previously missed was carried out by Preference Consulting—a Swedish company with strong links to the Kungliga Tekniska Högskolan (KTH Royal Institute of Technology)—Stockholm University and Mid Sweden University.5

Preference Consulting determined that sampling plans based on ISO 2859 are not optimal for all applications and may be costly. A quantitative investigation of the sampling plans specified in IMAS indicated that sampling has a relatively high cost when the lot size is small and the quality of the clearance is high but provides only a marginal increase of confidence in clearance. Therefore, reducing the level of sampling, or in some programs, ceasing external post-clearance inspection altogether may be beneficial.

In its report, Preference Consulting showed that the quality of mine clearance maintained by the Bosnia-Herzegovina Mine Action Center was far better than that sought by IMAS, because fewer nonconformities were found in actuality than were expected and would typically be acceptable for postclearance inspection.4,5 As a result, any actual increase in confidence provided by sampling tended to be quite low, as the confidence in the quality of the cleared land prior to sampling was already very high.

Preference Consulting states that when conforming to IMAS procedures for the inspection of a cleared area, in the event that a lot from this area fails inspection, “… the corresponding optimal sampling plan is … not to perform sampling at all.”5

Normative Effect

All of the countries included in the case studies wished to continue external QC in one form or another. The given reasons rarely involved increasing confidence in the quality of the cleared land, instead focusing on external QC’s normative effects on demining organizations. An operator who knows that a cleared area will be inspected post-clearance has more motivation to complete the task in a satisfactory manner as opposed to a program where no external postclearance inspections are applied. This is known as the normative effect.

  Number of Mines/ERW or critical non-conformaties found during external QC m2sampled during external QC 2010 Estimated cost USD (million) Critical non-conformities/mines found
Country A 8 mines since 2004
635,000
1.06
1.1
Country B 2 critical non-conformities in 2010 and 5 in 2009
500,000
1.10
2
Country C last missing mine in 2003
2,640,000
2.40
0
Country E 0 mines since the beginning of sampling in 2006
3,260,000
2.97
0
Country F 3 mines since the beginning of sampling in 2006
14,000
0.06
0.5
Total   7.05 million m2 USD 7.59 million 3.7 non-conformities/
mines
Table 2. Statistical Summaries of Case Studies for 2010 | Summary of sampling statistics from five of the six case studies for 2010. Country D was excluded since data was not available for this period. To calculate the cost of sampling during 2010, an estimate of 0.91 USD per sq m (a comparatively low price) was used for countries C and E for which a specific value was not available. The number of critical nonconformities or mines found in 2010 included in the table are inferred from available data for countries A and F and represented as an average.

During 2010 a total of 7.05 million sq m was sampled at a cost of USD 7.59 million. Over this 12 month period 3.7 mines or critical nonconformities were discovered during the sampling activities with an average of 1.90 million sq m of land re-cleared for each mine or nonconformity found. On average USD 2.05 million were spent per mine or critical nonconformity. No data was available on whether mines were functional when discovered or whether the critical nonconformities (which contribute to more than 50 percent of the data) were missed mines/ERW or a further item such as a large piece of metal.

Despite this, the majority of mine action programs choose not to use external QC and instead apply a rigorous QA approach. When processes are appropriate and carried out in accordance with accredited standard operating procedures, the quality of the cleared land naturally follows.

External QC teams sample millions of square meters of cleared land, but they find very few missed mines/unexploded ordnance. An operator with a more stringent QA process could have potentially detected the majority of those found. Others were missed because the initial threat assessment was not conducted correctly.

Limited official statistics exist on the number of accidents that occur on land that has been cleared and handed back to the local population. However, the general impression within the mine action sector is that the quality of land cleared in mine action programs without external quality control is lower than that of mine action programs that employ external QC.

Summary of conclusions and recommendations of the GICHD report:

• External QC comes at a high cost but provides a negligible increase in confidence that a cleared area is in fact free from explosive hazards.
• The ISO-2859 may not be the optimal reference standard for mine action.
• Alternative sampling methodologies better suited for mine action should be explored, such as targeting of high risk areas opposed to random sampling.
• By emphasizing QA rather than QC, quality throughout the clearance process is ensured with little or no need for QC.
• Focus should be on the quality of management of the survey process rather than the clearance process.
• In certain situations, mine action programs may benefit from applying external QC (i.e., in areas cleared by new operators/teams or by operators with a poor track record.) Post-clearance sampling should therefore remain an option for IMAS.

Cost

The additional costs of external QC are difficult to accurately quantify. However, in two of the seven case-study countries, the annual costs were conservatively estimated at more than US$1 million per country. This was based on the average cost of clearing one square meter of ground, multiplied by the number of square meters sampled.

In 2010 the findings of the study show that 7.05 million sq m (272 sq mi) of previously cleared land in five countries was subject to re-clearance during external QC. This cost $7.59 million, yet only four mines were found, illustrating why the costs and the added value of external QC should be reconsidered. In addition to cost, the extra time and necessary resources should also be taken into account as clearance assets are diverted away from clearance activities. During field visits, external QC often added up to 10 or more days to the overall duration of a task.

Based on the discussions held during case studies and with other stakeholders, evidence suggests that the general understanding of quality management and the systems involved are limited within the mine action sector.

IMAS and most national mine action standards provide only a fairly narrow description of the issue. This implies that the field is missing opportunities to achieve efficiency and effectiveness, as well as to learn from and improve upon past experiences.

Based on findings from the case studies (see Table 2) and the work done by Preference Consulting, the general quality of the majority of cleared areas appears to be high, and sampling provides little additional confidence as to whether a particular area is free from explosive hazards. c

The GICHD study’s research and report was made possible through the generous support of the government of Switzerland and through additional contributions from the governments of Norway, Sweden and Australia. The complete version of the study report is available at GICHD’s website: http://www.gichd.org/publications/a-review-of-external-post-clearance-inspection-in-mine-action-en.

Biographies

Asa GilbertÅsa Gilbert held the position of land release project manager at the Geneva International Centre for Humanitarian Demining and now works as operations manager for the Norwegian Peoples Aid Myanmar Mine Action Program. She previously worked as a quality assurance officer at the Mine Action Coordination Centre in South Lebanon with the United Nations Office for Project Services, an explosive ordnance disposal coordinator at the Regional Mine Action Coordination Centre in Sudan, employed by the Swedish Rescue Services Agency, and as an instructor at the Swedish Armed Forces EOD and De-mining Centre. She holds a Bachelor of Science in mechanical engineering from University of Skovde (Sweden).

Aron LarssonAron Larsson is a researcher at Stockholm University and Mid Sweden University. He has a doctorate in computer and systems science and a Master of Business Administration, both from Mid Sweden University. Larsson has worked on the development and application of methods and tools for evaluation of decisions under multiple objectives, uncertainties and risks. He has been developing the DecideIT decision tool and has experience applying decision theories in public and private sectors, such as environmental planning and investment decision-making.

 

Contact Information

Åsa Gilbert
Operations Manager
Norwegian Peoples Aid
24 (L) Natmauk Avenue,
Kohine Bahan Tsp
Yangon / Burma
Tel: +951 540 694 ext. 202
Email: asag@npaid.org
Website: http://gichd.org

Aron Larsson, Ph.D.
Associate Professor, Dept. of Computer
and Systems Sciences
Stockholm University
SE-16440 Kista / Sweden
Risk and Crisis Research Center
Mid Sweden University
SE-85170 Sundsvall / Sweden
Tel: +46 60 148616
Email: aron.larsson@miun.se

Endnotes

  1. “International Mine Action Standards.” GICHD. http://www.gichd.org/publications/a-guide-to-the-international-mine-action-standards-en. Accessed 16 August 2012.
  2. ISO 9000 provides a management discipline that encourages an organization to deliver products or services that meet agreed requirements. “Guide for the Management of Demining Operations.” IMAS 07.10. First Edition, 1 October 2001. http://www.mineactionstandards.org/fileadmin/user_upload/MAS/documents/imas-international-standards/english/series-07/IMAS-07-10-Ed1.pdf. Accessed 17 August 2012.
  3. “The Inspection of Cleared Land: Guidelines for the Use of Sampling Procedures.” IMAS 09.20, Second Edition, 1 January 2003. http://www.mineactionstandards.org/fileadmin/user_upload/MAS/documents/imas-international-standards/english/series-09/IMAS-09-20-Ed2-Am4.pdf. Accessed 17 August 2012.
  4. “ISO 2859-10:2006: Sampling Procedures for Inspection by Attributes—Part 10: Introduction to the ISO 2859 Series of Standards for Sampling for Inspection by Attributes.” ISO. http://www.iso.org/iso/catalogue_detail.htm?csnumber=39991.
  5. Internal report on The Use of ISO 2859 Sampling by Attributes as Quality Control of Cleared Land in Humanitarian Demining, Preference Consulting, January 2012.