Post-clearance Inspection: How Much is Enough

by Russell Gasser[ GICHD ] - view pdf

Post-clearance inspection serves to check contamination of land on a per square meter basis. Although inspection does little in the way of explaining the quality of the work done in demining operations, it can be important in providing an incentive for deminers to produce higher quality work.

Post-clearance inspection for quality control has a significant impact on the overall cost of mine clearance operations. Post-clearance inspection is part of a zero sum game—spending more resources on inspection may give project managers satisfaction that the cleared land is safe, but what it really means is that resources are being diverted away from clearing more land that is still hazardous. Every dollar spent on unnecessary inspection increases the chance of an accident on land remaining to be cleared by increasing delays. For those waiting on the land to be cleared, the risk of an accident is greater than any benefit of reduced risk from spending a large amount of money on post-clearance inspection. A 2012 study showed one missed mine is found for every one million dollars spent on inspection. The net effect of high percentages of inspection will be higher costs and more casualties overall as less land is cleared.1 As part of its technical support to IMAS, the GICHD has been reviewing IMAS 07.30 (accreditation), 07.40 (monitoring) and 09.20 (post-clearance inspection); this article is based on this review.

How Much Inspection is Enough?

Figure 1. Front cover view of GICHD’s report on post-clearance inspection.
Figure courtesy of GICHD.
Figure 1. Front cover view of GICHD’s report on post-clearance inspection.
Figure courtesy of GICHD.

International Mine Action Standard (IMAS) 09.20 has played a part in creating confusion about sampling for post-clearance inspection. The complex calculations included in this standard can mislead inspectors that sampling 10 percent of the cleared area can give 90 percent confidence of finding a missed mine. In actuality, sampling 10 percent of a cleared area can give up to a 10 percent chance of finding a missed mine.2 The calculations in IMAS 09.20 are derived from an International Organization for Standardization method for sampling industrial production that is not applicable to demining. If you look carefully at the calculation in IMAS 09.20 you will see there are two important, but usually overlooked, entries in the equation that are concerned with how many errors are allowed before the batch of products is rejected.3

This makes sense when inspecting a machine’s output in a factory that is producing thousands of copies of the same item every day. In the factory, a few items that don’t reach the quality standard will be acceptable in return for cheaper or faster production. If an item from the factory isn’t good enough then it’s simply thrown away. However, there is no equivalent in demining; a square meter of land that has not been cleared properly cannot be removed by simply throwing it away. The land remains a potentially hazardous area that can kill or injure someone in the future.

Post-clearance inspection uses a measure of contamination per square meter, so a missed target is equivalent to one square meter not properly cleared. The calculations in IMAS 09.20 permit a missed item for every 300 m2 (3,230 sq ft) cleared, and also state that in post-clearance inspection, the first missed item can be ignored before the land clearance is rejected as unsatisfactory. The missed item discovered during post-clearance inspection might be no more than a small metal fragment, but it could be a missed mine. By allowing missed mines without rejecting the mine clearance work, IMAS 09.20 is non-compliant with IMAS 09.10.

Factory quality control by sampling is based on the principle that most non-conformities are due to a problem in the process, e.g., a machine that is out of adjustment or needs maintenance. In demining, individual, one-of-a-kind errors are the more common causes of missed mines.

Industrial health and safety uses four types of error to explain some important differences:

Will inspection after the vegetation grows back make any improvement to the deminer’s quality of work?Photo courtesy of Helen Gray, GICHD 2013.
Will inspection after the vegetation grows back make any improvement to the deminer’s quality of work?
Photo courtesy of Helen Gray, GICHD 2013.

Post-clearance inspection cannot help us to understand if the cause of a problem is a slip, lapse, mistake or violation. Unless the cause is known, then procedures cannot be changed to avoid repetition of the problem. Quality management needs more information than the results of post-clearance inspection.

On a large site, a deminer cannot be compared to another deminer who was working at the other end of the site months earlier, with different weather, soil and vegetation. One deminer might be suffering from a personal crisis—perhaps a family member has died, and added to his grief, he is now seriously in debt from paying for the funeral. Such a person might have a moment of inattention that leads to poor clearance. Both deminers could have been well trained and supervised or badly trained and supervised; however, this information is indiscernible from a single inspection after the end of clearance. Demining does not operate like a machine in a factory doing a repeated task.

When a square meter of land is inspected after clearance and is found not to contain any hazard, the square meter of land in question likely never had any explosive contamination. The percentage of land with explosive hazards is low in a mined area, usually well under one percent. When the inspection declares the land is free from metal particles or explosive hazard, the land may not have been contaminated but was indeed thoroughly checked to a suitably high standard. Another possibility is that the land was never contaminated and was inadequately cleared due to poor training and supervision. At worst, the square meter sampled by post-clearance inspection was never processed but overlooked by a tired deminer at the end of the day, or by an unscrupulous team leader wanting to turn in better clearance data. In this case, nothing more was done other than declare the land cleared without any clearance work. Post-clearance inspection tells us nothing to help separate these completely different outcomes. However, quality management relies on being able to tell the difference between good and bad quality work.

Is post-clearance inspection completely worthless? In terms of defining the quality of clearance on a site, it has little value. The extensive survey by the GICHD illustrates this. Independent statistical experts contributing to this review wrote in 2012 “…the optimal sampling plan is …not to perform sampling at all.”1

But post-clearance inspection can have real value and should not always be discarded. Statistical data about the quality of work aside, there can be value when inspection provides an incentive to do higher quality work. Even as a small percentage, inspection can produce an overall increase in the quality of work in repetitive tasks.4,5 This is a reverse lottery effect. In a lottery, many people will buy a ticket provided the prize is large enough even though their chances of winning are small. In sampling we have the opposite situation in which people improve their work because they feel there is a real risk of being caught, even when the sampling rate is very low. If this is linked to serious sanctions for nonconformity, then deminers, team leaders and site managers are motivated to provide higher quality. IMAS 09.20 always allowed for no postclearance inspection where quality assurance showed it was not necessary.

Once the purpose of post-clearance inspection is understood to be improving people’s motivation and attention to their work, and not as a statistical sample, it’s easy to see how it should be done.

How Much Sampling is Enough?

The key question is, of course, how much sampling is necessary?

The lottery effect works with odds of millions to one against. One percent sampling should be far more than enough to motivate a reverse lottery effect. If inspection is reduced to one percent or less, how will it be possible to provide assurance that the clearance has been done properly?

Quality assurance has always been about implementing a system of efficient and effective methods in addition to processes, training, supervision and good documentation. This is done so that mine clearance organizations can demonstrate they are competent, have the right people and tools, and complete the work at a high standard. Post-clearance inspection tells us far too little about the quality of the work done in demining operations to be useful. Effort spent on checking that a demining organization has its own internal quality management system, and that this system functions and is well documented will be far more effective and efficient in terms of the overall quality of clearance than any amount of sampling by post-clearance inspection—even 100 percent sampling. Inspection that finds nothing does not indicate that high quality clearance was done.

By the time this article appears in print the IMAS may already have been updated as this has been the topic for revision for some time by the review board. The IMAS review board will also consider a thorough overhaul of the IMAS quality standards that incorporate a new approach to post-clearance inspection. This brings post-clearance inspection into focus as one part of an overall quality management system that is based on getting things right the first time rather than on making mistakes and then trying to find them later. c

 

Biography

Ursign HofmannRussell Gasser joined the GICHD in 2013 as advisor on quality management systems and results-based management. An engineer by training with a Ph.D. in mine action technology, he became interested in landmine issues while working in Nicaragua in the 1980s where he met mine survivors at an International Red Cross prosthetics workshop.



Contact Information

Russell Gasser
Advisor, Quality Management Systems and Results Based Management
Geneva International Centre for Humanitarian Demining (GICHD)
Chemin Eugène-Rigot 2C
P.O. Box 1300
1211 Geneva / Switzerland
Tel: +41 (0)22 730 93 19
Fax: +41 (0)22 730 93 62
Email: r.gasser@gichd.org
Website: http://www.gichd.org

 

Endnotes

  1. GICHD. “A Review of External Post-Clearance Inspection in Mine Action.” (February 2012) http://bit.ly/1VaFcm4.
  2. To be more precise: the maximum confidence of 10 percent sampling is 10 percent, but would be less if the original and inspection detection methods are correlated. In an honest lottery there is no correlation between successive results. This is simplified in the main text to improve clarity.
  3. IMAS 09.20 Annex B, section B.3.1.
  4. Paragraph 2: Specified Quality Limit is 0.35% for calculations. This is the “maximum fraction of contaminated land” after clearance. Annex B, section B.3.4.
  5. In the equation: p is the maximum fraction of land still contaminated after clearance, which should be zero according to IMAS 09.10 and not 0.35%.
  6. d is the maximum number of non-conforming items found during inspection before rejecting the clearance, which should also be zero according to IMAS 09.20.
  7. “Vroom Expectancy Motication Theory.” YourCoach. Accessed 14 May 2015. http://bit.ly/1ExOXQj. The source provides an introduction to the positive reward view of Expectancy Theory, but instead of reward for a job well done there is avoidance of serious negative consequences of a job not well done.
  8. “Motivation Theory and Industrial and Organizational Psychology.” University of Minnesota 140-157. http://bit.ly/1G3ei9q.