Metal Detector Pinpointing Accuracy Under Field Conditions

by Kazunori Takahashi [ Leibniz Institute for Applied Geophysics ] - view pdf

As ordnance and landmine-detection technology advances, mine-action organizations across the world are increasingly using more sophisticated types of metal detectors. Each metal detector contains its own strengths and weaknesses, and until now, no accurate way exists to quantify the differences between the models. In this article, a method is shown to successfully evaluate metal-detector accuracy in a controlled field condition and provides data on the differences between single coil, double-D coil and other metal-detector types. The International Test and Evaluation Program for Humanitarian Demining conducted the 2009 evaluation in Germany that provides the data used in this article.1

Metal detectors are commonly used to detect landmines and metal pieces during clearance operations. Because of the dangerous nature of clearance, metal detectors must accurately pinpoint targets to make excavations and removal as safe and precise as possible. Therefore, detection probability and location-accuracy performance must be tested to ensure proper performance. Previously, tests such as the Systematic Test & Evaluation of Metal Detectors Laboratory Test by the Joint Research Centre, European Commission2 evaluated these criteria. In the field, however, an operator does not know if or where a target exists, and the accuracy will differ from laboratory tests.3 Subsequently, in the test described here, the purpose was finding a specific target in order to understand the detector’s accuracy, not simply discovering if a target existed.

Figure 1: Schematic illustration for calculating the location error.
(Click image to enlarge)
Figure 1: Schematic illustration for calculating the location error.
Table 1: Detectors tested in the ITEP 2009 test. The technical data were compiled from the Geneva International Centre for Humanitarian Demining detector catalogue<sup>4</sup> 2009 and the Ebinger<sup>5</sup> website.
(Click image to enlarge)
Table 1: Detectors tested in the ITEP 2009 test. The technical data were compiled from the Geneva International Centre for Humanitarian Demining detector catalogue4 2009 and the Ebinger5 website.
Figure 2. Examples of the location-error distributions calculated for (a) Vallon VMH3CS, (b) Vallon VMC1, (c) CEIA MIL-D1, (d) Ebinger 422GC, (e) Minelab F3S and (f) Advanced Landmine Detection System. The histograms show the actual occurrences of the location error. The blue curves and circles show the modeled probability density functions of the location errors and its mode that indicates the most frequently occurred location error. The red curves and circles show the modeled cumulative density functions of the location errors and 95th percentile indicating the location errors that include 95% of detections.
(Click image to enlarge)
Figure 2. Examples of the location-error distributions calculated for (a) Vallon VMH3CS, (b) Vallon VMC1, (c) CEIA MIL-D1, (d) Ebinger 422GC, (e) Minelab F3S and (f) Advanced Landmine Detection System. The histograms show the actual occurrences of the location error. The blue curves and circles show the modeled probability density functions of the location errors and its mode that indicates the most frequently occurred location error. The red curves and circles show the modeled cumulative density functions of the location errors and 95th percentile indicating the location errors that include 95% of detections.
Figure 3. Mode (blue circles) and 95th percentile (red circles) of location error for each metal-detector model and operator. A lower location error shows a more accurate pinpointing.
(Click image to enlarge)
Figure 3. Mode (blue circles) and 95th percentile (red circles) of location error for each metal-detector model and operator. A lower location error shows a more accurate pinpointing.

Pinpointing Error and Analysis

Pinpointing Error and Analysis A target is located at a position (x0, y0) and is detected at (x0’, y0’) as shown in Figure 1. The pinpointing error is the distance between the true and detected positions which is calculated as noted in Equation 1 (below).
equation 1

Assuming that the location errors in x and y (Δx and Δy) are uncorrelated and normally distributed, the location error d is characterized by a Rayleigh distribution whose probability density function is given as shown in Equation 2 (below).
equation 2

The parameter σ denotes the mode of distribution, which exhibits the location error that most frequently occurred. By estimating the parameter, the pinpoint accuracy of metal detectors can be evaluated. Note that in this article the word mode is used only for the statistical term mode, which indicates a random variable that happens most frequently (i.e., random variable where the histogram or probability function is the highest). The word mode is also commonly used to describe the way metal detectors work: static or dynamic. This use of the word mode is referred to as either static mode or dynamic mode3 in this article.

Data Analysis

The above-mentioned analysis was applied to the data obtained in the ITEP 2009 test.1 The tested detectors are listed in Table 1. In the test, several types of targets were used. The burial locations of all targets were measured at the center of outer casings with the expected accuracy of 1–2 cm. Targets containing multiple metal parts, or holding a metal part off the center, were not tested, as they could skew results and were not suitable for these tests. Targets used in the analysis include only bullets and calibration targets containing a relatively small metal piece at a known location.

All the detectors with various metal-detector models were observed and the location errors d were calculated for detections that were within 10 cm from the selected targets. Figure 2 shows the histogram of the location error d for each metal-detector model. From the histograms with N random variables, the parameter σ in Equation 2 was estimated by the maximum likelihood estimate given as Equation 3 (below).
equation 3

In Figure 2, the modeled Rayleigh probability density functions are plotted with blue curves, and the estimated modes σ^ are plotted with blue dots. The curves were well fitted to the histograms. Further, the cumulative density functions were also calculated as Equation 4 (below).equation 4

Table 2. Location errors as mode (error that most frequently occurred), 95th percentile (error that includes 95% of detections) and percentage of detections that fall within 5-cm radius, obtained from the data of the ITEP Test 2009. The results were averaged over different operators for each metal detector model. The label “mfr.” means being operated by personnel from the manufacturer.
(Click image to enlarge)
Table 2. Location errors as mode (error that most frequently occurred), 95th percentile (error that includes 95% of detections) and percentage of detections that fall within 5-cm radius, obtained from the data of the ITEP Test 2009. The results were averaged over different operators for each metal detector model. The label “mfr.” means being operated by personnel from the manufacturer.
Figure 4. Percentage of detections that fall within 5-cm radius. A higher percentage shows a more accurate pinpointing.
(Click image to enlarge)
Figure 4. Percentage of detections that fall within 5-cm radius. A higher percentage shows a more accurate pinpointing.
Figure 5. Pinpointing a target with (a) single-receive-coil detectors and (b) double-D coil detectors (reproduced from the STEMD Lab Test report and Metal Detector Handbook).<sup>2,3</sup>
(Click image to enlarge)
Figure 5. Pinpointing a target with (a) single-receive-coil detectors and (b) double-D coil detectors (reproduced from the STEMD Lab Test report and Metal Detector Handbook).2,3
Figure 6. Schematic illustration of a metal-detector search head and its sensitivity profile. A circular coil detector has axes a = b, thus the axes of the sensitivity profile ellipses in the fore-aft, and transverse directions are assumed to be the same (i.e.,  a<sup>f</sup> = a<sup>t</sup>). For an oval-shaped coil detector, the ratio of the sensitivity-profile widths is assumed to be the same as that of the coil widths (i.e., a/b = a<sup>f</sup>/a<sup>t</sup>).
(Click image to enlarge)
Figure 6. Schematic illustration of a metal-detector search head and its sensitivity profile. A circular coil detector has axes a = b, thus the axes of the sensitivity profile ellipses in the fore-aft, and transverse directions are assumed to be the same (i.e., af = at). For an oval-shaped coil detector, the ratio of the sensitivity-profile widths is assumed to be the same as that of the coil widths (i.e., a/b = af/at).
Figure 7. Estimated sensitivity profiles in the fore-aft direction (solid lines) and transverse direction (dashed lines) for a target equivalent to a 10-mm 100Cr6 ball. The profiles were obtained by fitting ellipses to the STEMD Lab Test data and interpolation. Since all the detectors except Vallon VMH3 have circular search heads, their sensitivity profiles in transverse direction are assumed to be the same as those in fore-aft direction.
(Click image to enlarge)
Figure 7. Estimated sensitivity profiles in the fore-aft direction (solid lines) and transverse direction (dashed lines) for a target equivalent to a 10-mm 100Cr6 ball. The profiles were obtained by fitting ellipses to the STEMD Lab Test data and interpolation. Since all the detectors except Vallon VMH3 have circular search heads, their sensitivity profiles in transverse direction are assumed to be the same as those in fore-aft direction.
Figure 8. Location error (obtained from the ITEP 2009 test) as a function of the sensitivity-profile width (obtained from the STEMD Lab Test). The width is defined as twice the axis length of the sensitivity profile in the transverse direction. The circles and dots indicate mode and 95th percentile, respectively. The dashed lines are the linear data regressions for the single-coil detectors.
(Click image to enlarge)
Figure 8. Location error (obtained from the ITEP 2009 test) as a function of the sensitivity-profile width (obtained from the STEMD Lab Test). The width is defined as twice the axis length of the sensitivity profile in the transverse direction. The circles and dots indicate mode and 95th percentile, respectively. The dashed lines are the linear data regressions for the single-coil detectors.

These functions are plotted with red curves in Figure 2. The red dots show 95th percentiles that indicate location errors containing 95% of detections obtained by Equation 5 (below).

equation 5

Here, q is the quantile to calculate, which is 0.95 in this case. Figure 3 shows the modes and 95th percentiles estimated for each metal-detector model and operator, and Table 2 shows values averaged over all operators for each metal-detector model. Moreover, the percentage of detections within 5cm radius were obtained by setting d = 5cm in Equation 4 and is shown in Figure 4.

Discussion

Prior to the discussion on the results, note that CEIA MIL-D1 is the only detector in this analysis using a double-D coil configuration and requires a different way to pinpoint a target as shown in Figure 5. With single-receiver-coil detectors (all the detectors other than CEIA MIL-D1 in this experiment), an operator tries to find signalstart positions from different sides, and the center of the area indicates the target’s location. With double-D detectors, an operator tries to define lines where the signal tone changes from two or more sides, and the intersection indicates the target location. Thus, double-D coil detectors can indicate the location directly and more accurately. Only two major manufacturers produce metal detectors with the double-D coil configuration for demining purposes: CEIA and Foerster.

The difference in mode (errors most frequently occurred and blue circles in Figure 3 among detector models is not large; they are all in the 2–3 cm range. However, the differences in 95th percentiles (error that includes 95% of detections and red circles in Figure 3 and the percentage of detections within a 5cm radius (circles in Figure 4) are relatively clear. The main cause of the different pinpointing accuracies is the sensitivity-profile size, also known as the footprint, which is a three-dimensional area below the search head where a metal detector gives an alarm for a certain target. As depicted in Figure 5(a), the center of the area can be estimated more accurately if the perimeter where the metal detector signal starts is closer to the target location. A detector with a smaller sensitivity profile gives the perimeter closer to the target location.

To observe the pinpointing accuracy in relation to the sensitivity profile, the STEMD Lab Test data2 was analyzed. The data was measured for various targets, making it impossible to directly compare between different detector models. Therefore, the data was further processed as follows. Figure 6 shows that the data measured various sizes of 100Cr6 balls6 fit to ellipses, and that their major and minor axes (af, bf) indicate widths of estimated sensitivity profiles. The axes were interpolated for each detector model to obtain those for a target equivalent to 10mm 100Cr6 balls, so that different models can directly be compared. Figure 7 shows the obtained sensitivity profiles. The profiles (solid lines) are in the fore-aft direction, which is assumed the same in the other direction (i.e., af=at ) for CEIA MIL-D1, Ebinger 421GC and Minelab F3 because of their circular-shaped coils (i.e., a = b). Vallon VMH3 has oval-shaped coils and the profile in the transverse direction is narrower than that in the fore-aft direction (i.e., a > b and af > at). The detector’s operators usually turn the search head and always scan in the transverse direction to define the lines for pinpointing. The profile in the transverse direction (red ellipse in Figure 6) therefore needs consideration; it was calculated by the ratio of the search head’s length and width (i.e., assuming that the length-to-width ratio for the search head is the same as that for the footprint, a/b = af/at). The obtained sensitivity profile of Vallon VMH3 in the transverse direction is shown with the blue dashed line in Figure 7.

The indications of the pinpoint accuracy (modes and 95th percentiles) obtained from the ITEP 2009 test are plotted as a function of the sensitivity profile width, defined as twice the axis lengths of ellipses in the transverse direction (i.e., 2at) in Figure 8. Only four metal detector models are available in both tests that can be compared. Although such a small number of data is available, a relationship between the sensitivity width and pinpointing accuracy can be observed as a linear correlation. A detector having a smaller sensitivity profile can seemingly achieve higher accuracy, and a detector with a larger sensitivity profile seems less accurate. This observation confirms the source of the location error. Single-coil detectors pinpointing a target in the way, shown in Figure 5(a), do not indicate the target location directly. The operators estimate it from the perimeter where the detector signal starts. Therefore, the distance from the perimeter to the target location can cause errors. The linear regressions showing the correlation between the sensitivity-profile width and pinpoint accuracy for single-coil detectors are plotted in Figure 8 (the regressions do not include the data of CEIA MIL-D1).

Double-D coil detectors (e.g., CEIA MIL-D1) may not have a direct relationship between the sensitivity-profile width and accuracy. This detector type is considered to be capable of locating a target very accurately, as the STEMD Lab Test demonstrated.2 However, more training or more experience may be required to achieve such a high pinpointing accuracy as the comparison between CEIA MIL-D1 and ALIS indicates (operators who had a two-day training prior to the test used CEIA MIL-D1, while ALIS operators have occasionally used the detector for a longer period). The operators of CEIA MIL-D1 in the ITEP 2009 test might not have enough working experience on the detector to demonstrate the high accuracy in a field condition and to achieve a similar accuracy level as with the other detectors.

The detectors used in the STEMD Lab Test to measure the sensitivity profiles are not exactly the same models used in the ITEP 2009 test; however, these sensitivity profiles were assumed to be similar between models, because the size of the search head never changed. Therefore, little modification between models can be assumed, and results between older and newer models will be similar.

Submunition.
Objects used in the test. From left to right: metal clutter (ammunition belts, cartridges, bullets) and mine-like targets (Gyata-64, PPM-2, ERA calibration target).
Photo courtesy of BWB.

Conclusion

A method to analyze blind-test data of metal detectors for evaluating the pinpoint (location) accuracy is discussed and demonstrated with the data from the ITEP 2009 test. By this method, the pinpointing accuracy of metal detectors under field conditions is obtained as a mode and 95th percentile, indicating a pinpoint error that frequently occurs and includes 95% of detections. Additionally, the percentage of detections within a certain area is also calculable. Using the method for the data acquired in a blind test, a metal detector’s location error can be assessed, and the results can be used for the selection of a detector model. Moreover, the information may be used to establish an operating procedure for detection and safe excavation of landmines. For example, the perimeter of the path where deminers should begin excavating toward a target can more accurately be defined if the success and error rate of metal detections based on the model they use is known to the operators.

The location-accuracy stats obtained from the ITEP 2009 test was also discussed in relation to the way to pinpoint correctly and the differences in the sensitivity profiles of detectors. The data show a linear correlation between the pinpoint accuracy and the sensitivity profile for single-coil detectors. The result shows that a detector with a smaller search head produces more accurate results than larger search heads, making the smaller search heads generally better for locating targets. However, consider some other points when selecting a metal-detector model: A smaller search head is less sensitive to clutter, which also means it takes more time to thoroughly scan an area.7 Oval-shaped coils and double-D configuration may be good approaches for this trade-off. On the other hand, even with a larger coil and wider sensitivity profile, accurately pinpointing a target is possible. As shown in Figure 7, a sensitivity profile is elliptical in the vertical section, and the width becomes narrower farther from the coil. By lifting up the search head from the ground surface, a smaller part of the sensitivity area can be used for pinpointing. Experienced operators often use this technique to increase accuracy.

Sensitivity profile is influenced by many properties as theoretical works 7,8 and experiments9 have exhibited, such as the coil and electronic design of the devices, metal content and shape of the target, magnetic and electrical properties of the soil, etc. When clearance operations are planned at a site, the metal-detector model is the only choice users can make, and this determines the sensitivity profile and associated performance. Therefore, the choice is very important.

In the detection-performance analysis of blind tests, the concept of halo radius that sets a circular area around a target to define hit or miss was commonly used. In the CEN Workshop Agreement, the halo radius is “half of the maximum horizontal extent of the metal components in the target plus 100mm.”10 It is a circular area with a 5cm radius for a pointlike metal target. According to the results shown in Figure 4, 60–80% of the detections are correctly counted as a hit, but the remaining 20–40% of detections are not, by the halo definition, counted as a hit, because despite detecting the targets, these detections are outside of the halo. Obtained in this way, results may not show detection performance, but they include pinpointing performance in part. Thus, the definition in the CEN Workshop Agreement sounds a little too strict to evaluate only the detection performance.

In the ITEP 2009 test, only a few operators per detector model were available. The number is unfortunately too small to discuss the difference between different operators. Since the accuracy of metal-detector pinpointing probably depends on the operator’s skill and experience, this point could be investigated further, if and when more operators are available. J

The author is grateful to Dieter Gülle with Mine Action Consulting, Berlin, Germany, for his helpful suggestions and Dr. Adam Lewis with the Joint Research Centre, European Commission, Ispra, Italy, for providing the STEMD Lab Test data. He also wishes to acknowledge colleagues at the Technical Center for Protective and Special Technologies and the Federal Office of Defense Technology and Procurement (BWB), Oberjettenberg, Germany, for their support in collecting data. A special thanks to the manufacturers for providing their products (i.e., metal detectors) and trainers, and for actively seeking testing. The author is also grateful to BWB for supporting the work.


Biography

Asa GilbertKazunori Takahashi is a Research Scientist formerly employed with Federal Institute for Materials Research and Testing (BAM), and currently at Leibniz Institute for Applied Geophysics. His research activities include development and evaluation of dual-sensor systems for humanitarian demining, GPR signal-processing, and reliability analysis of nondestructive testing methods.



Contact Information

Kazunori Takahashi
Research Scientist
Leibniz Institute for Applied Geophysics
Stilleweg 2
30655 Hannover / Germany
Tel: +49 511 643 3572
Fax: +49 511 643 3665
E-mail: kazunori.takahashi@liag-hannover.de
Web site: http://www.liag-hannover.de/en/home.html


Endnotes:

  1. Takahashi, K. and D. Gülle. “ITEP Evaluation of Metal Detectors and Dual-sensor Detectors.” The Journal of ERW and Mine Action, 14.3 (Fall 2010): 76-79. http://maic.jmu.edu/journal/14.3/r_d/takahashi/takahashi.htm. Accessed 10 October 2011.
  2. http://www.itep.ws/pdf/STEMD_Interim_Lab.pdf. Accessed 10 October 2011.
  3. Guelle, D., A. Smith, A. Lewis and T. Bloodworth. “Metal Detector Handbook for Humanitarian Demining.” European Commission, 2003. “When a detector search head is held still above a metal object and the alarm sounds continuously, the detector is working in ‘static mode.’ In ‘dynamic mode’ detectors, the alarm turns off after a few seconds because the circuit compares the induced voltage with its value a few seconds earlier and only sounds the alarm if the voltage has changed.” http://www.itep.ws/pdf/ metal_detector_
    handbook.pdf
    . Accessed 10 October 2011.
  4. Geneva International Centre for Humanitarian Demining (GICHD). “Detectors and Personal Protective Equipment Catalogue 2009.” http://www.gichd.org/publications/detectors-and-personal-protective-equipment-catalogue-en. Accessed 10 October 2011.
  5. Ebinger. “EBEX 422 GC Metal Detector for Humanitarian Mine and Battle Area Clearance.” http://www.ebingergmbh.com/downloads/3/ebex_422gc_en.pdf. Accessed 10 October 2011.
  6. Steel ball bearings.
  7. Das, Y. “Effects of Soil Electromagnetic Properties on Metal Detectors.” IEEE Transactions on Geoscience and Remote Sensing, Vol.44, No.6 (June 2006): 1444-1453. IEEE Xplore. http://ieeexplore.ieee.org/stamp/
    stamp.jsp?arnumber=01634708&tag=1
    . Accessed 10 October 2011.
  8. Druyts, P., Y. Das, C. Craeye and M. Acheroy. “Modelling the Response of Electromagnetic Induction Sensors to Inhomogeneous Magnetic Soils with Arbitrary Relief,” IEEE Transactions on Geoscience and Remote Sensing, Vol. 47, No. 8 (August 2009): 2627-2638. http://ieeexplore.ieee.org/stamp/
    stamp.jsp?tp=&arnumber=4806100
    . Accessed 10 October 2011.
  9. Gülle, D., M. Gaal, M. Bertovic, C. Müller, M. Scharmach and M. Pavlovic. “South-East Europe Interim Report Field Trial Croatia (Continuation of the ITEP-Project Systematic Test and Evaluation of Metal Detectors - STEMD) 25 September–18 October 2006.” Berlin, March 2007. http://www.itep.ws/pdf/
    STEMD_Interim_Croatia_final.pdf
    . Accessed 10 October 2011.
  10. “CEN Workshop Agreement, Test and Evaluation of Metal Detectors,” CWA 14747, June 2003. http://www.itep.ws/pdf/CWA_metal_detectors.pdf. Accessed 10 October 2011.

TOP OF PAGE