most recent issue
Systematic Test and Evaluation of Metal Detectors: The EC's STEMD Project

There is an international need for people within the humanitarian demining community (HDC) to be informed about appropriate metal detectors for the clearance process. Those involved in information exchange within the HDC will know that metal detector trials are performed regularly. User requirements and test results can easily be published because a network exists within the HDC. The Journal of Mine Action itself is part of this network and readers will probably also be familiar with the websites and publications of the International Test and Evaluation Program for Humanitarian Demining (ITEP) and the Geneva International Centre for Humanitarian Demining (GICHD). The GICHD periodically publishes a metal detector catalogue containing manufacturers' specifications and references to test reports. Since the middle of the 1990s, increased efforts have been made to improve testing and develop reliable and systematic methods. The European Commission's (EC's) project, Systematic Test and Evaluation of Metal Detectors (STEMD), is a new step in this development.

Current Practice in Detector Testing

Different detectors perform best under different trial conditions, so these are often set to satisfy one customer with specific needs and requirements, such as certain targets or soil types. We cite, for example, the series of tests under the U.N. umbrella, which began in January 1997 when the United Nations Mine Action Service (UNMAS) performed a trial in Sarajevo and Mostar. The tests continued with the Mine Action Programme for Afghanistan (UNMAPA) trials in Peshawar, Jalalabad and Kabul (September 1999 to March 2000)1 and with those of the Accelerated Demining Programme (UNADP) in three southern provinces of Mozambique: Inhambane, Gaza and Maputo (Autumn 1999 and Autumn 2000)2 and another UNMAPA trial3 in 2002.

Other users want to read reports of trials such as these and infer how useful a device would be for their own needs. It can be difficult to do so when each trial is conducted differently, so the problem is twofold: to gather sufficiently complete test data to be able to understand how a detector will perform under all conditions and to standardize test protocols to avoid more variation in test methods between trials than is necessary.

A breakthrough was achieved in 1998 when the International Pilot Project for Technology Co-operation (IPPTC) was launched. IPPTC was the most comprehensive trial of metal/mine detectors ever conducted. Five research organisations and two national mine action centres carried out a broad range of tests on 28 devices under controlled conditions in laboratories and in blind field trials in two countries, with the intention of assessing their performance under as wide a range of conditions as possible. The results were made broadly available in a consumer report4 in 2000.

By the time the IPPTC Final Report was published, some new detectors had already been released to the market. It was recognised that it was impractical to organise repeats of IPPTC to keep the information current. Instead, it was better to use the experience to agree to a standardized set of test procedures. Other standards do exist for metal detectors, but they are designed for users such as the military, security organisations and police, whose requirements are somewhat different from those of humanitarian deminers. In humanitarian demining, the detectors are in constant use in rough conditions. Any mistake can be instantly life-threatening. Sensitivity to minimum metal mines, compensation for soil properties, ergonomics, ruggedness, battery life and electromagnetic immunity are the main factors to be assessed.

In 2001, the European Committee for Standardization (CEN) organised CEN Workshop 7, "Humanitarian Mine Action—Test and Evaluation—Metal Detectors" (CW07) under the leadership of the Joint Research Centre (JRC), the EC's in-house research organisation, to examine the detector situation. ITEP recognized this project as Project 2.1.1.1.8 It resulted in the publication of CEN Workshop Agreement (CWA) 14747:2003 for Test and Evaluation of Metal Detectors for Humanitarian Demining in June 2003. Details of the process have been described previously in this journal.5 The CWA incorporates contributions from experts worldwide and distils the experience gained in IPPTC and other recent projects.

Applying and Disseminating CWA Methods and the Birth of STEMD

After the publication of the CWA, it became obvious that it would remain just a paper exercise unless someone took the initiative to apply the methods and to organise training and dissemination—to "fill it with life" and get it used by the wider HDC.

A validation study led by the German Federal Materials Laboratory (BAM) was conducted in partnership with the JRC in the second half of 2003 in which just five models were subjected to CWA tests, including exhaustive statistical testing in three blind field trials (ITEP project 2.1.1.2).8 In the same year, the JRC published a handbook9 on metal detectors, which included descriptions of the CWA test methods. The JRC started to select and buy new detectors that had not been included in the IPPTC for testing to the CWA.

So, at the end of 2003, a tool for testing had been provided in the form of the CWA, new detectors were emerging and a means to disseminate the results was available via the existing HDC network. Missing was the organised in-depth training of the detector users. A training seminar was held in January 2004 at Ispra, Italy, in which users and manufacturers worked through the CWA tests in hands-on exercises. During this seminar, we found that the commercial clearance organisations and non-governmental organizations would indeed be interested in improving their knowledge of the use of metal detectors under field conditions. Representatives of southeast European and African mine action centres, demining organisations, UNMAS and detector manufacturers participated in this seminar, and, to our surprise, every participant learned something new.

At that time, the idea of a project with the following aims was born:

The EC's EuropeAid Cooperation Office agreed to fund the JRC to conduct the STEMD project. ITEP recognized STEMD as project 2.1.2.3.8 The intention of STEMD is to cover as many of the lab and field tests prescribed in the CWA as possible, which requires significant time and effort. We therefore sought help from ITEP partners and the GICHD, which was forthcoming (see "Acknowledgements" below).

Current Progress of STEMD

It was initially planned to conduct all laboratory measurements first at the JRC's test facility in Ispra and then proceed to field trials in southeast Europe and southern Africa using the same detector set. A final field trial would be conducted in southeast Asia. As the project was in final planning, UNMAS said that the Lao National Unexploded Ordnance Programme (UXO Lao) was planning a procurement in autumn 2004 and had urgent need for test results. Therefore, the project plan was revised so a trial in Laos would be conducted first. A preliminary investigation was undertaken to select suitable sites in Laos, and metal detector manufacturers were informed. Manufacturers were then able to provide trainers and propose models adapted to the requirements of UXO Lao. When the project started on 1 August 2004, detailed plans were made and the trial took place between 27 September and 5 November 2004 at three villages in the south of the country—Saravan, Thateng Tai and Pakxong.

A full report of the UXO Lao trial is available from the ITEP website.7 We focussed on two items of UXO of particular interest to UXO Lao: BLU 26B cluster bomb submunitions and 20-mm cannon projectiles. We followed the methods of CWA Chapter 8, "Detection Capability for Targets Buried in Soil," in particular those of Sections 8.3, "Detection Capability for Specific Targets in Soil," and 8.5, "Detection Reliability Tests." The detection capability tests were performed at all three sites and the reliability tests at two of them. Eight commercial off-the-shelf detector types from six manufacturers were tested—four large-head types designed for UXO items and four small-head types for mines. All but one manufacturer sent representatives to the training. Particular attention was paid to the selection of sites with a range of soil properties, from "neutral" to "very severe," according to the classification of the CWA. Detailed magnetic measurements were made on-site. Care was taken with the design of the reliability tests to ensure that the results would be unbiased and statistically significant; the experience gained during the 2003 validation study with BAM was very valuable here.


Reliability test of a CEIA Mil D-1 DS detector at Test Site 1, Saravan, Laos.

Laboratory measurements were conducted in Ispra during the periods before and between the field trials. So far, most of the in-air sensitivity and speed test measurements of CWA Chapter 6 were performed, but more work is required on sensitivity profile (footprint) measurements. A substantial fraction of the ergonomic tests of CWA Chapter 7 also were conducted. More work is required on temperature and battery-life tests. In-soil measurements, according to CWA Chapter 8, were performed at Ispra using ferromagnetic soils from the Siena and Napoli areas, which complement the results obtained during the Laos trial.

Further Work: Can Detector Performance be Predicted?

The main factors that influence the maximum performance of a metal detector are quite well-known: the quantity, type, shape and disposition of metal in the target, the target depth, and the soil's electromagnetic properties. Actual performance also depends on human factors, such as sweep pattern and how the operator behaves when the detector alarms.

The key soil properties are magnetic susceptibility and its frequency dependence and, to a lesser extent, the electrical conductivity. Simple measurements are possible that allow a general picture of the conditions of the soil and its influence on the detector performance to be assembled. A new ITEP research project (2.1.1.4),8 led by the Canadian Centre for Mine Action Technologies and supported by the Agropedology Institute of Sarajevo, the JRC, and others, is under way and should increase our knowledge in this area. Data on the relevant soil characteristics are not available for most mine-affected countries but they could be quite easily gathered during the Country and Technical Surveys, which are carried out before clearance activities start. The STEMD project should provide enough empirical data to study the possibility of extrapolating detector performance under different conditions and recommending detectors for particular targets under those conditions.

Conclusion

The STEMD project now is well under way and already has returned useful data. A substantial body of laboratory results has been collected and the first field trial has been completed. From this data, it is possible to compare the performance of the individual detectors in the key areas of interest, including maximum detection depth for various targets, soil compensation ability and probability of detection as a function of depth. It is also possible to compare the large-head detectors as a group to the small-head detectors as a group. The next field trial in Mozambique is scheduled for April 2005 and will focus on detection capability tests for AP mines of interest there. The third field trial in southeast Europe is scheduled for the summer of 2005. After STEMD, trials of the emerging ground-penetrating radar/metal detector dual sensors are planned under the title "Test and Evaluation of Dual Sensors".

Acknowledgements

We wish to thank Dr. A. J. Sieber, as well as colleagues and ex-colleagues of the JRC's Sensors, Radar Technologies, and Cybersecurity Unit-in particular Dr. T. J. Bloodworth, M. A. Pike and F. R. Littmann; J. van Dromme of AIDCO; A. Carruthers of the GICHD; and D. Lewis and S. Bowen of QinetiQ. For the Lao field trial, we also wish to thank Bounpone Sayasenh, Phommachan Khammanichan and the staff of UXO Lao; J. P. Salmon, P. Vegucht and J. L. Trullemans of the Belgian Ministry of Defence and E. Gagnon of the United Nations Development Programme.

This work forms part of the EC JRC's Action 4341 "Test and Evaluation of Technology for Humanitarian Demining." STEMD is funded by the EuropeAid Cooperation Office under Administrative Arrangement AA MAP/2004/078-257.

*Photo courtesy of EC JRC

References

  1. Mine Action Programme for Afghanistan: Mine Detector Trial Report, September/October 1999, February/March 2000.
  2. International Detector Tests UNADP, Final Report, Dieter Guelle, UNADP Mozambique, December 2000.
  3. Metal Detector Trial, U.N. Mine Action Programme, Afghanistan, February/March 2002, www.itep.ws/pdf/Summary_Detector_trial_report_afghanistan_w_annex_1.pdf.
  4. International Pilot Project for Technology Co-operation Final Report; A multi-national technical evaluation of performance of commercial off-the-shelf metal detectors in the context of humanitarian demining; Editors Y. Das (CA), J.T. Dean (EC), D. Lewis (UK), J.H.J. Roosenboom (NL), G. Zahaczewsky (U.S.), Pub. No. EUR 19719 EN—2001 European Commission, http://www.humanitarian-demining.org/demining/pubs/detection/ipptc/index.htm.
  5. T. J. Bloodworth and A. J. Sieber, "Standardized Testing of Metal Detectors." Journal of Mine Action, Issue 7.3 (2003): pp. 16–19, http://maic.jmu.edu/journal/7.3/focus/bloodworthsieber/bloodworthsieber.htm.
  6. CEN Workshop Agreement: Humanitarian Mine Action—Test and Evaluation—Metal Detectors (CWA 14747:2003), June 2003, http://serac.jrc.it/tethud/cw07/pdf/CWA14747.pdf.
  7. Systematic Test and Evaluation of Metal Detectors (STEMD), Interim Field Trial Lao. Special Publication No. S.P.105.24, European Commission JRC (2005), http://www.itep.ws/pdf/STEMD_Interim_Laos_final_Small.pdf.
  8. "International Test and Evaluation Program for Humanitarian Demining, ITEP Workplan," http://www.itep.ws/activities/itep_workplan/activities.html.
  9. D. M. Guelle, A. L. Smith, A. M. Lewis and T. J. Bloodworth. Metal Detector Handbook for Humanitarian Demining. Pub. No. EUR 20837, European Commission: Luxembourg (2003) ISBN 92-894-6236-1, http://serac.jrc.it/publications/pdf/metal_detector_handbook.pdf.

Contact Information

D. M. Guelle and A. M. Lewis
Sensors, Radar and Cybersecurity Unit
Institute for the Protection and Security of the Citizen
European Commission DG Joint Research Centre
TP 723 Via Enrico Fermi 1
Ispra (VA) 21020
Italy
E-mail: Adam.Lewis@jrc.it
Website: http://serac.jrc.it/tethud