Issue 7.2, August 2003
The author presents an overview of Test and Evaluation (T&E) projects within the International Test and Evaluation Program for Humanitarian Demining (ITEP). ITEP is involved in testing and evaluating equipment, systems and methods.
One of the major objectives for participants in ITEP is to share information and to join efforts in the area of T&E of humanitarian demining equipment, systems and methods, in order to leverage resources and promote the use of commonly agreed upon test protocols. An important instrument needed to obtain active collaboration in ITEP is the ITEP Work Plan. The following is an overview of ongoing and planned national and international collaborative T&E projects within the ITEP. Its main aims are is to increase the efficiency of T&E activities by avoiding duplications, providing feedback on lessons learned and using methodologies based on common practices or agreed upon test protocols and standards. Finalised projects are kept in the Work Plan and are classified as completed and/or archived.
Overview of the Work Plan
|Figure 1: Overview distribution of projects in each technical program.|
The Work Plan is structured on the basis of a systematic approach to T&E. Technologies are grouped into six technical programs covering a number of projects. Currently, the ITEP Work Plan contains a collection of 43 projects. Figure 1 provides an overview of the distribution of projects in each technical program. A separate program has been added to cover ITEP services provided by the Secretariat and project management groups. Examples of ITEP services are the establishment and maintenance of the ITEP website and the collection of lessons learned.
For each technical program and project, the systematic approach to T&E uses the following criteria: Input, Methodology, T&E and Output.
Figure 2 illustrates the current distribution of projects in each technical program, according to the above criteria.
|Figure 2: Current distribution of projects in each technical program.|
Examples of How ITEP Works
The ITEP Work Plan was first drafted in 2002 and is continuously updated whenever new national or collaborative efforts are set up. Participants report new projects to the ITEP Secretariat, who incorporates them into the Work Plan. A yearly major revision and update of the Work Plan is organised by a standing ITEP Work Plan Working Group, which includes representatives of all participants. They report on the progress and status of the projects and provide feedback on the finalised T&E activities. Some examples of how ITEP works and how this is reflected in the ITEP Work Plan are given as follows.
The plan has highlighted duplications of efforts, and where this has occurred, participants have been encouraged to collaborate. Such was the case in the currently planned T&E of Commercial Off-the-Shelf landmine neutralisation devices. Sweden, Canada and the United States were all planning a similar activity and have decided to combine their efforts, which resulted in ITEP project 6.2.1, Comparative T&E of Individual Mine Neutralization Devices. The project will start with a demonstration at a Swedish test range this summer and should bring the interested ITEP partners together to initiate the definition of a standard test protocol (e.g., test targets). This type of planned T&E activity, published in the ITEP Work Plan, is also aimed at informing users and/or manufacturers, who might want to include a particular product in the comparative T&E activity. As such, ITEP can provide a responsive T&E program.
The T&E of the ARMTRAC 100, requested by the UK’s Department for International Development, was brought under the ITEP umbrella (ITEP project 3.2.1, T&E Trial of the Mechanical Equipment ARMTRAC 100). This meant that interested ITEP participants were asked for input on the test protocol and for participation in the trial. Canada, Sweden and the European Commission (EC) all provided support. The resulting T&E report is available on the ITEP website. The test provides input on harmonising test protocols between the ITEP participants. As a result, a best practice for T&E of mechanical assistance equipment is currently being drafted by the Canadian Centre for Mine Action Technologies (CCMAT) (ITEP project 3.1.1, T&E Guidelines for MAE), and a standardisation CEN Workshop Agreement (CWA) approach on this matter began June 2003. The CWA is being led by the Swedish EOD and Demining Centre (SWEDEC) (ITEP project 3.1.5, CEN Workshop Agreement on T&E of “Demining Machines”). The goal is to draft a CWA for distribution and revision by the end of 2003. The 2003 goal must be met in order to deliver an agreed CWA to CEN (and later to the International Mine Action Standards [IMAS]) by 2004.
Two prototype handheld multi-sensor mine detectors, developed in the United Kingdom and tested for the United Kingdom’s Department for International Development, were evaluated under the ITEP umbrella as a bilateral (UK/U.S.) T&E project (ITEP project 18.104.22.168, T&E of QinetiQ and ERA Portable Mine Detectors). This meant that the U.S. large-scale detector test facilities at Fort A.P. Hill, Virginia, were available for blind tests against a large number of mines and clutter targets. Hence, both detectors were tested against a standard protocol and the obtained results are considered statistically valid and comparable. As the tests involved equipment under development, only summary test information has been made available on the ITEP website. Detailed test reports may be consulted from the respective companies under a non-disclosure agreement.
An important methodology-related T&E activity is currently ongoing. The applicability of reliability measures developed in non-destructive testing to the detection of mines using metal detectors is being investigated (ITEP project 22.214.171.124, Reliability Model for Metal Detector Evaluation). The project is a collaborative effort between Germany and the Joint Research Centre (JRC) of the European Commission (EC). Trials are taking place in EC, German and Croatian test facilities. Trials involve four different metal detector types. The major objective is to determine the number of repetitions, targets, operators, etc. that are needed to get reproducible results when executing metal detector performance tests. Furthermore, the relation between the performance results obtained in controlled laboratory tests and blind trials is also being evaluated.
Another project related to the evaluation of metal detectors is planned to start in the second half of 2003. It capitalises on several parallel efforts within the ITEP community to assess soil characteristics in order to evaluate the performance of metal detectors. Canada has decided to join efforts with the EC and Belgium and start an experimental study in Bosnia and Herzegovina (ITEP project 126.96.36.199, Soil Characterisation for Assessment of Metal Detector Performance). A large set of electromagnetic soil characteristics will be measured and related to the performance of metal detectors that are currently used the most. Discussion on co-operation with the Agropedology Institute of Sarajevo (AIS) and the Bosnia and Herzegovina Mine Action Centre (BHMAC) are currently being finalised.
A comparative T&E activity on various Commercial Off-the-Shelf protective mine shoes is taking place in the summer of 2003 as a collaborative effort between the United States and Canada, the two main players involved in T&E of protective footwear (ITEP project 5.1.1, Methodology for T&E of Personal Protective Equipment). The main objective is to develop and validate a standard test protocol to measure the effects of shock/blast and to evaluate and rank the performance of personal protective equipment. The T&E report will be made available on the ITEP website. The established test protocol and lessons learned will also constitute important input for the standardisation community. Furthermore, this activity could perform the function of a responsive T&E program by including products on request by the user and/or producer.
A systematic inventory of T&E activities, capabilities and needs in southeastern Europe (SEE) was finalised in November 2001. The project, under the leadership of the EC, and with input from Belgium, Canada and the United Kingdom, produced a detailed overview of ongoing projects and existing T&E capabilities that support demining efforts in the SEE region. Deficiencies in the current T&E operations and requirements were also identified and reported. The report is available on the ITEP website. Although this project took place before the ITEP Work Plan came into existence, it is mentioned here because it was the first collaborative effort executed under the ITEP umbrella.
Drafting the first version of the CWA on T&E of metal detectors is an important collaborative effort achieved under the ITEP umbrella (ITEP project 188.8.131.52, CEN Workshop Agreement on T&E of Metal Detectors). The EC performs a leading role in this ITEP activity, with the availability of a full-time Secretariat at JRC. Full details on the first results of this project and the future activities will be the subject of a separate article to appear in the next issue of the Journal of Mine Action.
ITEP works in close relationship with UNMAS and GICHD. ITEP participants are encouraged to reach out into the user community to seek feedback on the ITEP Work Plan and, together with other stakeholders, to identify user needs in order to update and adapt the T&E projects accordingly. The ITEP Work Plan is available through the ITEP website, http://www.itep.ws/. Its distribution is also being facilitated by UNMAS and GICHD.
ITEP recognises the fact that a considerable amount of T&E has been and is being conducted by many other organizations in the field of humanitarian demining. The hopes and expectations are that members of the demining community will consult the Work Plan, identify relevant T&E activities, request more information and possibly actively collaborate in them.
*All graphics courtesy of the author.
European Commission: Joint Research Centre
Via E. Fermi T.P. 723
I-21020 Ispra (VA)
Tel: +39 0332 78 5546
Fax: +39 0332 78 5772