ITEP/JMU Database of International Experiences: Supporting the Test and Evaluation Community
by Franciska Borry and Dieter Guelle, ITEP
A new database has been added to the James Madison University (JMU) Lessons Learned database, with more specific content and aim. The specific content reflects the main tasks that the International Test and Evaluation Program (ITEP) for humanitarian demining has agreed on in its Memorandum of Understanding, namely to evaluate and standardise the process of equipment testing in the humanitarian demining industry. The JMU Mine Action Information Centre (MAIC) maintains the database, whereas the ITEP Secretariat is responsible for its content.
The ITEP/JMU Database of International Experiences in Support of the Test and Evaluation Community (DIETEC) was created in order to summarise test and evaluation (T&E) experiences that are referenced to the original test reports. The most demanding part was to work out the database structure and to discuss the concrete contributions currently included. There are fine dividing lines, when at all, between lessons learned, experiences and test results. A remaining question is whether it is opportune to also include technical questions related to the use of tested equipment (test results) or to strictly focus on the testing process. In this regard, we would like to invite the reader to share his/her opinion with the ITEP Secretariat.
Currently, the database only contains experiences related to the T&E process. It was found that the test reports contain an important amount of information, either general or specific, which could be useful for the T&E community as a whole. In order to make these hidden experiences more widely and easily available to the test community, ITEP decided to join up with the MAIC. Funding was provided by the U.S. Department of Defense.
The main aim of DIETEC is to collect and publish information derived from experiences in T&E of humanitarian demining equipment. The collected "experiences" are intended to highlight key areas of consideration in the T&E process, as well as specific observations related to the evaluation of humanitarian demining equipment in operational use. Moreover, it attempts to provide a common structural approach for T&E of equipment used in humanitarian demining. The database should provide useful information for a variety of T&E stakeholders, ranging from test engineers and entities involved in large-scale T&E campaigns to the individual user at the field level interested in evaluating his/her specific piece of equipment.
Definitions and Approach
The database, in its current version, provides a list of experiences. The experience may be positive or negative but should meet the following criteria:
The experience may be generic or equipment-specific and can be submitted by any individual or organisation. However, a core set of experiences is extracted from publicly available resources (T&E reports on humanitarian demining equipment) and from T&E activities undertaken under the umbrella of the International Test and Evaluation Program for Humanitarian Demining Equipment.
The intention is that all experiences entered in the database will be periodically reviewed by an international panel of experts from the ITEP network and other organisations performing activities related to T&E, e.g., the United Nations or the Geneva International Centre for Humanitarian Demining (GICHD). During this process, the relevance of the experiences to T&E standards, technical notes and/or methodologies will be assessed.
An entry could become one of the following four things:
In its initial stage, the database will mainly include "experiences." After review, the database will be expanded with "lessons learned," referenced to a set of experiences. These lessons learned can then further be incorporated into standards and similar documents whenever relevant.
The structure of the database is illustrated in Figure 1 above. Category and sub-category fields have been selected in order to provide the user with a structured overview of the main stages and factors that should be considered during the T&E of humanitarian demining equipment. The structure may be used as a guide for users when drawing up their own test plan. The utilized categories expand on the structure given in the International Mine Action Standards (IMAS) 03.40 on Test and Evaluation of Mine Action Equipment (first edition, 01.01.2003) and include the main technical categories utilized by ITEP and the GICHD Mechanical Demining Equipment Catalogue.
The database consists of two main categories: Generic Considerations and Equipment-Specific Considerations. Each main category has subcategories to explore the experiences in detail and to guide the user to add relevant experiences. Each subcategory can be browsed by clicking on the corresponding individual cell or by executing a detailed search. The user can add an experience to a subcategory by clicking on the corresponding individual cell or on "Add Experience." Table 1 includes two examples extracted from the database in order to illustrate the type of information provided by DIETEC.
As mentioned in the introduction, during the analysis of the reports, valuable information was also encountered that is related to technical and operational deployment of the equipment. For instance, the report The Severe Duty Vegetation Shredder Technical Testing of Capability by the U.S. Night Vision and Electronic Sensors Directorate (NVESD, online at http://www.humanitarian-demining.org/demining/pubs/clearance/svc_test_report.asp) mentions the fact that fence wire tangled up in the rotating machinery and inflicted considerable damage/disturbances during the testing. This finding could also have implications for the operational use of this type of equipment, not only in fenced minefields but also in dense vegetation where liana-like plants can have the same effect. Hours may be needed for the machinery to become operational again. Important feedback from the reader could be an indication that similar information would be worthy of being included in DIETEC.
Care should be taken when using the term "lessons learned." In general, the process followed for compiling a "lesson learned" is quite complex and consists of an information-gathering and -processing chain spread over a considerable time period. For instance, the Swedish EOD and Demining Centre (SWEDEC) lessons learned project includes several phases such as the collection of an "experience report" using a standard form, which is then analysed and commented on in an "extended experience report." This stage is followed by validation of the information (i.e., Did it happen several times? Is it useful for another organisation? etc.), which leads to the implementation phase and the compilation of the "lessons learned report." A similar process is being applied by the Department of Energy (DOE) Corporate Lessons Learned Collection database, for example. However, this is not the approach followed at present by the JMU MAIC lessons learned database and is, in our opinion, at this moment in time not necessary, mainly due to the given structure of the database.
Both the JMU MAIC database and DIETEC are currently being evaluated by an international team of "experts." No final results are available yet, but preliminary results of the assessment indicate that an important percentage of the DIETEC experiences have been classified as relevant to the T&E process.