Mine Detection Rats: Effects of Repeated Extinction on Detection Accuracy

by Amanda Mahoney, Amy Durgin, Alan Poling [ Western Michigan University, APOPO ], Bart Weetjens, Christophe Cox, Tess Tewelde, TeKimiti Gilbert [ APOPO ] - view pdf

This article describes the performance of Giant African Pouched Rats where reinforcement (reward) or
extinction (no reward) conditions affected landmine identification. Accuracy deteriorated quickly in the absence
of reinforcement, suggesting that reinforcement is essential.

Illustration of the experimental setup with one trainer and one notetaker.
Illustration of the experimental setup with one trainer and one notetaker.
All photos and figures courtesy of the authors.

As a result of almost 30 years of war, landmines are a devastating problem in Mozambique. According to a United Nations’ report, an estimated 20 people step on landmines every month in Mozambique and, due in part to lack of adequate health care, 60% of those people die.1 Since the mid-1990s, efforts have been made to clear Mozambique of landmines, but millions are believed to still contaminate the country. Anti-Persoonsmijnen Ontmijnende Product Ontwikkeling (Anti-Personnel Landmine Detection Product Development) started using Giant African Pouched Rats (Cricetomys gambianus) for landmine detection in Mozambique in 2007. Details on how the rats are trained and used operationally are provided elsewhere.2,3,4 In brief, the rats are trained through operant conditioning in which food reinforces (rewards) appropriate indication responses (i.e., those that occur within 1 m of a mine).5 Incorrect indication responses are not reinforced. Training begins in a controlled laboratory setting and proceeds through a series of steps to a large training field.

An early evaluation conducted in 2005 in which seven rats searched 20,234.28 sq m of land in Mozambique indicated that their detection accuracy exceeded 95%.6 In a more recent evaluation, teams of two rats searched 93,400 sq m of land in Mozambique, revealing 41 mines.7 This area was then searched with metal detectors, revealing a 100% detection rate by the rats. Such findings suggest that pouched rats are acceptably accurate in detecting landmines and, as a result, they are used operationally in Mozambique.

The mine detection rats in Mozambique work on training fields and actual minefields (operational sites). The training field comprises several 100 sq m, 200 sq m and 400 sq m boxes indicated by ropes along each side. Between zero and four deactivated landmines are buried within each box. The rats are attached to a rope (via a harness) held by two handlers on either side of the box. The rats walk across the box they are searching. When an indication response (pausing and digging) occurs within 1 m of a landmine, the trainer clicks to signal reinforcement and food is delivered.

When the rats are used operationally, the location of mines (and other explosive remnants of war) is unknown prior to clearance operations. Therefore, knowing whether an indication response is correct (i.e., within 1 m of a mine) or incorrect is impossible. To avoid the possibility of reinforcing incorrect responses and thereby potentially reducing the rat’s subsequent detection accuracy, no reinforcers are delivered when the rats are used operationally.

In technical terms, the rats work under extinction (no reinforcement) conditions when used operationally and under differential reinforcement (food reinforcement for correct responses, no reinforcement for incorrect responses) conditions during training. Extinction inevitably weakens previously reinforced responses.8,9 For this reason, the rats rotate between the training field and the operational site. The rationale for this arrangement is that reinforcement of correct responses on the training field will sufficiently strengthen such behavior to compensate for the response- weakening effects of extinction at the operational site. The rats’ performance at the operational site strongly suggests that this is the case, but we have not systematically evaluated the extinction effects, though studies are under way. In an effort to gain information of value to maximize the effectiveness of APOPO’s MDR team, the present study evaluated the effects of extinction on the detection accuracy of five rats performing under controlled conditions that allowed for accurate assessment of their performance.

Setting, Subjects and Materials

Trials took place in Morogoro, Tanzania on the APOPO training field, which contains approximately 1,200 landmines buried in a fenced 283,279.95 sq m site. In the portion of the training field used, one mine was buried in a marked 100 sq m box. Some of the boxes in APOPO's training field have markings to indicate landmine locations and some do not. The boxes without markings were used in the present study to provide blind testing conditions, under which the trainers were unaware of mine locations. The tests used six boxes, each containing just the one mine. Each test took an average of 17.8 minutes with a range in time of 8 to 25 minutes.

Five rats participated in this test. Each rat had recently passed a blind test in which it located each of eight unmarked mines in a 400 sq m area with no more than one false alarm. The rats were distributed between two trainer teams; each team comprised two trainers and one notetaker. The notetakers were APOPO minefield supervisors. APOPO certified all trainers and selected them because they demonstrated good adherence to standard operating procedures. Materials included clickers to signal availability of the food rewards, data sheets, a banana (the food reinforcer) and mine detection training box materials.

Training box materials consisted of measuring tape stretched along one side of the box and a rope that stretched across the box between the two trainers and guided the rat as it walked in the box. The rats were attached to the rope via a harness and lead cord and could walk back and forth along the rope. The trainers held two measuring tapes between them. One end of each tape was attached to the rat’s harness at zero. Thus, the exact location of the rat’s indications could be determined through the coordinates of the measuring tape value in the trainer’s hand and the measuring tape value at the trainer’s feet. After the rat walked down the rope in one direction, the trainers took a 0.5 m step forward and the rat walked in the opposite direction across the box. In all tests, the rats were allowed to traverse the rope only once before they were moved forward.

Data were recorded on graph paper that depicted the box measurements. Each test box was displayed as a grid comprised of 0.5 m x 0.5 m squares. Shaded gray squares corresponded to the mine locations. The indication response was scratching the ground for any length of time within 1 m of the landmine. Upon a rat indication, the trainer informed the notetaker, who recorded the location of the response and whether or not the trainer should sound a click and deliver food to the rat. In the reinforcement condition, the trainer was instructed to sound a click and deliver food (i.e., provide a reinforcer or reward) each time an indication response within 1 m of a mine was emitted. Reinforcers were never provided in the extinction condition.

Experimental Design

A multiple baseline with reversal design evaluated detection accuracy under reinforcement and extinction conditions.10 In a multiple baseline design, different subjects are initially exposed to the conditions of interest on different days. This design demonstrates that the changes observed when conditions change are the result of the change in conditions and not the result of some other factor (e.g., weather conditions, day of the week, time of exposure to a condition). A reversal design calls for returning to a prior condition, which in this case was the reinforcement condition. Thus, all of the rats were exposed to a reinforcement condition, then extinction, reinforcement and finally extinction.

When performance remained at 100% accuracy under the reinforcement condition over at least four consecutive days, the extinction condition began. Since there was only one mine per box, if the rat found it, the detection accuracy was 100%; if it did not indicate a mine, the detection accuracy was 0%. The rat worked under the extinction condition until detection accuracy fell to 0% for at least two consecutive days. This sequence was then repeated.

All rats worked in one box per day, and sessions were conducted up to five days per week. Sessions were not conducted on weekends, holidays or days with heavy rain. Data recorded each day for each rat were the location of indications, the number of hits (indication responses within 1 m of a mine), the number of false alarms, indication responses further than 1 m from a mine) and the number of misses (mines with no indication response within 1 m).

Reinforcement Condition. In this condition, when an indication response occurred within 1 m of a mine, the trainer produced a click sound using a handheld clicker. If the rat began to approach the trainer within 3 seconds of the click, which usually occurred, the trainer delivered food. If the rat did not approach the trainer within 3 seconds of the click, the trainer did not present food. If a rat walked over a mine without indicating, the rat continued clearing the rest of the box. Each rat searched each area of the box only once.

Extinction Condition. Extinction sessions were the same as reinforcement sessions, with the exception that neither a click nor food was presented following either correct or incorrect identification responses.

Second Reinforcement and Second Extinction Conditions. The second reinforcement condition, which was identical to the first reinforcement condition, occurred after the first extinction condition. The second extinction condition was the same as the first one and was the last condition arranged for each rat. Figure 1 shows the number of days that each rat was exposed to each experimental condition.

Independent-observer Agreement. A second observer independently collected data during 21.3% of sessions. The second observer agreed with the primary data collector on 98.1% of rat indications.

Results

Figure 1. Percentage of hits per day by individual rates in reinforcement and extinction.

Figure 1 shows the percentage of hits (correct identification responses) per day by individual rats during reinforcement and extinction conditions. Because each box had one mine, accuracy was either 0% or 100%. During the initial reinforcement condition, the rats identified all mines except for a single mine missed by Nijad in the third session. In general, because accuracy was 100% on the first day, the rats did not appear to learn from the use of the same six boxes. The trainers may have learned the location of the mines, and at some point they may not have been operating under blind conditions. However, a second observer was present during approximately 20% of the sessions to ensure that procedures were followed as written and that there was agreement in recording.

When extinction was introduced, accuracy declined for four of the five rats within three sessions. Enda’s performance did not fall until the seventh session but remained at 0% for six of the next seven sessions. Typically, the rats continued emitting an indication response over the mine on some days during extinction, but failed to indicate on about as many days as they indicated. Upon return to the reinforcement condition, detection accuracy for Toyota remained variable for six days while performance for Mar remained at 0% for eight out of nine days before improving to the initial reinforcement-condition level. Performance for Nijad and Bila recovered to 100% accuracy in two days, and Enda’s performance improved to this level after three days. Upon return to extinction, responding fell within two to four days for all rats. Performance again took several days to recover to prior reinforcement levels for Enda and Mar, although the performance of Bila, Toyota and Nijad recovered in zero to two days.

Figure 2 summarizes findings across the five rats. This figure clearly shows that overall the rat’s accuracy in detecting landmines was high during the first reinforcement condition and quickly declined when extinction was arranged. Accuracy remained inconsistent and relatively low after reinforcement was again arranged but eventually reached a high level. The rats’ accuracy again declined even more rapidly when extinction was introduced a second time. For this reason, these rats will not be used in actual future detection operations.

Few false alarms (incorrect identification responses) occurred under any condition, and the number of false alarms per session did not consistently differ under reinforcement and extinction conditions. None of the rats emitted more than three false alarms on any given day, and an individual rat typically emitted zero or one false alarm each day.

Figure 2. Average percentage of hits per day under reinforcement and extinction conditions.

Discussion

This study evaluated the performance of APOPO’s MDRs under reinforcement and extinction conditions and found that, in general, the rats demonstrated high accuracy and stable performance after sufficient exposure to the reinforcement condition and variable but substantially lower accuracy during the extinction condition. There was high carryover from the reinforcement condition in that performance remained variable after the reinforcement condition was reinstated. Sometimes several reinforcement sessions were necessary for performance to recover to 100% accuracy.

In APOPO’s operational setting, the rat does not receive reinforcers, because it is unknown where mines lay and, consequently, whether the rat’s indications are correct (i.e., within 1 m of a mine) while searching. The study’s aim was to determine how many days an MDR can work on a minefield, without reward, before performance degrades. Under the conditions of the present study, this period was conclusively determined to be quite short. The rats’ accuracy in detecting mines fell, on average, after 3.1 days of exposure to extinction, although their false alarm rates did not change systematically. Furthermore, recovery of the asymptotic accuracy level following extinction took up to nine days.

To maximize experimental control, the present study only used 100 sq m boxes containing a single mine. In operational demining in Mozambique, the overall density of landmines is substantially lower. For example, in one study the rats located 41 landmines in a 93,400 sq m area, which yields an average of 0.04 mines per 100 sq m box, although in some cases a rat may pass over two or more mines in a small area. The effects of extinction on the performance of MDRs under such conditions, where target density is highly variable but low overall, remain to be determined. Of course, performance in extinction depends on a number of environmental variables. These variables seemingly would include the number of responses emitted without reinforcement and the manner in which reinforcement was arranged prior to extinction.

Future research in this area might investigate the effects of training with intermittent reinforcement, which is well-known to prolong accurate performance under extinction. 11 Though APOPO has not yet evaluated this methodology, it has used intermittent reinforcement, with trainers rewarding 85% of indications. APOPO plans to study intermittent reinforcement and evaluate optimal parameters and effectiveness.

APOPO is currently investigating the utility of exposure to reinforcement conditions, prior to or following the extinction condition.12 The success of this procedure depends largely upon how well the rats discriminate
between training (reinforcement) and operational (extinction) conditions.

These tests were conducted for experimental purposes to provide relevant information to APOPO management. Prior research conducted under operational conditions indicates that APOPO’s rats are accurate in detecting landmines under the conditions arranged in Mozambique.6,7 APOPO draws upon several means of reinforcement delivery in operational conditions: frequent quality control checks, data collected regularly on individual rat performance and ample opportunity for reinforcement on the nearby training field. How the rats would perform under other conditions, for example, if they worked for longer periods each day or in areas with different landmine concentrations, is speculative. The present data strongly suggest, however, that their accuracy would decline significantly if they worked for periods during which several indication responses occurred and were not reinforced. This study and previous ones provide a research base that informs APOPO’s operating procedures in a way that continually optimizes operating procedures and ensures the rats’ performance is maintained at high levels under operational settings.

APOPO’s primary goal is using pouched rats effectively and efficiently for humanitarian purposes, not conducting scent-detection research. Such research is, however, the best means to that end and for that reason is given high priority by the organization. Conducting research uses personnel, time and financial resources that could go directly toward mine clearance or land release. Therefore, we attempt to choose research topics carefully and to design studies in a way that minimizes cost. Small-N research strategies characteristic of behavior analysis have proven especially valuable in this regard, and we recommend them to the humanitarian demining community.10,13 globe

 

Biographies

Ian McLeanAmanda Mahoney is a behavioral researcher at APOPO and a doctoral candidate at Western Michigan University (U.S.). She completed this work in partial fulfillment of her doctoral dissertation.


Ian McLeanAmy Durgin is a behavioral researcher at APOPO and a doctoral candidate at Western Michigan University (U.S.). She assists APOPO with conducting research designed to improve and maintain an efficient and effective rat training program.


Jennifer BrownAlan Poling is a psychology professor at Western Michigan University (U.S.). He
has played an integral role in research and development at APOPO since 2009.



Håvard BachBart Weetjens is a product engineer and APOPO’s founder. He conceived of using scent-detecting rats for humanitarian purposes.


Håvard BachChristophe Cox is APOPO’s CEO and developed much of the tuberculosis detection application. Cox is a product engineer and has worked frequently in Africa.


Håvard BachTess Tewelde is the Program Manager for APOPO’s Mozambique Mine Action Program. Tewelde has 11 years of experience in mine action in Africa in humanitarian and commercial sectors.

Tess TeweldeTeKimiti Gilbert joined APOPO as the Head of Mine Action in March 2012. Prior to that time, he worked extensively with the United Nations and has worked in land clearance around the world.


Contact Information

Amanda Mahoney
SUA-APOPO
Sokoine University of Agriculture
PO Box 3078
Morogoro / Tanzania
Tel: +001 586 292 0644
Email: Amanda.mahoney@apopo.org

Amy Durgin
SUA-APOPO
Sokoine University of Agriculture
PO Box 3078
Morogoro / Tanzania
Email: amy.durgin@apopo.org

Alan Poling
Western Michigan University
Department of Psychology
3700 Wood Hall
Kalamazoo, MI 49008-5439 / USA
Tel: +001 269 387 4500
Fax: +001 269 387 4550
Email: alan.poling@wmich.edu

Bart Weetjens
SUA-APOPO
Sokoine University of Agriculture
PO Box 3078
Morogoro / Tanzania
Email: apopo@apopo.org

Christophe Cox
SUA-APOPO
Sokoine University of Agriculture
PO Box 3078
Morogoro / Tanzania
Email: apopo@apopo.org

Tess Tewelde
Mozambique Program Manager
SUA-APOPO
PO Box 649
Maputo / Mozambique
Tel: +258 827 273 378
Email: tess.tewelde@apopo.org

TeKimiti Gilbert
Head of Mine Action
SUA-APOPO
Tel: +33 4504 10889
Email: Tekimiti.gilbert@apopo.org

 

Endnotes

  1. Lemish, Michael G. War Dogs: A History of Loyalty and Heroism. Potomac Books Inc., 1999.
  2. Hayter, Dan. “Training Dogs to Detect Tripwires.” Mine Detection Dogs: Training Operations and Odour Detection, June 2003. Geneva International Centre for Humanitarian Demining. http://www.gichd.org/fileadmin/pdf/publications/MDD/MDD_ch2_part3.pdf. Accessed 14 June 2012.
  3. Duggan, Jennifer M., Heske, Edward J., Schooley, Robert L., Hurt, Aimee and Alice Whitelaw. “Comparing Detection Dog and Livetrapping Surveys for a Cryptic Rodent.” The Journal of Wildlife Management, 75 (2011), 1209-1217. http://onlinelibrary.wiley.com/doi/10.1002/jwmg.v75.5/issuetoc. Accessed 14 June 2012.
  4. McLean, Ian. “Environmental applications in demining.” Journal of Mine Action, 9.2 (2006), 59-60. http://maic.jmu.edu/cisr/journal/9.2/feature/mclean/mclean.htm. Accessed 14 June 2012.
  5. Horwood, Chris. “The Use of Dogs for Operations Related to Humanitarian Mine Clearance.” Lyon: Handicap International, 1997.
  6. Mine Detection Dogs: Operations, Case Studies of Operational Systems, September 2005. Geneva International Centre for Humanitarian Demining. http://www.gichd.org/publications/subject/animal-detection/mine-detection-dogs-operations-case-studies-of-operational-systems. Accessed 14 June 2012.
  7. Jane's Mines and Mine Clearance. IHS Jane's: Defense & Security Intelligence and Analysis. http://jmmc.janes.com/public/jmmc/index.shtml. Accessed 14 June 2012.
  8. IMAS 09.42: Operational Testing of Mine Detection Dogs and Handlers, Second Edition, United Nations Mine Action Service (1 March 2008). http://www.gichd.org/operations/operational-quality-management/reference-library/. Accessed 14 June 2012.
  9. “Detection of Landmines by Dogs: Environmental and Behavioural Determinants.” November 2005. Geneva International Centre for Humanitarian Demining. http://www.gichd.org/lima/reports-publications/detail/publications/detection-of-landmines-by-dogs-environmental-and-behavioural-determinants/. Accessed 14 June 2012.
  10. Fisher's Least Significant Difference test is a statistical test that compares the mean of one group with the mean of another.
  11. Phelan, James M. and Stephen W. Webb. “Chemical Sensing for Buried Landmines: Fundamental Processes Influencing Trace Chemical Detection.” Mine Detection Dogs: Operations, Case Studies of Operational Systems, June 2003. Geneva International Centre for Humanitarian Demining. http://www.gichd.org/publications/mine-detection-dogs-training-operations-and-odour-detection-en. Accessed 14 June 2012.

 

TOP OF PAGE