|
The use of computers in student proficiency testing has become increasingly common. In numerous settings around the United States, computer-based tests (CBTs) are being implemented and important decisions are being made about examinees on the basis of these tests. For example, ETS's Graduate Record Examination (GRE) is now offered in a computer-based format, and a CBT version of the Scholastic Achievement Test (SAT) will soon become operational. In addition, computer-based licensure and certification tests have been used for a number of years in many professional fields.
At JMU, a number of CBTs in General Education tests are currently in use, including three technology-proficiency tests; a Speech Communication final exam; and Assessment Day tests in the areas of communication, fine arts, scientific and quantitative reasoning, and history and global experience. In addition, the JMU Mathematics department now uses a computer-based placement test that is taken by all incoming freshmen. Because many academic programs may consider implementing CBTs to assess their student outcomes, it is important that such decisions be based on a consideration of all of the relevant issues-advantages and liabilities of CBTs to provide guidance regarding their most effective use.
Advantages of CBTs
There are number of reasons why computer-based testing is attractive. First, computer-based testing can be more responsive to the needs of both the test giver and the student. In contexts where "on-demand" testing of examinees is needed, the use of computerized tests does not require scheduled administration times. Also, during testing, examinees can be provided feedback regarding their progress. Such feedback might take the forms of item feedback regarding answer correctness, a running total score, or simply information on how many items are left or how much time is remaining in the testing period.
Moreover, scoring of computerized tests is immediate, which facilitates decisions to be made on the basis of an examinee's test score. An additional advantage of computerized tests is that they can be used to test physically disabled examinees for whom administration of conventional tests is infeasible.
A second--and potentially more exciting--advantage of CBTs is their capability to present items in new, and potentially more realistic, ways. For example, on a conventional physics examination, a complex text description or a series of static diagrams may be required to present an item concerning a particular property of motion. On a CBT, the motion in question might be more simply and clearly depicted using a brief video clip demonstrating the physical motion. A computer-based version of the test item is likely a purer measure of the student's understanding of the physics concept because it is less confounded with other student skills such as reading proficiency. This implies that inferences made regarding student knowledge and skills are potentially more valid under a computer-based format.
Being able to introduce audio and video into test items is a very attractive feature of CBTs. Innovative tests may also create a simulated environment in which students are asked to demonstrate particular proficiencies. For example, a virtual chemistry lab could be set up in which students might be asked show that they know how to "make" a particular compound. Such computer-based performance assessments should be appealing to many academic programs. The computer provides many possibilities for new methods to assess student outcomes, and a great deal of innovation should emerge in the coming years.
A more advanced type of CBT is the computerized adaptive test (CAT). In a CAT, the computer adjusts the characteristics of the administered items to match the proficiency level of each examinee. That is, examinees higher in proficiency will receive a more difficult set of items that those lower in proficiency. Because each examinee receives a unique set of items that is matched to his or her proficiency level, testing is more efficient and can be completed much more quickly. A CAT typically requires about half as many items as a conventional test to attain the same precision of measurement (with commensurate savings in testing time), which has made it very attractive to large-scale testing programs. For instance, the GRE uses a CAT format, as will the SAT when it is offered in a computer-based format.
Disadvantages of CBTs
The disadvantages of computer-based testing are all related to the resources required. Basically, these resources include a sufficient numbers of computers, a room to install them, appropriate software, and adequate technological expertise. The number of computers needed is a function of two factors: the total number of students to be tested and the time frame within which testing must be completed. If it is important, for instance, that 100 students be tested simultaneously, then obviously one would need 100 computers. If, however, those 100 students could be tested over a two-day period, and five test administrations could be conducted each day, then only 10 computers would be needed. Thus, lengthening the time frame can compensate for smaller numbers of computers.
It might be challenging to find a large enough room to contain the computers. Often, a computer laboratory is developed that can be used for other purposes (instruction, student projects, E-mail) when it is not needed for testing. Such a multi-use room is an efficient way for departments to use limited space.
Obtaining good testing software is common obstacle to adoption of computer-based testing. There are two basic choices: either develop your own software or purchase software from a commercial software vendor. If you develop your own, then you can get exactly what you want, but you have to locate and pay a programmer to set it up for you. Commercially-available software is often less expensive (but not cheap), but you may have difficulty finding software that does exactly what you want. A hybrid option is to hire a software company to modify their existing software to meet any particular needs that you might have. As you might expect, however, this is more expensive than purchasing off-the-shelf software.
A final key element to adopting computer-based testing is having available an individual with substantial technological expertise. Such a person will be very useful in (a) planning the testing room, (b) purchasing the most appropriate computers, (c) making decisions involving software acquisition, and (d) maintaining computer hardware and software once CBTs have been implemented.
|
