|
Welcome to Adaptex Assessment System
The use of computers in the assessment of student proficiency is becoming increasingly common. A number of computer-based testing programs have been established and important decisions are being made about examinees on the basis of these tests. A highly visible example of a CBT is the Educational Testing Service’s Graduate Record Examination (GRE), which is taken by thousands of examinees each year. In addition, the number of computer-based licensure and certification tests used in many professional fields has been steadily increasing.
A general objective of virtually any assessment testing is the measurement of what examinees know and can do. For decades, conventional paper-and-pencil tests have been used in the assessment of student outcomes. Assessment professionals are beginning to appreciate, however, how computers can be effectively used to enhance and improve our assessment practices.
Adaptex Overview
The Adaptex Assessment System was developed to provide a computer-based testing system that colleges and universities can use in their student outcomes assessment programs. The development of Adaptex was intended to help remove an obstacle to the adoption of computers-based tests (CBTs) in higher education.
The design of any computer software requires an array of decisions about its capabilities. Often, such decisions require balancing a desire to enhance the software’s capabilities against maintaining its ease of use. Many such decisions were required during the design of Adaptex. To guide our decision-making, we adopted the following principles to guide our software development efforts:
- The software should be easy to learn to use by test developers.
- The software should allow test developers to readily create and store banks of multiple-choice test items.
- The software should allow test developers to easily incorporate multimedia files (e.g., audio, video) into their test items.
- There are many ways to administer CBTs. The software should provide test developers the flexibility to administer a variety of types of computer-based tests, corresponding to the diverse array of test types that are used in assessment practice.
- The software should be developed in a way that allows test developers to efficiently use other software that is likely already on their computers in developing test items.
Thus, our intent was to create a powerful, yet user-friendly testing software package that would allow test developers to easily create multiple-choice items banks and then administer CBTs containing desired testing features.
At this point, a disclaimer is in order. Although we have created Adaptex to be valuable to many assessment professionals, we make no claims that it will serve all of one's computer-based assessment needs. In any assessment context, a variety of assessment methods may be appropriate that do not require an examinee to select a response from a set of alternatives (e.g., multiple-choice). For example, constructed response items (e.g., performance assessments, written essays) might be the most effective way to assess a particular learning objective. Our point is that no one software package would be able to meet all of an assessment professional’s needs. One should view Adaptex as a tool that can be a useful part of the collective strategy that an assessment professional uses.
The Adaptex Components
The Adaptex Assessment System consists of four basic components:
- Item Banking Module
- Test Specification Module
- Test Administration Module (both server-based and web-based)
- Test Results Module
Collectively, these components allow test developers to create and modify item banks, design multiple choice tests with a variety of features, and administer those tests to examinees on stand-alone computers, in server-based labs, or as web based tests. In addition, test developers will be able to access results for the examinees taking the tests.
Adaptex uses Microsoft Access databases to store item bank information, test specifications, and examinee results. Access databases were used because of the near-ubiquity of Microsoft Office at higher education institutions. This implies that those assessment professionals who download and install Adaptex are highly likely to already have the needed database software on their computers. This eliminates the need for users of Adaptex to acquire additional software for its use. Our assumption, therefore, is that the computers used for building test items banks, developing tests, and administering tests each have Microsoft Access software installed on them.
Item Bank Module
In the item banking module, which is described in detail in Chapter 4, test developers can set up banks of multiple-choice items that can be used to construct particular tests. These items can include graphics, multimedia clips (both audio and video), and reading passages. In addition, users can enter item text information in several ways: by typing the stem and response options, by cutting and pasting the stem and response options from a word-processed document, or by creating an image (e.g., JPEG) of the intact item. The test developer can store a variety of statistical information about each item (both classical and item response theory statistics). Items specified as ¡§unscored,¡¨ will be administered and the responses to these items stored, but they will not count toward the examinee’s final score. This feature will be useful for embedding pilot tested items into operational tests. See figure 1.
Figure 1. Item Bank screen capture.
Test Specification Module
In planning a test, a number of decisions must be made regarding exactly how the test will be administered to an examinee. In the test specification module of Adaptex, test developers provide the specific information needed for their particular tests. This includes information such as which item bank will be used, which items drawn from the bank, the order in which they will be administered (including adaptive test administration), the scoring method to be used, whether item review or item feedback will be provided, whether a time limit will be used (and whether the time remaining will be shown to the examinee), and whether a cutoff score will be used to judge an examinee¡¦s score (along with appropriate messages for those passing or failing the test). See figure 2.
Figure 2. Test Specification screen capture.
Test Administration Module
One of the decisions we faced when designing Adaptex concerned how its tests should be delivered to examinees. We decided to provide two different methods. In the first, Adaptex can be used as a stand-alone application that is also suitable in a server-based laboratory environment. In addition, we have developed a web-based test administrator that has all of the capabilities of the first method, with the exception that large multimedia files cannot be used. See figure 3.
We have tested the performance of the two test administration methods. The stand-alone application method has worked well in a testing lab that contains 100 computers. The web-based test administrator has performed well with up to many simultaneous users.
Figure 3. Test Administration screen capture.
Adaptex/2his47.rm
Test Results Module
Once Adaptex is used to administer tests to examinees, the results are stored in a Results database. Test users can either inspect this Microsoft Access database directly, or use the Test Results Module to extract and display the desired results information. The information stored in the database includes examinee name and ID, the date and time of a test administration, the responses given to each item, whether those responses were correct or incorrect, final score, and the student response times for each item.
Last Modified 09/30/2004
|
