Writing Multiple-Choice Tests (Format Suggestions)

Ch. 13, p. 484

Writing Multiple-Choice Tests (Format Suggestions)

Here are some guidelines for constructing multiple-choice items (see Haladyna, 1997):

1. Make the stem sufficiently specific to stand on its own without qualification. In other words, the stem should contain enough information to set the context for the concepts in it. Here is an example of a stem for which insufficient context has been established:

   Applied behavior analysis can be

   1. classical conditioning.
   2. punishment.
   3. reinforcement contingencies.
   4. self-actualization.

   An improved version of this stem is as follows:

   What is a modern classroom use of applied behavior analysis?

   1. classical conditioning.
   2. punishment.
   3. reinforcement contingencies. [correct choice]
   4. self-actualization.
   
   
   
2. Avoid long and complicated stems unless the purpose of the item is to measure a student’s ability to deal with new information or to interpret a paragraph. The stem should not be too wordy; a test is not the place to incorporate instruction that should have been given in the lessons. Writing the item stems as simple sentences in question form often helps to focus them appropriately.
   
   
3. The stem and every choice in the list of potential answers ought to fit grammatically. In addition, phrases or words that would commonly begin each of the alternatives should be part of the stem. It is also a sound idea to have the same grammatical form (say, a verb) at the beginning of each choice. For example:

   The task of statistics is to

   1. make the investigation of human beings more precise and rigorous.
   2. make the social sciences as respectable as the physical sciences.
   3. predict human behavior.
   4. reduce large masses of data to an interpretable form. [correct choice]
   
   
   
4. Take special care in using no-exception words such as “never,” “all,” “none,” and “always.” These words, called specific determiners, are most commonly found in incorrect statements because the admission of no exceptions usually makes statements wrong. In multiple-choice items these words often give clues to the test-wise but concept-ignorant student. Hill (1977) notes also that words allowing qualification, such as often, sometimes, seldom, usually, typically, generally, and ordinarily, are most often found in correct statements (or responses that are true) and, along with the _no-exception words, this type of specific determiner should be avoided whenever possible, or at least distributed among correct answers and distractors.
   
   
5. Avoid making the correct choice the only one that is qualified (e.g., by an “if” clause). Also, it should be neither the longest nor shortest of the alternatives (usually the longest, because absolutely correct answers often require qualification and precision). These features make a choice stand out, called clang. If the choices vary considerably in length, then having at least two short ones and at least two long ones will reduce clang.
   
   
6. Do not allow an item to be answered on the basis of information contained in another item on the same test. This is another form of clang because it allows a student to identify the answer without knowing it beforehand.
   
   
7. Avoid overinclusive options that contain other options. For example, the choices “dogs” and “setters” should not be in the same item since a setter is a type of dog. Similarly, be cautious in using “all of the above” as an alternative, because it also often reduces the possible correct choices to one or two alternatives. Here is an example illustrating how a student might know very little and get the correct answer. By knowing that only one of the choices is incorrect, a student will reduce the number of plausible choices from four to two:

   What type of research is best for investigating the effects of a new instructional program on mathematics achievement?

   1. correlational.
   2. experimental. [correct choice]
   3. historical.
   4. all of the above.

   The student who knows that “historical” is not a good choice also knows that d must be incorrect, and the answer must be a or b.
   
   

8. After a test, discuss the items with students, and note their interpretations of the wording of the items. Students often interpret certain phrases quite differently from the way the teacher intended. Such feedback will help you revise items for the next test, as well as informing you about students’ understandings.
   
   
9. Do not include a choice that is transparently absurd. All choices should seem plausible to a student who has not studied or otherwise become familiar with the subject. Besides these guidelines for writing multiple-choice items, here are some suggestions about format:
   • List the choices vertically rather than side by side.
   • Use letters rather than numerals to label the choices, especially on scientific and mathematical tests.
   • Use word structures that make the stem agree with the choices according to acceptable grammatical practice. For example, a completion-type stem would require that each of the choices begin with a lowercase letter (unless it begins with a proper noun).
   • Avoid repeating the same word or phrase in the stem and in only one alternative.
   • Avoid overusing one letter position as the correct choice, as well as a pattern in the correct answers. Instead, correct choices should appear in random letter positions.