The Secret of Writing Multiple Choice Test Items
Multiple-choice questions (MCQ’s) play an important role in testing and are among the most widely used of selection-type items on examinations today. The popularity of the MCQ stems from the fact that they can be designed to measure a variety of learning outcomes.
It takes a great deal of time, skill, and adherence to a set of well-recognized rules for item construction to develop a good MCQ item.
The Nature of the Multiple-Choice Question (MCQ)
Multiple-choice questions consist of a stem, options (distractors), and a key.
1. Stem – a question or an incomplete statement, which presents the problem. The stem of a MCQ may be presented either as a question or as an incomplete statement. The choice of form makes no difference in the overall effectiveness of the stem, as long as the writer presents a clear and specific problem. An example of the two different types of stem are found below:
Sample MCQ Stem Written as a Question:
Which of the basic food groups is richest in calcium?
A. Breads and grains
* B. Dairy products
C. Fruits and vegetables
D. Meat and poultry
Sample MCQ Stem Written as an Incomplete Statement:
An example of a supply-type test item is the:
A: Matching item
B: Multiple-choice item
C: Short-answer item
* D: True-false item
2. Options (distractors) – the answer choices or options, referred to as distractors, given to provide possible solutions to the problem. They function to distract those individuals who are uncertain of the answer. Typically, there are four options, that consist of:
a. One correct answer (the key), and,
b. Three distractors.
3. Key – the correct or best answer. The key is typically noted with an *.
Rules for Writing Multiple Choice Questions
The goal of the multiple-choice (MCQ) item format is to present students with a task that is both important and clearly understood, and that can be answered correctly by anyone who has achieved the intended learning outcome. There should be nothing in the content or structure of the item that would prevent an informed student from responding correctly. Similarly, nothing in the content or structure of the item should enable an uninformed student to select the correct answer. The following rules for item writing are intended to guide the item developer to design MCQs that function as intended. The rules are divided into two categories: (1) rules for developing the stem and (2) rules for developing the options.
The Stem
1. Present a single, clearly formulated problem in the stem of the item.
The stem should contain only information that is pertinent to the question or problem and should be worded so that the examinee can understand it without reading the options.
Poor: The cell islets of the pancreas:
A. are located around the edge of the pancreas.
B. contain ducts.
C. disappear as one grows older.
* D. produce insulin.
Better: The cell islets of the pancreas secrete the substance called:
A. trypsin.
* B. insulin.
C. tryptophan.
D. adrenaline.
The first example is merely a collection of true-false statements with a common stem. However, the stem in the second example presents a single problem. A good test to check the clarity and completeness of a multiple-choice stem is to cover the alternatives and determine whether it could be answered without the choices.
2. State the stem of the item in simple, clear language.
The problem in the stem of a MCQ should be stated clearly, using straightforward vocabulary and should be free of unnecessary complex wording and sentence structure.
Poor: Cells of one kind belong to a particular group performing a specialized duty. We call this group of cells a tissue. All of us have different kinds of tissues in our bodies. Which of the following would be classified as epithelial tissue?
A. adenoids and tonsils
B. cartilage
* C. mucous membranes
D. tendons
Better: Which of the following would be classified as epithelial tissue?
A. adenoids and tonsils
B. cartilage
* C. mucous membranes
D. tendons
The stem in the first example contains unnecessary material which increases the amount of reading needed for an item, making it more difficult for the examinee to distinguish between relevant and irrelevant material.
3. Put as much of the wording as possible in the stem of the item.
The stem should include as much of the item as possible. If the same words or phrases are repeated in all or most of the options, rewrite the stem to include the repetitious material. Note the following examples:
Poor: In objective testing, the term objective:
A. refers to the method of identifying the learning outcomes.
B. refers to the method of presenting the problem.
C. refers to the method of selecting the test content
* D. refers to the method of scoring the answers.
Better: In objective testing, the term objective refers to the method of:
A. identifying the learning outcomes
B. presenting the problem
C. selecting the test content
* D. scoring the answers.
4. State the stem of the item in positive form, wherever possible.
Avoid using negatives such as NO, NOT, and EXCEPT. If you must use a negative word, write it in capital letters and underline it so that the test-taker will not miss it.
Poor: Which of the following structures of the ear is NOT concerned with hearing?
A. cochlea
B. eardrum
C. oval window
* D. semicircular canals
Better: Which one of the following structures of the ear helps to maintain balance?
A. cochlea
B. eardrum
C. oval window
* D. semicircular canals
Poor: Which one of the following is not a desirable practice when preparing multiple- choice items?
* A. Shortening the stem by lengthening the alternatives.
B. Stating the stem in positive form.
C. Underlining certain words in the stem for emphasis.
D. Using a stem that could function as a short-answer item.
Better: All of the following are desirable practices when preparing multiple-choice items EXCEPT:
* A. shortening the stem by lengthening the alternatives.
B. Stating the item in positive form.
C. underlining certain words in the stem for emphasis.
D. using a stem that could function as a short-answer item.
Please note that when negative wording is used in the stem of an item, not only should it be emphasized by placing it in capital letters and underlining it, but also placed near the end of the statement.
5. Make certain that the intended answer is correct or clearly best.
When the correct-answer form of a MCQ item is used, there should be only one unquestionably correct answer. With the best-answer form, the intended answer should be one that content experts would agree is clearly the best.
Poor: Which of the following types of cattle is a dairy breed?
A. Angus
* B. Guernsey
C. Hereford
* D. Holstein
Better: Which of the following types of cattle is a dairy breed?
A. Angus
B. Brahman
C. Hereford
* D. Holstein
In the first example, both B and D are correct answers. They both are dairy breeds of cattle. In the second example, the only correct answer is D. All the other options are breeds of meat cattle.
The Options
1. Make all options grammatically consistent with the stem of the item and parallel inform.
Be sure to check the options against the wording in the stem to make sure they are grammatically consistent. This will help to avoid easy elimination of options.
Poor: Penicillin is obtained from a:
A. bacteria..
B. coal-tars.
* C. mold.
D. tropical tree.
Better: Penicillin is obtained from:
A. bacteria..
B. coal-tars.
* C. mold.
D. tropical tree.
In first example, a coal-tars is not grammatically correct when the stem and the option is read as a complete sentence.
2. Avoid giving clues in the correct answer by providing common verbal associations to words that are in the stem.
Poor: What is the best way to locate a lawyer for assistance?
* A. Contact a lawyer referral service.
B. Contact a police officer.
C. Look for an ad in the newspaper.
D. Wait until you go to jail.
Better: What is the best way to locate a lawyer for assistance?
A. Ask a police officer.
B. Call a bail bonding agency.
* C. Contact a legal referral service.
D. Look for an ad in the phone book.
Note that in the first example, the word lawyer is in both the stem and the correct answer.
3. Be sure that the wrong answers are plausible.
The distractors must be attractive to examinees who are lacking in knowledge about the material the item is intended to assess. Therefore, the incorrect answer choices should be logically consistent with the stem and should represent common errors made by students at a particular ability level.
Poor: Subtract 8032 – 5743
* A. 2289
B. 2288
C. 2378
D. 3378
Better: Subtract 8032 – 5743
* A. 2289
B. 2389 [failing to change 0 to 9]
C. 3399 [failing to decrease two digits borrowed from]
D. 3711 [subtracting the big number from the small one]
4. Vary the relative length of the correct answer to eliminate length as a clue.
All options should be of approximately the same length. Avoid any tendency to make the correct answer consistently longer or shorter than the distractors.
Poor: One advantage of multiple-choice items over essay questions is that they:
A. depend more on recall.
B. measure more complex outcomes.
* C. provide for a more extensive sampling of course content.
D. require less time to score.
Better: One advantage of multiple-choice items over essay questions is that they:
A. place greater emphasis on the recall of factual information.
* B. provide for a more extensive sampling of course content.
C. provide for the measurement of more complex learning outcomes.
D. require less time for test preparation and scoring.
5. Use the option “none of the above ” only when the keyed answer can be classified unequivocally as correct or incorrect.
The “none of the above” option works better when the stem is stated as a question rather than a sentence to be completed. An incomplete sentence rarely works because “none of the above” seldom completes the stem grammatically. Nevertheless, the use of the “none of the above” option should be avoided.
6. Avoid the use of “all of the above “.
The “all of the above” option makes it possible to answer the item on the basis of partial information. Since the examinee is to select only one answer, they can detect “all of the above” as the correct choice simply by noting that two of the alternatives are correct. They can also detect it as a wrong answer by recognizing that at least one of the alternatives is incorrect, thus enhancing their chance of guessing the correct answer from the remaining choices.
Poor: Which of the following factors must be considered in computing basal energy requirements?
A. age
B. height
C. weight
*D. all of the above
Better: Which of the following factors must be considered in computing basal energy requirements:
A. age only
B. weight only
C. height and weight only
*D. age, height, and weight
7. Alphabetize the distractors and arrange numerical options in ascending order.
To ensure that the placement of the correct answer, or key, does not follow a pattern that may be apparent to the examinee, the options should be listed in alphabetical order. If the options are numerical, then the options should be listed in ascending order.
Poor: A communication system for the blind that includes raised dots that are arranged to represent letters of the alphabet is called:
* A. Braille
B. Sign language
C. Morse code
D. Arabic lettering
Better: A communication system for the blind that includes raised dots that are arranged to represent letters of the alphabet is called:
A. Arabic lettering
*B. Braille
C. Morse code
D. Sign language
Poor: How long should an orphan calf be fed warm whole milk?
A. 14 days
B. 21 days
* C. 10 days
D. 28 days
Better: How long should an orphan calf be fed warm whole milk?
* A. 10 days
B. 14 days
C. 21 days
D. 28 days
Every organization is different and not all processes/steps need to necessarily be followed. Two things to consider when developing new items are that it is a very expensive process and it is difficult to find qualified editors.
Security is always an issue when developing new questions. The following security tips should be considered:
Limit access to questions to only specified developers/reviewers
Limit the paper trail and do work electronically
Have everyone sign a privacy and confidentiality agreement
Follow a defined workflow process
About The Author
I am an Assessment Specialist with the Oklahoma Department of Career and Technology Education. We create and maintain skills standards and online competency tests for over 100 career majors.
Copyright 2009
Oklahoma Department of Career and Technology Education
All right reserved
The author invites you to visit: http://careertechtesting.blogspot.com/
