Workshop Objectives
List MCQ structures, benefits, issues Develop good writing tips Analyze some MCQ problems Investigate guess-work
Which of the following is a physical change?
a. rusting of a nail b. burning of wood *c. freezing of water d. ignition of gasoline
The parts of a multiple choice question
Advantages of well designed Multiple-Choice questions
can measure all levels of cognitive ability simplify test taking/grading allowing wider
sampling than essay questions machine scoring, efficient and accurate versatile: appropriate for use in many
different subject-matter areas objective measurement - no human
judgment in the scoring (huh?) test banks available from publishers
Knowledge and Comprehension
Application and Analysis
Synthesis and Evaluation
Objective: Identifies the effect of changing a parameter (rule using).
A pendulum consists of a sphere hanging from a string. What will happen to the period of the pendulum if the mass of the sphere is doubled? (Assume that the effects of air friction and the mass of the string are negligible, and that the sphere traces an arc of 20° in a plane as it swings.)
a. It will increase.b. It will decrease.*c. It will remain unchanged.d. More information is needed to determine what will happen.
Measuring COMPREHENSION with a MCQ
Objective: Identifies the correct application of principle (problem solving).
In the diagram above, parallel light rays pass through a convex lens and converge to a focus. They can be made parallel again by placing a:
a. Concave lens at point B.b. Concave lens at point C.c. Second convex lens at point A.d. Second convex lens at point B.*e. Second convex lens at point C.
Measuring APPLICATION with a MCQ
Objective: Analyzes poetry and identifies patterns and relationships.
[The poem is included here.]
The chief purpose of stanza 9 is to:
a. Delay the ending to make the poem symmetrical.b. Give the reader a realistic picture of the return of the cavalry.c. Provide material for extending the simile of the bridge to a final point.*d. Return the reader to the scene established in stanza 1.
Measuring ANALYSIS with a MCQ
Limitations of Multiple-Choice questions
difficult and time consuming to construct can favor simple recall of facts place a high degree of dependence on the
student’s reading comprehension and the instructor’s writing ability
can be easier than open-ended questions asking the same thing… it is harder to recall an answer than to recognize it.
diagnosis of learning problems hard
Why did the student select the wrong answer?
Writing Stems
Write in a direct, positive format.• Minimize use of negatives (capitalize if needed, i.e.
NOT)
Place the bulk of the problem, information or situation in the stem.
Use complete sentences or questions in the stem.
A question is better than an open ended phrase.
The Leaning Tower of Pisa …a. will fall over in 100 years.b. was built on weak soil.c. was the site of Galileo’s experiment with gravity.
Why will the Leaning Tower of Pisa fall over?a. Because it is getting very old.b. Because it was built on weak soil.c. Because of Galileo’s gravity experiments from the tower.
Writing Stems
Stem should not pose more than one problem, although the solution to that problem may require more than one step
Clarity of language is essential• Eliminate unnecessary words or extra info
• Ensure grammar is correct
• "All of the above" NOT good, one recognized distracter eliminates it.
• Review your test with peers
The French flag has three colours. One of them is...a.Redb.Blackc.Green
Which of the following is a colour on the French flag?a. Redb. Blackc. Green
Writing Stems
Add complexity • Present several interconnected concepts in
the initial problem or situation
• Present a case
• Present a cause and effect situation
Require differentiation through analysis• i.e. Which of the following are correct?
STEM with interconnected concepts:
McGregor’s economic theory on emerging markets applies to Vietnam’s monetary exchange policy in the following way.
CONCEPT 1
CONCEPT 2
Which is the better stem?
Alloys are ordinarily produced by ...
How are alloys ordinarily produced?
When possible, state the stem as a direct question rather than as an incomplete statement
Which is the better stem?
Psychology...
The science of mind and behavior is called ...
Present a definite, explicit and singular question or problem in the stem
Which is a better stem?
While ironing her formal, Jane burned her hand accidentally on the hot iron. This was due to a transfer of heat by...
Which of the following ways of heat transfer explains why Jane's hand was burned after she touched a hot iron?
Eliminate excessive verbiage or irrelevant information in the stem
Which is a better stem? In Canada the leader of a political party is officially chosen by a. the Canadian peopleb. members of parliamentc. the Senate*d. the party delegates at a convention
In Canada the leader of a political party is officiallya. chosen by the Canadian peopleb. chosen by members of parliament c. chosen by the Senate *d. chosen by the party delegates at a convention
Include in the stem any word(s) that might otherwise be repeated in each alternative
Which is a better stem?
Which of the following is not cited as an accomplishment of the Mulroney government?
Which of the following is NOT cited as an accomplishment of the Mulroney government?
Use negatively stated stems sparingly. When used, underline and/or capitalize the negative word
Is this a good stem?
What led to the formation of the Provincial Rights Party?
a. The level of federal taxation b. The demand of provinces for the right
to make their own laws c. The industrialization of the Ontario d. The corruption of federal legislators on
the issue of provincial taxation
Writing alternatives
Write the correct answer first, then the alternatives
How many alternatives? Avoid copying phrases from your
textbook “Never”, “always”, “all of the above”,
“none of the above” and multiples (a,c and d) – be careful!
You will never know if students know the correct answer!
Writing alternatives
Plausible alternatives reflect• Errors in logic
• Common mistakes Use parallel sentence structures (and/or
homogeneous phrases - using same categories)
Make alternatives internally consistent• Similar in length if possible
Make all alternatives concise
Writing alternatives
Include key words from the correct answer in several alternatives
Organize alternatives in a logical order (shortest to longest, least to most, etc)
Include more than 1 decision-making factor in each alternative
Avoid presenting polar opposites
Writing alternatives
Present a true answer that does not answer the question
Use correct terms incorrectly Use correct answers from previous in-
class tests or practice quizzes as incorrect alternatives
Which alternatives are better?
What led to the crash of flight ET109? a. Faulty fan blade b. Failed electrical system *c. Bird impact d. Lightning strike
What led to the crash of flight ET109? a. Faulty fan blade b. Failed electrical system *c. Bird impact d. Passenger list errors
Make all alternatives plausible and attractive to the less knowledgeable or skillful student
Which alternatives are better? What would do most to advance the application of atomic discoveries
to medicine? *a. Standardized techniques for treatment of patients. b. Train the average doctor to apply radioactive treatments. c. Remove the restriction on the use of radioactive substances. d. Establishing hospitals staffed by highly trained radioactive therapy
specialists.
What would do most to advance the application of atomic discoveries to medicine?
*a. Development of standardized techniques for treatment of patients. b. Training of the average doctor in application of radioactive treatments. c. Removal of restriction on the use of radioactive substances. d. Addition of trained radioactive therapy specialists to hospital staffs.
Make alternatives grammatically parallel with each other, and consistent with the stem
Which alternatives are better? The daily minimum required amount of milk that a 10 year old
child should drink is a. 1-2 glasses. *b. 2-3 glasses. *c. 3-4 glasses. d. at least 4 glasses.
What is the daily minimum required amount of milk a 10 year old child should drink?
a.1 glass. b. 2 glasses. *c. 3 glasses. d. 4 glasses.
Make the alternatives mutually exclusive
Which alternatives are better? At 7 a.m. two trucks leave a diner and travel north. One truck averages 42 miles per
hour and the other truck averages 38 miles per hour. At what time will they be 24 miles apart?
a. 6 p.m. b. 9 p.m. c. 1 a.m. *d. 1 p.m. e. 6 a.m.
At 7 a.m. two trucks leave a diner and travel north. One truck averages 42 miles per hour and the other truck averages 38 miles per hour. At what time will they be 24 miles apart?
a. 1 a.m. b. 6 a.m. c. 9 a.m. *d. 1 p.m. e. 6 p.m.
When possible, present alternatives in some logical order (e.g., chronological, most to least, alphabetical)
Which alternatives are better?
The two key problems that led to the 2008 economic collapse were toxicdebt instruments and
a. banker’s excessive bonuses. *b. the credit crisis. c. international debt. d. currency fluctuations.
The two key problems that led to the 2008 economic collapse were toxicdebt instruments and
a. banker’s excessive bonuses. *b. the credit crisis. c. international debt. d. failure of inter-bank lending.
Be sure there is only one correct response to the item
Which alternatives are better?
The most general cause of low individual incomes in Canada is *a. lack of valuable productive services to sell. b. unwillingness to work. c. automation. d. inflation.
What is the most general cause of low individual incomes in Canada? *a. Lack of valuable productive services to sell.b. The population's overall unwillingness to work.c. The nation's increased reliance on automation.d. An increasing national level of inflation.
Make alternatives approximately equal in length
Instructions
Give Clear InstructionsQuestions 1 - 10 are multiple-choice questions designed to assess your ability to remember or recall basic and foundational pieces of knowledge related to this course. Please read each question carefully before reading the answer options. When you have a clear idea of the question, find your answer and mark your selection on the answer sheet. Please do not make any other marks on this exam sheet.
Questions 11 – 20 are multiple-choice questions designed to assess your ability to think critically about the subject. Please read each question carefully before reading the answer options. Be aware that some questions may seem to have more than one right answer, but you are to look for the one that makes the most sense and is the most correct. When you have a clear idea of the question, find your answer and mark yourselection on the answer sheet.
What & why
How
“The Art of Guesswork”
READ, ELIMINATE, RE-READ• Read carefully
• Slow down at words which link by causation such as "due to" and "because" or "as a result of' and at words of totality such as "never" or "always."
• Eliminate wrong choices
• Do not jump to the answer that you think is correct.
• Re-read the question, as if you were reading it for the first time. Now choose your answer from your remaining answers based on this re-reading.
Guesswork
Rule of thumb: "Pick the longest answer.“
Way to defeat this strategy: make sure the longest answer is right about a fifth of the time (if there are five alternatives for each question)
Guesswork
Rule of thumb: "Pick the B or C alternative.“ Way to defeat this strategy: make sure each
answer is used the same number of times, in random order.
Guesswork
Rule of thumb: "Never pick an answer which uses the word 'always' or 'never' in it.“
Way to defeat this strategy: make sure such answers are correct about a fifth of the time
Guesswork
Rule of thumb: "If there are two answers which express opposites, pick one or the other and ignore other alternatives.”
Way to defeat this strategy: sometimes offer opposites when neither is correct.
Guesswork
Rule of thumb: "If in doubt, guess.“ Way to minimize the impact of this
strategy: use five alternatives instead of three or four
Guesswork
Rule of thumb: "Pick the scientific-sounding answer.”
Way to defeat this strategy: use scientific sounding jargon in wrong answers
Guesswork
Rule of thumb: "Don't pick an answer which is too simple or obvious.“
Way to defeat this strategy: sometimes make the simple, obvious answer the correct one.
Guesswork
Rule of thumb: "Pick a word which you remember was related to the topic.“
Way to defeat this strategy: when drawing up distracters use terminology from the same area of the text as the right answer, but in distracters use those words incorrectly so the wrong answers are definitely wrong.
Practice
Form teams of two or three Write 2 versions of a MCQ (any subject):
• Problematic version
• Ideal version
Present to class and explain