Question Writing Guide ABNM SAM Questions
Post on 24-Feb-2016
61 Views
Preview:
DESCRIPTION
Transcript
Question Writing GuideABNM SAM Questions
J. Anthony Parker, MD PhDAssociate Executive Director, ABNM
Beth Israel Deaconess Medical CenterBoston, Massachusetts
Tony_Parker@BIDMC.Harvard.edu
Caveat Lector
PsychometricsOrigin for Questions Writing Rules
Psychometrics is the field of study concerned with the theory and technique of psychological measurement, which includes the measurement of knowledge, abilities, attitudes, personality traits, and educational measurement. The field is primarily concerned with the construction and validation of measurement instruments such as questionnaires, tests, and personality assessments.
– Wikipedia
Test Goal is to Measure AbilityLogistic regression where the probability of a correctanswer is a function of
Fn, the ability of person, nbi, the difficulty of question, i
Prob
abili
ty
Ability of person , n, Fn
Difficulty of question, i, bi
Ability 0.3
Wikipedia
Can Also Measure Question DifficultyLogistic regression where the probability of a correctanswer is a function of
Fn, the ability of person, nbi, the difficulty of question, i
Prob
abili
ty
Difficulty of question, i, bi
Ability of person , n, Fn
Difficulty 0.3
Wikipedia
Question DiscriminationThe slope determines how well a questiondiscriminates between candidate abilities
ai, ability discriminationlog-odds correct answer = ai • (Fn - bi)
Wikipedia
Prob
abili
ty
Difficulty of question, i, bi
Ability of person , n, Fn = 0.3
Difficulty 0.3
Slope, aiDiscrimination
ABNM Procedure
Assume questions are good and correctly keyedMeasure candidate tentative abilityUsing tentative abilities measure question difficulty
Discard questions that are too hard or too easyAlso measure question-ability discrimination
Correct wrong keysDiscard questions that not correlated with ability
Using improved test, measure final candidate abilityAdjust to standard independent of candidates and test
Writing Good Questions:Questions that Correlate with Ability
Psychometricians have developed rulesfor writing good questions
Rules are based on question performanceBad questions test abilities unrelated to
the test goal
Some Rules for Good Questions
No negatives (not, except, none)No absolutes (always, never, all, every, only, must)No overlapping rangesNo “all of the above” or “none of the above”No mutually exclusive answers (one is correct)Logical order (numerical, alphabetical, category)Single concept per question
No multiple true / false questions
Which of the following is truein Nuclear Medicine?
MIBG is used to image somatostatin receptorsA photon is completely absorbed during a
Compton interaction18F-FDG is transported into cells by GLUTLymphoma is never FDG-avid
Multiple true / false “which is true?”Multiple unrelated concepts“Never” is an absolute
With regard to the thyroid which is correct?
Pentetreotide is taken up in the parafollicular cellsFollicular cells are non-iodine-avidStromal cells take up iodidePentetreotide is not taken by the thyroid
Multiple true / false “which is correct?”First and last answers are mutually exclusive
One of them must be correct“Non” and “not” are negatives
In the thyroid, pentetreotide istaken up MOST avidly in:
parafollicular cells (C cells)follicular cellscolloidstroma
In the thyroid, pentetreotide istaken up MOST avidly in:
parafollicular cells (C cells)follicular cellscolloidstroma
Which of the following is not correct?
11C-acetate is metabolized in the the TCA cycle11C-choline can be used for prostate cancer18F-fluorodexoyglucose (FDG) is not
phosphorylated in the cell18F-fluorothymidine (FLT) is correlated with cellular
proliferation
Not in stem and in 3rd answerMultiple true / false “which is correct?”Multiple unrelated concepts
MOST closely correlatedwith proliferation?
11C-acetate11C-choline18F-fluorodexoyglucose (FDG)18F-fluorothymidine (FLT)
MOST closely correlatedwith proliferation?
Good distractors are often partially true,but not the best answer
11C-acetate11C-choline18F-fluorodexoyglucose (FDG)18F-fluorothymidine (FLT)
What is the BEST administered dose to use for a 18F-FDG PET pediatric scan?
≤ 1 MBq/kg (≤ 0.03 mCi/kg)1 - 4 MBq/kg (0.03 - 0.11 mCi/kg)4 - 16 MBq/kg (0.11 - 0.43 mCi/kg)10 - 64 MBq/kg (0.27 - 1.73 mCi/kg)
Overlapping ranges4-16 and 10-64 both include 10-161-4 and 4-16 both include 4
What is the BEST administered dose to use for a 18F-FDG PET pediatric scan?
1 MBq/kg (0.03 mCi/kg)4 MBq/kg (0.11 mCi/kg)16 MBq/kg (0.43 mCi/kg)64 MBq/kg (1.73 mCi/kg)
What is the BEST administered dose to use for a 18F-FDG PET pediatric scan?
1 MBq/kg (0.03 mCi/kg)4 MBq/kg (0.11 mCi/kg)16 MBq/kg (0.43 mCi/kg)64 MBq/kg (1.73 mCi/kg)
Gelfand: Pediatric radiopharmaceutical administeredDoses: 2010 north american consensus guidelines.J Nucl Med 2011;52:318
Parallel Answers / Avoid Teaching
Answers should be parallel in constructionand length
Good test takers know that “teachers like to teach”The correct answer is often the longest
Tendency to make correct answer exactly correctFor testing, avoid teaching
Urinary excretion of FDG is related to:
FDG concentration overwhelms GLUTrenal GLUT has low affinity for FDGboth of the proximal and distal convoluted tubule
sodium glucose transporters (SGLT1 & SGLT2respectively) have low affinity for FDG
there is little hexokinase in the kidney
Asymmetric answersTeaching
Urinary excretion of FDG is related to:
FDG concentration overwhelms GLUTrenal GLUT has low affinity for FDGSGLT1/2 have low affinity for FDGthere is little hexokinase in the kidney
Urinary excretion of FDG is related to:
FDG concentration overwhelms GLUTrenal GLUT has low affinity for FDGSGLT1/2 have low affinity for FDGthere is little hexokinase in the kidney
Glucose Reuptake in the Kidney
Stem (Question)
1. Ask a questionGiven …, what is the MOST likely diagnosis?
2. Give and incomplete statementGiven …, the BEST next step is:
3. Describe a situation followed #1 or #2A one-year-old girl had jaundice and a5 x 6 cm mass in the porta. Total bilirubinis 7 mg/dL. …
Stem (Question)
Put most of the content in stemAnswers short
Move common words in answers to stemRemove unnecessary words or sentences
In a 73-year-old man with metastaticmelanoma, what is the MOST likely causeof the FDG uptake shown in the figure?
prostate cancermelanomanormal bladdernormal urethra
In a 73-year-old man with metastaticmelanoma, what is the MOST likely causeof the FDG uptake shown in the figure?
prostate cancermelanomanormal bladdernormal urethra
Distractors (Incorrect Answers)
Should be plausibleLogical misconceptionsCommon misunderstandingsReasonable
not the BEST answernot the MOST likely answer
Testing vs. TeachingWright Map
candidates itemsmore able
less able
hard
easy
Testingdistinguish candidatesdifficult itemsjudgment requiredlow passing %
Teachinghelp learningeasier itemsemphasize key points
Question Taxonomy
Recall (fact)The energy of 99mTc is:
Interpretive (diagnosis)A patient with … had an FDG-PET/CT shownin the figure. The MOST likely diagnosis is:
Problem solving (management, judgment)A patient with … had an FDG-PET/CT shownin the figure. The BEST next step is:
Higher Taxonomy Preferred
Often involve more advanced reasoningMore complex medical taskStill single clinical concept
Distractors may be easier to writeSome topics require lower taxonomy questions
Stable iodine (SSKI or Lugol’s solution) should be administered prior to:
pentetreotide (Octreoscan®)iobenguane (MIBG)a contrast FDG-PET/CTibritumomab tiuxetan (Zevalin®)
Stable iodine (SSKI or Lugol’s solution) should be administered prior to:
pentetreotide (Octreoscan®)iobenguane (MIBG)a contrast FDG-PET/CTibritumomab tiuxetan (Zevalin®)
Recall questionpentetreotide confused with MIBG
ibritumomab tiuxetan confused with tositumomab
Based on the electrocardiogram, whatis the BEST diagnosis?
Difficult to come up with 4 plausible diagnoses
A 45 y/o diabetic man with weakness and shortness of breath for 3 days presents for exercise MPI. The pre-
exercise ECG is shown.
What should be done next?
Obtain right precordial leads.Use dipyridamole stress instead.Accompany the patient to the EW.Proceed as planned.
What should be done next?
Obtain right precordial leads.Use dipyridamole stress instead.Accompany the patient to the EW.Proceed as planned.
Writing Distractors
Difficulty writing reasonable distractorsChange taxonomy
Management distractors often easier to write
The BEST diagnosis is:
The BEST diagnosis is:
normal thymusmediastinal metastases??
Diagnostic question
The uptake in the chest is related to:
The uptake in the chest is related to:
the Na / I symportersomatostatin receptor, type 2brown fatmetastases
Diagnosis & biodistribution
A syringe of 90Y-ibritumomab tiuxetan (Zevalin ®) measuring 1025 MBq (27.7 mCi) needs to be discarded due to QC failure. After transfer to a vial, measurements are vial: 815 MBq (22.0 mCi), syringe: 15 MBq
(0.4 mCi). What should be done next?
Declare a medical eventSurvey the areaCall the radiation safety officer (RSO)Store for decay
A syringe of 90Y-ibritumomab tiuxetan (Zevalin ®) measuring 1025 MBq (27.7 mCi) needs to be discarded due to QC failure. After transfer to a vial, measurements are vial: 815 MBq (22.0 mCi), syringe: 15 MBq
(0.4 mCi). What should be done next?
Declare a medical eventSurvey the areaCall the radiation safety officer (RSO)Store for decay
Which Isotope is MOST Difficult toAccurately Measure in a Dose Calibrator?
18F90Y99mTc111In
Which Isotope is MOST Difficult toAccurately Measure in a Dose Calibrator?
18F90Y99mTc111In
What radiopharmaceutical was administered?
99mTc-pertechnetate67Ga-citrate111In-pentetreotide123I-MIBG
What radiopharmaceutical was administered?
99mTc-pertechnetate67Ga-citrate111In-pentetreotide123I-MIBG
What’s the Radiopharmaceutical?Stupid Questions, but I Use Them
Should knownormal and common variant biodistributionat organ, cell, and receptor level
To be able toidentify abnormal biodistributionidentify radiopharmaceutical QCunderstand unusual findings
After injection of 111In-DTPA in a patientwith normal renal function?
the half-time of excretion is 16 minutesthe half-time of excretion 8 minutesthe half-time of excretion 1.7 hoursthe half-time of excretion 36 minutes
Repeated words in answers
How long after injection of 111In-DTPA is 1/2 excreted with normal renal function?
16 minutes8 minutes1.7 hours36 minutes
Answers not in logical order
How long after injection of 111In-DTPA is 1/2 excreted with normal renal function?
8 minutes16 minutes36 minutes1.7 hours
How long after injection of 111In-DTPA is 1/2 excreted with normal renal function?
8 minutes16 minutes36 minutes1.7 hours
http://www.doseinfo-radar.com/
Renogram
Some Rules for Good Questions (1/2)
No negatives (not, except, none)No absolutes (always, never, all, every, only, must)No overlapping rangesNo “all of the above” or “none of the above”No mutually exclusive answers (one is correct)Logical order (numerical, alphabetical, category)Symmetric answers
Some Rules for Good Questions (2/2)
Put concepts and common words in stemDistractors should be reasonableTry to use higher taxonomy
Often easier to write distractorsSome fact questions are appropriate
Single concept per questionNo multiple true / false questionsMedically, not question, complexity
Reference Guidelines
3-page guideABNM_Well_Formatted_SAM_Question.docx
1-page list of does and don’tsABNM_QuestionWritingDoesAndDonts.docx
26-page guide (ABNM psychometric consultants)http://www.measurementresearch.com/
24-page guideABR_Item_Writers_Guide.pdf
Applause
top related