10/06/2011 1 QTI: IMS QTI: IMS Question Question & Test & Test Interoperability Specification Interoperability Specification Laura Farinetti Laura Farinetti Dipartimento di Automatica e Informatica Politecnico di Torino [email protected]1 IMS Question & Test Interoperability IMS Question & Test Interoperability (QTI) specification (QTI) specification y Describes a data model for the representation of question (assessmentItem) and test (assessmentTest) data and their corresponding results reports y Enables the exchange of this item, test and results data between authoring tools item results data between authoring tools, item banks, test constructional tools, learning systems and assessment delivery systems 2
62
Embed
QTI: IMS Question & Test Interoperability Specification
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
10/06/2011
1
QTI: IMS QTI: IMS QuestionQuestion & Test & Test Interoperability SpecificationInteroperability Specification
Laura FarinettiLaura FarinettiDipartimento di Automatica e InformaticaPolitecnico di Torino
IMS Question & Test Interoperability IMS Question & Test Interoperability (QTI) specification (QTI) specification
Describes a data model for the representation of question (assessmentItem) and test (assessmentTest) data and their corresponding results reportsEnables the exchange of this item, test and results data between authoring tools item results data between authoring tools, item banks, test constructional tools, learning systems and assessment delivery systems
2
10/06/2011
2
IMS Question & Test Interoperability IMS Question & Test Interoperability (QTI) specification (QTI) specification
U XML tUses XML syntaxPedagogy neutralSubject neutralAble to display formatted text, pictures, sound files video clips interactive sound files, video clips, interactive applications or applets
3
IMS Question & Test Interoperability IMS Question & Test Interoperability (QTI) specification (QTI) specification
Supports several types of userSupports several types of userresponse … ◦ item selection, text input, numeric input,
xy-position selection and group selection
… that can be combined with several diff i h i different input techniques ◦ radio button, check box, text entry box, mouse
xy position dragging or clicking, slider bar, …
4
10/06/2011
3
QTI elementsQTI elements
Tests are called “assessments”An assessment has one or more questionsAn assessment has one or more questionsQuestions plus their associated data are called “items” ◦ Data needed to deliver the question, such as
layout rendering information, the score if it is correct, the feedback to be given, …
5
QTI specificationsQTI specifications
AssessmentItem◦ Explains the internal structure of an individual ◦ Explains the internal structure of an individual
item (i.e. a question)◦ Composed of
several sectionsAssessment
Section
Section
6
Item
Item
10/06/2011
4
QTI specificationsQTI specifications
AssessmentTest◦ A test is a group of assessmentItems with an ◦ A test is a group of assessmentItems with an
associated set of rules that determine which of the items the user sees, in what order, and in what way the candidate interacts with them◦ The rules describe the valid paths through the
test, when responses are submitted for test, when responses are submitted for response processing and when (if at all) feedback is to be given
7
QTI specificationsQTI specifications
A test is a logical aggregation of several QTI itemsQTI itemsMoreover, a test contains instructions on◦ how to create groups of items within a test◦ shuffle the sequence of items presented to the
user◦ specify when the candidate may submit
answers for response processing◦ provide feedback to the candidate at
different points within a test 8
10/06/2011
5
QTI specificationsQTI specifications
A test is a logical aggregation of several QTI itemsQTI itemsMoreover, a test contains instructions on◦ how to create groups of items within a test◦ shuffle the sequence of items presented to the
user◦ specify when the candidate may submit
answers for response processing◦ provide feedback to the candidate at
different points within a test 9
Question “anatomy”Question “anatomy”Statement
Stylesheet
Attributes
Choices
Logic for marking (response processing)
Variable declaration
10
processing)
Example 1
10/06/2011
6
Question “anatomy”Question “anatomy”
SAME logic for marking (response processing)
Choices
11
Example 2
Question “anatomy”Question “anatomy”
Feedback
12
Example 3
10/06/2011
7
Question “anatomy”Question “anatomy”
Data generated automaticallyaccording to a logic (template
i )
If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women ?
minutes.2020
2 men 303 men
processing)
13
Example 4
QTI elementsQTI elements
Sections are used to group a series of questions within an assessmentquestions within an assessment◦ E.g. different section for each subtopic, to
know the score obtained foreach section as well as overthe assessment as a whole
At the end of the assessment
Assessment
Section
Section
At the end of the assessmenta result report is generated
14
Item
Item
10/06/2011
8
Item types Item types
Multiple choice SlideTrue falseMultiple responseImage hot spotFill in the blankS l t t t
Drag object, drag targetOrder objectsMatch itemsConnect pointsSelect text
Characteristic of an itemComposed ofComposed of◦ A name predefined by the QTI specification
(e.g. “Title”)◦ A corresponding value (e.g. “Test on set
associative cache”) supplied by the item author
Attribute values are read and written by the systems which process QTI itemsNo additional attribute can be created
17
AttributeAttribute
Set of attributes predefined by the specificationspecification◦ IDENTIFIER - Uniquely identifies item
(mandatory)◦ TITLE - A short descriptive title for the item
(mandatory)LABEL A h t l b l f th it ( ti l)◦ LABEL - A short label for the item (optional)◦ LANGUAGE - QTI supports different
languages (optional)
18
10/06/2011
10
AttributeAttribute◦ ADAPTIVE - Is this item adaptive? (mandatory);
adaptive means that it changes appearance or p g ppscoring (or both) in response to each of the user attempts; otherwise it is a non-adaptive item (the default)◦ TIMEDEPENDENT - Will time taken to respond
to the question affect user score? (mandatory)◦ TOOLNAME - Name of the tool which created
the item (optional)◦ TOOLVERSION - Version of the tool which
Template processing generates cloned itemsA template processing section specifies a sequence of template rules which are evaluated by a delivery engine◦ Template rules generate values for one or more
template variablestemplate variables◦ These values are used inside the ItemBody
section to populate values in the item text
29
Template Template variablesvariables
APEOPLE
If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women?
</templateProcessing> Assign response variable RESPONSE the value (120 / B)
Template processingTemplate processing<itemBody><p> If it takes<printedVariable identifier="A" /><printedVariable identifier="PEOPLE" /><printedVariable identifier="MIN" />minutes to dig a hole, how long would it take
<printedVariable identifier="B" /><printedVariable identifier="PEOPLE" />to dig a similar hole?
The ItemBody section uses these variables to generate a single cloned itemto generate a single cloned item◦ Uses the printedVariable element to insert the
value of a template value at a particular point in the question text
PEOPLE = “women”
37
A = 3B = 6 (note: because A=3, B can only take the value 6 or the value 12)MIN = 40RESPONSE = 20
If it takes 3 women 40 minutes to dig a hole, how long would it take 6 women?
Item BodyItem Body
Contains the text, graphics, media objects and interactions that describe the itemand interactions that describe the itemcontent and information about how it is structuredSpecifies◦ The item as seen on screen ◦ The interaction(s) between the user and the
item
38
10/06/2011
20
InteractionInteraction
Allow the candidate to interact with theitem item ◦ Through an interaction, the user selects or
constructs a response ◦ The user responses are stored in the
responseVariables E h i t ti i i t d ith ( t l t) ◦ Each interaction is associated with (at least) one response variable
39
Internal structureInternal structure
ItemBodyblockInteraction
choiceInteractionmatchInteraction
blockStatic< >
…
40
<p><div>
customInteractionpositionObjectStage
10/06/2011
21
Internal structureInternal structureblockInteraction blocks◦ the most commonly used block type: action types such
lti l h i t hi d lid bas multiple choice, matching gaps and slider bars(11 types)
blockStatic blocks◦ do not support interactions, but specify visual
components such as text, pictures and tablescustomInteraction blocks◦ allow to extend the specification and build new
interaction typespositionObjectStage blocks◦ blocks support images onto which the user can place
<choiceInteraction responseIdentifier="RESPONSE" shuffle="false" maxChoices="1"><prompt>What does it say?</prompt><simpleChoice identifier="ChoiceA">You must stay with
your luggage at all times.</simpleChoice><simpleChoice identifier="ChoiceB">Do not let someone
else look after your luggage.</simpleChoice><simpleChoice identifier "ChoiceC">Remember your
43
<simpleChoice identifier="ChoiceC">Remember your luggage when you leave.</simpleChoice>
</choiceInteraction></itemBody>
One choiceInteraction block
If user selects the second option, the response variable RESPONSE is set to ChoiceB
<prompt>The following F1 drivers finished on the podium in the first ever Grand Prix of Bahrain. Can you rearrange them into the correct finishing order?</prompt>
Else if current value of RESPONSE equals the sequence “DriverC,DriverB,DriverA” thenSet SCORE to 1 (i.e. give partial credit for nearly-correct response)
Offer an alternative to hard wiring logic inside an iteminside an itemTemplates are prewritten segments of response processing which the item authorcan reference inside a responseProcessing element
The item author can specify feedback at different points during user interaction with different points during user interaction with an itemThree types of feedback Modal Feedback: presented to the user immediately after response processing
60
10/06/2011
31
FeedbackFeedback
Integrated feedback: integrated into theitem itemBodyitem itemBody◦ users are free to update their responses while
viewing integrated feedback◦ not dependent on having submitted an answer
for response processing
T F db k d h Test Feedback: presented to the userconditionally based on the value of test Outcomes
61
Modal and integrated feedbackModal and integrated feedback
Integrated feedback
62
Example 3
Modal feedback
10/06/2011
32
Modal and integrated feedbackModal and integrated feedback
63
Modal and integrated feedbackModal and integrated feedback
<responseDeclaration identifier="RESPONSE"
Declaration of the item variables
espo se ec a at o de t e S O Scardinality="single" baseType="identifier">
Attributes of the whole assessmentTest elementelement◦ IDENTIFIER - Uniquely identifies the test
(mandatory)◦ TITLE - Short title of the test (mandatory)◦ TOOLNAME - Name of the tool which created
th t t ( ti l)the test (optional)◦ TOOLVERSION - Version of the tool which
created the test (optional)
71
OutcomeDeclarationOutcomeDeclaration
An assessmentTest element can contain zero one or more outcome declaration zero, one or more outcome declaration elementsAn outcome declaration declares one outcome variable whose scope is the entire test◦ Stores a value relevant to the candidate’s
performance at the test level
72
10/06/2011
37
OutcomeDeclarationOutcomeDeclaration
Other outcome variables are used at the item levelitem level◦ Item outcome variables generally determine
the value of test outcome variables◦ E.g. in a test composed of ten items, the value
of a test outcome variable may be calculated as the sum of the values of ten item outcome as the sum of the values of ten item outcome variables
73
OverallOverall approachapproach toto OutcomeDeclarationOutcomeDeclaration
Declare test outcome variables in the outcome declarations section of assessmentTestdeclarations section of assessmentTestFor each item response submitted by the candidate◦ First execute the item’s response processing (if
declared inside assessmentItem): this will update item’s outcome variablesitem s outcome variables◦ Then execute test’s outcome processing (if
declared inside assessmentTest): this will update test’s outcome variables
p g g q _ pxmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.imsglobal.org/xsd/imsqti_v2p1 imsqti_v2p1.xsd"identifier="RTEST-25"title="Basic Statistics as Outcomes">
Top-level sections that compose anassessmentTestassessmentTestA testPart contains one or more internal subsections calledassessmentSections◦ Allow the test author
assessmentTest
testPart
to disaggregate whatwould otherwise be alinear set of items
77
assessmentSection
assessmentItemRef
TestPartTestPart
An assessmentTest contains one or more testPartstestPartsA testPart contains one or more assessmentSectionsAn assessmentSection may reference one or more assessmentItems, via the assessmentItemRef element and/or contain one or more nested assessmentSections
78
10/06/2011
40
ExampleExample
A QTI test does not physically contain full items It “points to” items by specifying theirexternal filenames
TestPart TestPart -- attributes and elementsattributes and elementsidentifier (mandatory attribute )◦ Uniquely distinguishes an individual testPartUniquely distinguishes an individual testPart
within one assessmentTestnavigationMode (mandatory attribute )◦ determines the sequence in which the
candidate must answer items in the testPart◦ “linear”: the candidate must answer each item
one after the other and may not navigate back to a previously answered item◦ “nonlinear”: the candidate may answer the
items in any sequence
81
TestPart TestPart -- attributes and elementsattributes and elements
submissionMode (mandatory attribute )◦ determines when the candidate is permitted to ◦ determines when the candidate is permitted to
submit item responses◦ “individual”: the candidate submits the
response for one item before answering any other item in the testPart◦ “simultaneous”: responses for all the items in ◦ simultaneous : responses for all the items in
this testPart are submitted for response processing in one batch at the end of the testPart
82
10/06/2011
42
TestPart TestPart -- attributes and elementsattributes and elements
precondition (optional element)◦ evaluated just before a candidate enters a ◦ evaluated just before a candidate enters a
testPart◦ if a precondition expression evaluates to false,
then the associated testPart will not be presented to the candidate◦ more than one precondition can be associated ◦ more than one precondition can be associated
to a testPart◦ the precondition element is also defined in
sectionPart
83
TestPart TestPart -- attributes and elementsattributes and elements
branchRule (optional element)◦ determines “where next?” after a testPart has ◦ determines where next? after a testPart has
been completed by the candidate◦ if a branchRule evaluates to true, either the
candidate will be directed to another nominated testPart or the entire test will be terminated immediatelyterminated immediately◦ more than one branchRule can be associated
with a testPart
84
10/06/2011
43
TestPart TestPart -- attributes and elementsattributes and elements
itemSessionControl (optional element)◦ specifies a list of attributes which affect each ◦ specifies a list of attributes which affect each
item session in the testPart◦ attributes:maxAttempts, showFeedback,
timeLimits (optional element)timeLimits (optional element)◦ specifies a minimum and/or a maximum time
period for the candidate to answer all the items in the testPart
85
TestPart TestPart -- attributes and elementsattributes and elements
testFeedback (optional element)◦ specifies test (not item) level feedback which ◦ specifies test (not item) level feedback, which
can either be displayed during or at the end of a testPart◦ more than one testFeedback element can be
associated with a testPart
Note all the features apply to just one Note: all the features apply to just one testPart◦ different testParts within the same
AssessmentTest can have different features
86
10/06/2011
44
SectionPart SectionPart -- attributes and elementsattributes and elements
Contains attributes and elements which are common to both assessmentSection and common to both assessmentSection and assessmentItemRef ◦ sectionPart stores “a number of common
parameters shared by both types of child element”
id tifi ( d t tt ib t )identifier (mandatory attribute)◦ uniquely distinguishes the section or the item
reference within the overall test
87
SectionPart SectionPart -- attributes and elementsattributes and elements
required (optional attribute)◦ if set to true means this child element mustif set to true means this child element must
appear in the final selectionfixed (optional attribute)◦ relates to item shuffling within a section◦ if set to true, this element must remain in its
original position when the final shuffledg psequence is determined
preCondition (optional element)◦ determines if this item or section will be
skipped88
10/06/2011
45
SectionPart SectionPart -- attributes and elementsattributes and elements
branchRule (optional element)◦ contains rule(s) which identify under certain ◦ contains rule(s) which identify, under certain
circumstances, an alternative item or section
itemSessionControl (optional element)◦ contains parameters to control the allowable
states for each item session
timeLimits (optional element)◦ determines the period of time during which the
candidate may answer this item or this section
89
AssessmentSection AssessmentSection -- attributes and attributes and elementselements
title (mandatory attribute)◦ stores a string title which may or may not be◦ stores a string title,which may or may not be
displayed to the candidate
visible (mandatory attribute)◦ if set to true, the candidate is aware that this is
a distinct section within the overall testPart◦ if set to false, the candidate is not aware of
this section’s existence per se, and all items inside it will appear to belong to an outer section
90
10/06/2011
46
AssessmentSection AssessmentSection -- attributes and attributes and elementselements
keepTogether (optional attribute) applies to invisible sections only to invisible sections only ◦ if set to false, the items in this section may be
mixed with items from a parent section◦ if set to true, the items in this section must
remain en bloc
l i ( i l l )selection (optional element)◦ determines how many child elements may be
chosen by a delivery engine from an assessmentSection
91
AssessmentSection AssessmentSection -- attributes and attributes and elementselements
ordering (optional element)◦ determines the sequence in which the chosen◦ determines the sequence in which the chosen
child elements will be presented to the candidate◦ if the Boolean attribute shuffle is set to true,
the delivery engine will randomize the sequence in which the items are presentedsequence in which the items are presented
92
10/06/2011
47
AssessmentSection AssessmentSection -- attributes and attributes and elementselements
rubricBlock (optional element)◦ determines the introductory text displayed for ◦ determines the introductory text displayed for
this assessmentSection. ◦ the audience of this text is determined by the
attribute view, which can take one or more values from a predefined set (e.g. candidate, author, tutor, … )author, tutor, … )
93
AssessmentItemRef AssessmentItemRef -- attributes and attributes and elementselements
href (mandatory attribute)◦ includes an item in the test◦ includes an item in the test◦ the attribute value should be the URI of the
external file containing the item
category (optional attribute)◦ associates a user-defined category (e.g.
“biology”) with an item◦ these categories can be detected and filtered
within outcome processing
94
10/06/2011
48
AssessmentItemRef AssessmentItemRef -- attributes and attributes and elementselements
variableMapping (optional element)◦ renames within the scope of a test a variable ◦ renames, within the scope of a test, a variable
declared inside an item
weight (optional element)◦ assigns a weighting for this item specifically in
the context of this test
templateDefault (optional element◦ overrides the default value of a template
variable declared by the item based on an expression evaluated at testlevel
href="item063.xml">href item063.xml<!--this item only if the the previous response was identifier N --><preCondition><match><variable identifier="item034.RESPONSE" /><baseValue baseType="identifier">N</baseValue>
TestPart TestPart -- exampleexample...<!--special targetItem means exit this section of the test -->test ><branchRule target="EXIT_SECTION"><not><match><variable identifier="item347.RESPONSE" /><baseValue baseType="identifier">Y</baseValue>
Option to specify the time period available to the candidate for answering questionsto the candidate for answering questionsThe time constraints can be applied to◦ A single item◦ A group of items◦ The whole
test
101
TimeLimitsTimeLimits
For each, the test author may specify atimeLimits element which contains two timeLimits element, which contains two attributes◦ minTime (in number of seconds, integer value)◦ maxTime (in number of seconds, integer value)◦ Either or both of these attributes may be
fspecified
102
10/06/2011
52
TimeLimitsTimeLimitsTimeLimits element of assessmentTest◦ The “total time” covers not only the sum of The total time covers not only the sum of
times taken to answer each item, but also any extra time the candidate spends in navigating the screen to move amongst questions
A delivery engine can keep track of the time taken to complete a test◦ This value is held in a predefined QTI response
variable called “duration”◦ This variable value can be referenced inside
Specifies logic which assigns values to one or more test outcome variablesor more test outcome variablesOutcome processing takes place each time the candidate submits the responses◦ for an item, when in individual submission mode◦ or a group of items, when in simultaneous
submission mode
107
OutcomeProcessingOutcomeProcessing
The values assigned to outcome variables can be used tocan be used to◦ display test-level (not item-level) feedback to
the candidate◦ branch to another section or item within the
OutcomeProcessing OutcomeProcessing -- exampleexample...<outcomeProcessing><setOutcomeValue identifier="NCORRECTA">< b C t ti Id tifi " ti A" /><numberCorrect sectionIdentifier="sectionA" />