Constraint-based tutoring CPI 494 Feb 17, 2009 Kurt VanLehn ASU.
Post on 19-Dec-2015
214 Views
Preview:
Transcript
Constraint-based tutoring
CPI 494 Feb 17, 2009Kurt VanLehn
ASU
Outline
• SQL-Tutor (today)• ER-Tutor & Normit (next class)• Technical details (next class)• Evaluation & Dissemination (next class)
Framework for comparing step-based tutoring systems
Feature Constraint based Cognitive
Step granularity & user interface
Interpreting a student’s step
Suggesting good steps
Feedback and hints on steps
Task selection
Assessment
Evaluations
Dissemination
In class-exercises• Install SQL tutor• Try at least 3 problems• Check our your student model periodically• Try different levels of help• Try different ways of expressing the same step• Try different ordering of steps• What steps does it suggest?• Try to game it• Try fancy ways to say simple things: E.g., instead of
referring to Stanley Kubrick by name, retrieve the director of “Clockwork Orange”
Step granularity & user interface
• Type of user interface? Canvas, forms (i.e., goal boxes), text, simulation based?
• Learnability? Every problem the same UI? • In general, what are the steps? The Answer?• On the problem “Display the titles of movies
that are comedies or dramas”, what are the steps?– Would a finer granularity make sense? For who?
Framework for comparing step-based tutoring systems
Feature Constraint based Cognitive
Step granularity & user interface Large, fixed-size goal boxes Goal boxes, granularity varies
Interpreting a student’s step
Suggesting good steps
Feedback and hints on steps
Task selection
Assessment
Evaluations
Dissemination
Interpreting student steps: How does tutor know what is correct?• Is it example-based or knowledge-based or
both?• “Constraint-based modeling (CBM) is…not
interested in the exact sequence of states in the problem space…”
• “CBM does not require an executable domain model, and is applicable in situations in which such a model would be difficult to construct.”
• “CBM eliminates the need for bug libraries”
A syntactic constraint
• Msg: "The SELECT clause is a mandatory one. Specify the attributes/expressions to retrieve from the database.“
• Clause applied to: SELECT• Relevance condition: t• Satisfaction condition: (not (null (select-clause ss)))
A semantic constraint• Msg: "Check whether you should have ascending or descending
order!“• Clause: “ORDER BY“• Relevance: (and (not (null (order-by is)))
(not (null (order-by ss)))(bind-all ’?n (names (order-by ss)) bindings)(match ’(?*d1 ?n "DESC" ?*d2) (order-by ss) bindings)(not (qualified-name ’?n)))
• Satisfaction: (match ’(?*d3 ?n "DESC" ?*d4) (order-by is) bindings)
• Comment: “verifies that if the results of a query are sorted in decreasing order in the ideal solution (is), the the student’s solution (ss) uses the same ordering.”
Interpreting a student’s step:Details of the matching
• How is an individual step interpreted?• Spaces? Capitalization? • Ordering of conjuncts and disjuncts?• Ordering of steps?• Fancy ways to say simple things? E.g., instead
of referring to Stanley Kubrick by name, retrieve the director of “Clockwork Orange”
Framework for comparing step-based tutoring systems
Feature Constraint based Cognitive
Step loop: Granularity & user interface Large, fixed-size goal boxes
Goal boxes, granularity varies
Step loop: Interpreting student steps Constraints + ideal solution; CB matching
EB or KB; Exact, intervals, regular expressions
Step loop: Suggesting good steps
Step loop: Feedback and hints
Task selection
Assessment
Evaluations
Dissemination
Suggesting good steps
• What does its method/ordering appear to be?• What if there are multiple correct solutions?
Framework for comparing step-based tutoring systems
Feature Constraint based Cognitive
Step granularity & user interface Large, fixed-size goal boxes
Goal boxes, granularity varies
Interpreting a student’s step String match Exact, intervals, regular expressions
Suggesting good steps Next blank step Next step in order
Feedback and hints on steps
Task selection
Assessment
Evaluations
Dissemination
Feedback and hints
• What are the hint levels?• How can one game?• Does it detect cut-and-paste?• Does it detect common mistakes e.g., quotes,
“WHERE” as first word of WHERE clause…• Are the hints pedagogical?
Framework for comparing step-based tutoring systems
Feature Constraint based Cognitive
Step granularity & user interface Large, fixed-size goal boxes
Goal boxes, granularity varies
Interpreting a student’s step String match Exact, intervals, regular expressions
Suggesting good steps Next blank step Next step in order
Feedback and hints on steps Few bugs; selectable hint from sequence
Few bugs; hint sequence
Task selection
Assessment
Evaluations
Dissemination
Task selection
• What happens if you select next task in order?• Anyone graduate from novice to higher?• Do clauses make a good index for student task
selection?• Does student have enough info to do task
selection?
Framework for comparing step-based tutoring systems
Feature Constraint based Cognitive
Step granularity & user interface Large, fixed-size goal boxes
Goal boxes, granularity varies
Interpreting a student’s step String match Exact, intervals, regular expressions
Suggesting good steps Next blank step Next step in order
Feedback and hints on steps Few bugs; selectable hint from sequence
Few bugs; hint sequence
Task selection Some student choice Mastery learning
Assessment
Evaluations
Dissemination
Assessment
• What are the KCs? Clauses or constraints?– Displayed?– Learning curves? – How many?
• Are the colored sections of the bars informative?
• How updated?• Is history window useful?
Framework for comparing step-based tutoring systems
Feature Constraint based Cognitive
Step granularity & user interface Large, fixed-size goal boxes
Goal boxes, granularity varies
Interpreting a student’s step String match Exact, intervals, regular expressions
Suggesting good steps Next blank step Next step in order
Feedback and hints on steps Few bugs; selectable hint from sequence
Few bugs; hint sequence
Task selection Some student choice Mastery learning
Assessment ? Knowledge tracing
Evaluations
Dissemination
Outline
• SQL-Tutor (today)• ER-Tutor & Normit (next class)• Technical details (next class)• Evaluation & Dissemination (next class)
top related