Workshop: Automatic Assessment in Mathematics Education Matti Pauna, University of Helsinki, JEM 1
May 17, 2015
Workshop: Automatic Assessment in
Mathematics Education
Matti Pauna, University of Helsinki, JEM
1
Supporting learning with web-based
practicing and assessment exercises
2
Outline
!! Introduction and background
!!Using web based assessment in learning
!!Mobile assessment
!!Authoring exercises and building assignments
!!Case studies and results
!!Hands on
3
Background
!! WebALT project 2005 – 2006
!! JEM (Joining Educational Mathematics) network 2006 – 2009
!! Coordinator: University of Helsinki, department of Mathematics and Statistics
!! Expertise on using TEL in the field of mathematics, focusing on the use of interactive exercises for practicing and assessment
!! WebALT: build a multilingual database of exercises using language independent encoding grammar (OpenMath & GF) and tools to author exercises
4
linalg1:determinant(linalg2:matrix( linalg2:matrixrow(a,b),
linalg2:matrixrow(c,d))) Linguistic hints for mood,tense and directive
5
JEM Thematic Network
" ! Joining Educational Mathematics 2006-2009
" ! Continues dissemination
" ! EU funded network
" ! Gathers technology developers, authors and users of e-content
c!
http://jem-thematic.net
7
Exercises
!! Currently implemented in MapleTA, powered by the Maple computer algebra system
!! Different types: multiple choice, open answer, fill in
blanks…
!! Most of the exercises allow answering by a mathematical formula whose correctness the system evaluates
!! Exercises are algorithmic and randomized
!! Problems include hints and complete solutions
!! Grading is done automatically, using Maple to evaluate student’s response against defined criteria
8
Exercises
!! To assess and practice basic mathematical skills
!! Illustrative graphs and
images
9
Question Banks !! Questions are divided into
banks according to the MathTax classification
!! Manual and ad hoc encoding, the system does not provide support for sufficient metadata
!! Repository supporting
LOM was developed for better management of eContent
10
Examples of use
11
Homework Weekly tests
Diagnostic e-tests
Remedial studies
Distance learning
Virtual online classes
!! Best suited for practicing basic skills which are the
tools required to build understanding and problem solving
!! High-schools, first year college level mathematics
Some benefits of the system
!! Easy to use, only web browser is needed
!! Allows individual practicing at own time, place, and pace in a “neutral” environment
!! Grading is done automatically by the computer which saves time of the instructors
!! Immediate feedback for students
!! Students gain understanding of their learning process and can concentrate on the weak points
and practice more
!! Diagnostic testing helps to design the teaching, for example, dividing students into groups according to their individual learning needs
12
Exercises on Mobile Devices
Smartphones
SMS quizzes
13
Quizzes in iPod or Smartphones
14
SMS Exercises
15
!! Questions, answers and feedback as text messages
!! Developed a Moodle module to cater quizzes via SMS gateway
!! For true time and place independent practicing
!! Just in time classroom quizzes
!! In many countries mobile phones are much more common than computers with internet access
Real time quizzes with phones as personal
response system (clickers)
16
Real time SMS quizzes activate students
17
!! Immediate feedback to correct mistakes right in the beginning
!! Raises the level of attention: students follow the
presentation more carefully knowing that they have to answer questions after it
!! Everybody gets involved
!! Eliminates the need for special devices, which often cause
configuration and other technical problems
Authoring interactive
exercises and quizzes
Exercise structure
Algorithms
Setting up quizzes
18
Question components
19
Question text
Question text with algorithmic parameters
Correct answer
All of these elements are instantiated from the algorithmic parameters
Authoring questions
!! Question bank editor
!! LaTeX macros
!! Editing the source (EDU) code
\begin{question}{MultipleChoice} \name{example 1} \qutext{Given $f(x)=(x+3)^2$, find $f(x+5)$} \choice*{$(x+8)^2$} \choice{$x^2+6x+14$} \choice{$x^2+10x+24$} \choice{none of these} \end{question}
20
Source format
21
22
Algorithm
$a = range(-5,5); condition:ne($a,0); $b = range(-3,3); condition:ne($b,0); condition:le(abs($b-($a)),5); $f = maple("($a)*x+($b)"); $p1=maple("[0,($b)]"); $p2=maple("[0,($b)-($a)]"); $p3=maple("[-1, ($b)-($a)]"); $y1=($b)-($a); $y2=if(lt($b,0), "($b)", $b); $deltay= -($a); $plot = plotmaple("plots:-display([plot($f, x = -5 .. 5, y = -5 .. 5, color = blue, thickness=2), plots:-coordplot(cartesian, [-5 .. 5, -5 .. 5], color = [gray, gray], linestyle = [solid, solid], grid = [11, 11])], axes = normal, tickmarks=[11,11], labels=[``,``], numpoints=300)");
23
Algorithm editor
24
Evaluating correctness of answer
Cannot analyze student’s response, partial grading not possible
25
Equations
!! Encoded in MathML
1.! LaTeX converted to MathML
2.! MathML source
3.! mathml function
$Deq = mathml(“$a*x^2+$b*x+$c”);
4.! Maple call
$Drad = maple(“printf(MathML[ExportPresentation]( sqrt(a+b)<>sqrt(a)+sqrt(b) )) ”);
5.! Equation editor
!! Displayed as images on the web page
26
Equation editor
27
Setting up assignments
28
!! Collect questions into quizzes
!! Set policies
Case studies
30
1. Exercises for Single Variable
Calculus
!! First year university course
!! About 100 exercises for limits, differentiation, integration, series, etc.
!! Maple T.A. exercises used as homework assignments and also a diagnostic test as self-assessment before the course
!! Bonus points were given to students from homeworks and they could take the assignment any number of times, the best attempt counted
!"#$%&'()&*+
,-./0$1/+2$%+
3%4*0*&1#+54/&*+
'.*(1&,-./+$2+
64"*-"-/+
2. High School exercises
!! Exercise sets for all 10 long term math courses in the Finnish curriculum developed since 2005
!! Algorithmic exercises for training basic methods and techniques
!! Students do not usually practice enough which is needed to learn mathematics and solve more demanding problems
74/&*+,-./0$1/+
2%$)+8&9.%.1'+
'$3&*/+&1+(&#(+
/*($$"+)4'(+
Essay questions
!! For expressing mathematical thinking verbally
!! For self evaluation
!! Implemented as essay questions or essay parts in
exercises and they cannot be graded automatically
3. Diagnostic Testing at Polytechnic
schools
!! To assess the level of mastering basic mathematics skills of new students, both for students themselves and instructors
!! Problem: Students have very varied background and teaching needs to be designed accordingly
!! A diagnostic test is organized in computer lab, first week of the semester, a brief introduction to the system was given
!! Algorithmic questions are different for each student, so cheating is harder
39
Results
:&/'%&5-0$1+;<=>+?+=@>+?+A=>++$2+3$&1'/BC+
Research: Kari Lehtonen
Diagnostic tests and practicing in technical engineering school. Results in the test before and after practicing.
Purple (four groups) n = 80, not using online assignments.
Blue group n = 40, using online assignments.
http://jem-thematic.net/en/node/872