Presented by Jennifer Fager For University of Wisconsin-Superior Enhancement Day 1/19/2011
Jan 19, 2016
Presented by Jennifer Fager For
University of Wisconsin-SuperiorEnhancement Day
1/19/2011
Advice for Assessing Student Learning in an Online EnvironmentMultiple Choice and Short Answer Questions
Not always the best technique in either online or ground formats
Lend themselves to cheating more so in onlineQuestions, regardless of ground or online,
must be written wellThe “test” has been overused and relied upon
unless the instructor is an expert in writing good test items
Advice continuedUse multiple forms of assessment
Has corroborating evidence of student achievement
More personalized assessments reduce the likelihood of plagiarism “Term paper” vs. application paper using personal
examples General research paper vs. select topics focus
Use examples more “authentic” in formatConsider problems of the fieldEngage in “conversation’s” online
Advice continuedSecurity options
Lockdown browser Student can’t go to another website during the test NOTE: Most students now have multiple devices so
they can go to their IPhones, blackberries, laptops, IPads, etc. to research the “answers”
Advice ContinuedSet test time limits
Even if the student gets help, time restricts the amount
Can punish the slower readers/processorsWhat do you want in terms of meaningful,
thoughtful responses
Advice continuedAssume any online test is, by definition, open
bookAnd perhaps open notes, and open neighbors
Don’t try to make it otherwise
Advice continuedQuiz/test software products are available
Can generate multiple versions of testsEach student can receive a “different” test
http://www.fau.edu/irm/instructional/respondus.php
Have students explain their answers25% have to explain ‘A’ answers, 25% ‘B’
answers, etc.
Advise continuedOther security ideas
“thumb” identificationWebcamsFace-to-face testing at a testing center
Online Assessment ResourcesCommercial Assessment Data Management Tools –
E.g. TracDat, WeaveOnline, Chalk & WireCurriculum Design Tools
E.g. WIDS – link outcomes to Performance Assessment Tasks
Survey InstrumentsSurvey MonkeyZoomerang
PortfoliosCourse Management Systems Quiz and Analysis
FunctionsBlackboard and others
Resources ContinuedGoogle DocsJingReflective Instruments
LeadershipPersonality StyleTeamwork
Merlot (has content template builder)Clickers AdobeConnect with polling featuresLock-down Browsers for online testing
Fundamental QuestionsWhat evidence do you have that students
achieve your stated learning outcomes?How have you articulated these outcomes to
students and adjunct faculty?In what ways do you analyze and use
evidence of student learning?
Assessment ProcessThe same process is used for online programs
and courses as for ground (traditional):Needs Assessment
Needs of the profession Needs of the learners
Articulate your expectationsMeasure achievement of expectationsCollect and analyze dataUse evidence to improve learningAssess effectiveness of improvement
Measure Online LearningDirect Methods
Mostly the same as groundCan use any student work products that can be
saved in an electronic formSome challenges exist with lab work, oral
performances, etc.Indirect Methods
Again, same as groundSurvey response rates are notoriously low
Advantages of Online AssessmentArtifact collection can be automated through
the course management softwareCan work for ground as well
Assignments, rubrics can be standardized and mapped to specific competencies/outcomesAgain, can work for ground as well
New Opportunities with OnlineAssess quality of discussions, group work
Particularly effective with asynchronousAssess individual learning in more depth
Student advising can be more robustUse of standardized rubrics can result in
greater consistency in gradingParticularly effective when combined with
periodic instructor norming and use of anchor samples
Discussion & Group WorkFaculty can isolate and organize
contributions to a threaded discussion and give feedback on how to improveCan archive individual contributions and group
discussionsFormative and summativeCourse, program, and institutional levels
Create a record of instructor effectivenessUseful for annual review, tenure
Individual Student LearningIndividual learning/achievements can be
tracked across the educational experienceE-format allows for wide dissemination of
collected dataCurrent software packages provide various
levels of data collection & dissemination
RubricsClear articulation of expectations that are
linked to specific course & program outcomesA means through which correspondence of
online & ground rigor can be ensured (alignment)
Increases consistency in grading across sections, courses, programs, even colleges
Holistic vs. Analytic
Example of a Successful Online Assessment StrategyDevelop course/program maps which
articulate learning outcomes and link them to specific courses and course assignments
Collect and archive artifacts through the course management software
Post rubrics through course management software for student and faculty use
Ensure Validity, ReliabilityTracking inter-rater reliability helps achieve
consistency in gradingPost rubrics, anchor samples for 24/7 faculty
useFull-time/core faculty should formulate
competencies, achievement standards & interpret results
Sample AssessmentsReview the Video Gaming Assignment
included at the end of this packet.Determine how this assignment could be
adapted for online learning
General Education Outcomes ExampleThe ability and inclination to think and make
connections across academic disciplinesHow would you assess this outcome in
ground or online courses?
Websites and Resourceshttp://fod.msu.edu/oir/TeachWithTech/online-
courses.asphttp://www.uwstout.edu/soe/profdev/assess.cf
mhttp://www.fgcu.edu/onlinedesign/index.htmlhttp://www.pvc.maricopa.edu/~lsche/resourc
es/onlineteaching.htm
http://sloanconsortium.org/
Questions and ConcernsWhat issues have you encountered when
assessing online learning?What are your concerns regarding the use of
online assessment techniques?What policies and/or procedures need to be
in place to reduce or alleviate these concerns?
Questions Still Burning?What else do you need to know?
The Bottom Line?Assessment in online learning can be the
same as assessment on ground—artifacts and tools can be the same even if the medium is different!