Research & Statistics Student Learning Assessment comparing Online vs. Live (Face to Face) Instruction.

Research & Statistics Research & Statistics

Student Learning Assessment Student Learning Assessment comparing Online vs. Live comparing Online vs. Live (Face to Face) Instruction(Face to Face) Instruction

A Goal of the Social Work A Goal of the Social Work graduate program graduate program 

SL Goal 7        The goals of research are to SL Goal 7        The goals of research are to launch research-practitioners who engage launch research-practitioners who engage in lifelong learning based on knowledge in lifelong learning based on knowledge and skills acquired in the MSW program, and skills acquired in the MSW program, to develop methods of evaluating social to develop methods of evaluating social work practice and information used in work practice and information used in practice, and to instill more practice, and to instill more discriminating and critical consumption of discriminating and critical consumption of professional knowledge and professional knowledge and research.          research.         

Research Sequence Research Sequence

Composed of Graduate Composed of Graduate Courses in:Courses in:►Research Methods Research Methods ►StatisticsStatistics

(Not Favorite Courses!)(Not Favorite Courses!)

Research Methods Course Research Methods Course

►Research Methods course covers:Research Methods course covers: Problem FormulationProblem Formulation Conceptualization & Conceptualization &

OperationalizationOperationalization MeasurementMeasurement SamplingSampling DesignDesign Qualitative methodsQualitative methods

Research Methods CourseResearch Methods Course

►Traditionally course required:Traditionally course required: MidtermMidterm FinalFinal Research ProposalResearch Proposal

Was highly stressful for Was highly stressful for students!students!

Research MethodsResearch Methods

►F2F course consisted of:F2F course consisted of: LectureLecture Question & AnswersQuestion & Answers DiscussionDiscussion Group work applying covered aspect of Group work applying covered aspect of

course to a real social work problemcourse to a real social work problem

Online Research CourseOnline Research Course

►Frequent student inquires about Frequent student inquires about online courses online courses

►Social Work students driving in from Social Work students driving in from all corners of the stateall corners of the state

►Shift from program with primarily full Shift from program with primarily full time students to part-time studentstime students to part-time students

►Students with work or family Students with work or family demands demands

Online Research CourseOnline Research Course

►Chose course very comfortable Chose course very comfortable withwith

►Excellent textbookExcellent textbook►Many notes and hand-outs Many notes and hand-outs

already developedalready developed

Online Research CourseOnline Research Course

►Progression from web-Progression from web-enhanced to total online enhanced to total online coursecourse

►E-pakE-pak►STaR Training and SupportSTaR Training and Support

Online Research CourseOnline Research Course

►Taught live f2f section in addition Taught live f2f section in addition to online sectionto online section

►After f2f class, would tape a video-After f2f class, would tape a video-stream of condensed lecture (20 stream of condensed lecture (20 min.)min.)

►Covering important points and Covering important points and explain difficult materialexplain difficult material

Online Research CourseOnline Research Course

►Move from midterm & finals to Move from midterm & finals to weekly chapter quizzesweekly chapter quizzes

►Assessments of articles Assessments of articles highlighting topic being highlighting topic being covered that weekcovered that week

►Group discussion of topics Group discussion of topics

Research Methods Courses Research Methods Courses

►Research Proposal Research Proposal Broken into several components:Broken into several components: Research idea and bibliographyResearch idea and bibliography Peer review of research ideaPeer review of research idea Problem Statement—Lit ReviewProblem Statement—Lit Review Full proposalFull proposal Short presentationShort presentation

Course comparisonCourse comparison

► F2FF2F LectureLecture Class hand-outsClass hand-outs

Class discussionClass discussion Group exercises with Group exercises with

real social work real social work problemproblem

TestsTests

►On-line:On-line: Video-streamVideo-stream Handouts and E-pak Handouts and E-pak

materialmaterial Group discussionGroup discussion Group exercises with Group exercises with

real social work real social work problemproblem

Chapter quizzesChapter quizzes

Course comparisonCourse comparison

► F2FF2F :: Assignments Assignments

interpreting articles interpreting articles Research Proposal in Research Proposal in

installmentsinstallments PresentationPresentation AssessmentAssessment

►On-line:On-line: Assignments Assignments

interpreting articlesinterpreting articles Research Proposal in Research Proposal in

installmentsinstallments PresentationPresentation AssessmentAssessment

StatisticsStatistics

►Covers descriptive statistics through Covers descriptive statistics through multivariate regressionmultivariate regression

►Approach is toward understanding and Approach is toward understanding and application of statistics to help application of statistics to help understand real problems rather than understand real problems rather than formulas and calculations.formulas and calculations.

Statistics Teaching ApproachStatistics Teaching Approach

►LectureLecture►TextbookTextbook►Class discussionClass discussion►Hand-outs of Statistical TablesHand-outs of Statistical Tables►SPSS and dataSPSS and data

Student assignments:Student assignments:

►Chapter quizzesChapter quizzes►Small group interpretation of tablesSmall group interpretation of tables►SPSS analysis of dataSPSS analysis of data►Write-ups interpreting SPSS Write-ups interpreting SPSS

results and discussing findingsresults and discussing findings►Write-ups on questions regarding Write-ups on questions regarding

journal articlesjournal articles►Final assessmentFinal assessment

Actual CourseActual Course

►Captivate snipCaptivate snip

Final AssessmentFinal Assessment

►Final assessment is a critical review of Final assessment is a critical review of a journal articlea journal article

►Students have one week to review and Students have one week to review and critique an article assigned by critique an article assigned by professorprofessor

►The assessment results are used for The assessment results are used for assessment of entire research assessment of entire research sequence sequence

AssessmentAssessment

►Questions on Introduction, Method, Questions on Introduction, Method, Results, and Discussion components of Results, and Discussion components of articlesarticles

►Assessment data comparing f2f to Assessment data comparing f2f to online students compiled for 2006 and online students compiled for 2006 and 2007.2007.

►Data shown here compare f2f to online Data shown here compare f2f to online sections of this instructorsections of this instructor

IntroductionIntroduction

►Clear Research Question? Clear Research Question? Hypotheses?Hypotheses?

►Appropriate Literature Review?Appropriate Literature Review?

►TypeType NN MeanMean SDSD tt pp

(20)(20)►f2ff2f 3838 18.9718.97 1.901.90 .84 ns.84 ns►OnlineOnline 3030 18.5718.57 2.022.02

MethodMethod

►Sample? Design? Measures? Sample? Design? Measures? Procedure?Procedure?

►TypeType NN MeanMean SDSD tt p p

(20)(20)►F2fF2f 3838 17.217.2 3.63.6 1.63 1.63

nsns►OnlineOnline 3030 15.915.9 3.03.0

ResultsResults

►Appropriate Statistics? Interpretations of Appropriate Statistics? Interpretations of analyses? Statistical vs clinical analyses? Statistical vs clinical significance?significance?

► TypeType NN MeanMean SDSD tt pp►F2fF2f 3838 42.2142.21 7.37.3 -.368 ns -.368 ns►OnlineOnline 3030 42.7842.78 4.84.8

DiscussionDiscussion

►Proper interpretation and discussion of Proper interpretation and discussion of findings? findings?

►TypeType NN MeanMean SDSD tt pp

(10)(10)►F2fF2f 3838 8.988.98 2.192.19 -.095 -.095 nsns►OnlineOnline 3030 9.039.03 1.711.71

Total AssessmentTotal Assessment

► Intro, Method, Results, DiscussionIntro, Method, Results, Discussion

►TypeType NN MeanMean SDSD tt pp

(100)(100)►F2fF2f 3838 87.4187.41 9.019.01 .527 ns .527 ns ►OnlineOnline 3030 86.2886.28 8.428.42

ConclusionConclusion

►End of sequence assessment indicates End of sequence assessment indicates that graduate students critically that graduate students critically evaluate articles by end of semesterevaluate articles by end of semester

►Analyses indicate no difference in Analyses indicate no difference in learning between students in f2f and learning between students in f2f and online course deliveryonline course delivery

ConclusionConclusion

►Comparisons with a f2f section from Comparisons with a f2f section from another instructor (not shown) found another instructor (not shown) found small but significant differences with small but significant differences with BOTH f2f and online sections of this BOTH f2f and online sections of this instructor!instructor!

RecommendationsRecommendations

►There are many valid reasons why There are many valid reasons why students may prefer online to f2f students may prefer online to f2f courses.courses.

► Important for students to be cognizant Important for students to be cognizant of their learning styles and self of their learning styles and self direction when choosing their courses direction when choosing their courses and its delivery method. and its delivery method.