1 Is online learning cost- effective? EERS May 2 nd, 2011 Fiona Hollands, Ph.D. Center for Benefit-Cost Studies in Education Center for Technology and.

11

Is online learning cost-Is online learning cost-effective?effective?

EERS May 2EERS May 2ndnd, 2011, 2011

Fiona Hollands, Ph.D.Fiona Hollands, Ph.D.

Center for Benefit-Cost Studies in EducationCenter for Benefit-Cost Studies in EducationCenter for Technology and School ChangeCenter for Technology and School Change

Teachers CollegeTeachers CollegeColumbia UniversityColumbia University

[email protected]@columbia.edu

22

How can cost-effectiveness analysis help How can cost-effectiveness analysis help educators and policymakers take better educators and policymakers take better

informed action?informed action? Facilitate more efficient use of Facilitate more efficient use of

educational resources byeducational resources by

– Allowing the Allowing the lowest costlowest cost alternative to alternative to be chosen to achieve a target objectivebe chosen to achieve a target objective

– Allowing the Allowing the most effectivemost effective alternative alternative to be chosen given a fixed budget or to be chosen given a fixed budget or resource allocationresource allocation

33

There are two parts to the There are two parts to the question…question…

Compared with traditional face-to-Compared with traditional face-to-face (f2f) classroom teaching: face (f2f) classroom teaching:

1) Does online learning cost less, the 1) Does online learning cost less, the same or more?same or more?This is what most administrators and This is what most administrators and

policymakers focus onpolicymakers focus onExisting published studies are usually based Existing published studies are usually based

on self-reported data and do not tie into on self-reported data and do not tie into effectiveness or benefitseffectiveness or benefits

44

Part 2 of the cost-effectiveness Part 2 of the cost-effectiveness questionquestion

2) Is online learning more, less or just 2) Is online learning more, less or just as effective in promoting positive as effective in promoting positive academic outcomes? academic outcomes?

This is what most people forget to consider This is what most people forget to consider thoroughly (or just assume parity of thoroughly (or just assume parity of effectiveness) but what teachers and effectiveness) but what teachers and parents want to knowparents want to know

Most published studies of effectiveness are Most published studies of effectiveness are at the higher ed. levelat the higher ed. level

55

Online learning is not monolithic - the Online learning is not monolithic - the answers will depend on the modelanswers will depend on the model

Use and delivery of online learning varies dramatically:Use and delivery of online learning varies dramatically:

– i) at one extreme some students learn full time online from i) at one extreme some students learn full time online from home: home: virtual schoolingvirtual schooling

– ii) more commonly many students who attend school daily opt ii) more commonly many students who attend school daily opt to take one or two pure online coursesto take one or two pure online courses

– iii) a growing number of models incorporate aspects of both iii) a growing number of models incorporate aspects of both face-to-face and online learning in the same classroom: face-to-face and online learning in the same classroom: blended blended learninglearning

– iv) at the other extreme students learn full time in face-to-face iv) at the other extreme students learn full time in face-to-face situations with limited online aspects to enhance the experience situations with limited online aspects to enhance the experience

Generalizing, the Generalizing, the apparentapparent costs per student increase from i) thru iv) costs per student increase from i) thru iv)

66

What are the cost numbers currently What are the cost numbers currently estimated for online learning?estimated for online learning?

Cavanaugh surveyed 20 directors of virtual Cavanaugh surveyed 20 directors of virtual schools in 14 states – ave. cost of full time online schools in 14 states – ave. cost of full time online students in 2008 was estimated at $4,310students in 2008 was estimated at $4,310

This is 42% of the ave. per pupil expenditure in This is 42% of the ave. per pupil expenditure in 2007-08 of $10,297 (NCES)2007-08 of $10,297 (NCES)

Compare this 42% with the general finding that Compare this 42% with the general finding that the instructional program counts for about 60-the instructional program counts for about 60-65% of total education expenditures (Levin).65% of total education expenditures (Levin).

The virtual school estimates exclude services The virtual school estimates exclude services such as such as – transportation, nutrition, counseling, nursing, college transportation, nutrition, counseling, nursing, college

guidance, libraries, media specialists and resources, guidance, libraries, media specialists and resources, clubs, activities and professional development services. clubs, activities and professional development services.

77

To make a fair comparison between virtual To make a fair comparison between virtual and “brick and mortar” schools:and “brick and mortar” schools:

We must considerWe must consider– ““costs” of losing these services orcosts” of losing these services or– costs of providing alternate access to them in costs of providing alternate access to them in

the communitythe community Anderson Anderson et alet al (2006) estimated costs of (2006) estimated costs of

virtual schooling to be about the same as virtual schooling to be about the same as regular brick and mortar schools when regular brick and mortar schools when similar services are being provided, similar services are being provided, excluding transportation and capital costsexcluding transportation and capital costs

88

Some costs of online learning that Some costs of online learning that are often not accounted for:are often not accounted for:

Development costs of the delivery Development costs of the delivery mechanism. Can be amortized over the mechanism. Can be amortized over the expected lifetime use.expected lifetime use.

Ongoing maintenance and adjustment of Ongoing maintenance and adjustment of course contentcourse content

Professional development costs for f2f Professional development costs for f2f teachers incorporating online aspects or teachers incorporating online aspects or for pure online teachers/tutorsfor pure online teachers/tutors

Content costs – selection and purchase Content costs – selection and purchase from outside vendor or development in-from outside vendor or development in-house; integration with current curriculumhouse; integration with current curriculum

99

How should we measure the How should we measure the effectiveness of online learning?effectiveness of online learning?

Effectiveness can mean different things:Effectiveness can mean different things:– Higher standardized test scoresHigher standardized test scores– More content learned in a fixed time frameMore content learned in a fixed time frame– Same amount of content learned faster (in Same amount of content learned faster (in

some cases, speedier graduation)some cases, speedier graduation)– Higher course completion/graduation rate Higher course completion/graduation rate

than f2f (e.g. in credit recovery programs)than f2f (e.g. in credit recovery programs)

1010

A few credible effectiveness studies A few credible effectiveness studies exist – mostly for higher ed.exist – mostly for higher ed.

U.S. DoE meta-analysis of online learning (2009) U.S. DoE meta-analysis of online learning (2009) found 99 studies, comparing learning outcomes in found 99 studies, comparing learning outcomes in online/blended learning with f2fonline/blended learning with f2f– only 9 involved K-12 learnersonly 9 involved K-12 learners

Conclusions of meta-analysisConclusions of meta-analysis– Blended instruction is more effective than conventional Blended instruction is more effective than conventional

f2f classes for older learners (undergrads and adults)f2f classes for older learners (undergrads and adults)– Pure online learning offers a “modest advantage” over Pure online learning offers a “modest advantage” over

conventional instruction for older learnersconventional instruction for older learners– Treatment conditions often included additional learning Treatment conditions often included additional learning

time, materials and opportunities for collaborationtime, materials and opportunities for collaboration– Based on a small sample of 5 studies, positive effects Based on a small sample of 5 studies, positive effects

not found for K-12not found for K-12

1111

CAI cost-effectiveness for math CAI cost-effectiveness for math instructioninstruction

Barrow, Markman & Rouse (2007)Barrow, Markman & Rouse (2007)– Randomly assigned classrooms to computer labs vs. reg Randomly assigned classrooms to computer labs vs. reg

classrooms for algebra and pre-algebra instruction.classrooms for algebra and pre-algebra instruction.– Concluded that students in CAI classes scored 0.17 of a Concluded that students in CAI classes scored 0.17 of a

s.d. higher on tests of algebra achievement than s.d. higher on tests of algebra achievement than controls.controls.

– (Very rough and limited) estimate of lab cost is $1600 (Very rough and limited) estimate of lab cost is $1600 per student per yearper student per year

CE ratio = 1600/.17 = $9,411 per student to raise CE ratio = 1600/.17 = $9,411 per student to raise algebra test score 1 s.d. algebra test score 1 s.d.

Need to compare with another intervention Need to compare with another intervention aiming to affect same outcomeaiming to affect same outcome

1212

Is CAI cost-effective?Is CAI cost-effective?

Authors compared CAI costs and effects with Authors compared CAI costs and effects with Tennessee STAR class size reduction which, at Tennessee STAR class size reduction which, at $1850 per student per yr., resulted in 0.22 s.d. $1850 per student per yr., resulted in 0.22 s.d. increase in test scores (CE ratio = $1850/.22 = increase in test scores (CE ratio = $1850/.22 = $8,409)$8,409)

Concluded that CAI may be a cost effective way Concluded that CAI may be a cost effective way to increase math test scoresto increase math test scores

Note, however, that the two interventions were Note, however, that the two interventions were not judged by the same test scores. Also STAR not judged by the same test scores. Also STAR aimed to affect more than just math outcomes.aimed to affect more than just math outcomes.

1313

Some specific examples of online Some specific examples of online learninglearning

Blended learning for middle school Blended learning for middle school mathmath– School of One (So1), New York, NYSchool of One (So1), New York, NY

Blended learning in the humanitiesBlended learning in the humanities– Roslyn High School, Roslyn, New YorkRoslyn High School, Roslyn, New York

Online Credit recoveryOnline Credit recovery– Cross High School, New Haven, CTCross High School, New Haven, CT

1414

School of One (So1) School of One (So1) http://www.schoolofone.orghttp://www.schoolofone.org//

So1 is a blended learning math program for 6So1 is a blended learning math program for 6th, th, 77th th

and 8and 8thth graders developed by the NYC DoE graders developed by the NYC DoE Serves 1500 students across 3 schools in NYCServes 1500 students across 3 schools in NYC Students spend 70 minutes a day learning mathStudents spend 70 minutes a day learning math Students learn in a variety of modalities both Students learn in a variety of modalities both

virtual and livevirtual and live Learning with software Learning with software Learning with remote tutorsLearning with remote tutors IndependentIndependent Large group instructionLarge group instruction Small group instructionSmall group instruction Small group collaborationSmall group collaboration Peer tutoringPeer tutoring Integrated learning projectsIntegrated learning projects

Each student’s daily lesson plan or “playlist” is Each student’s daily lesson plan or “playlist” is generated by a computer-based “Learning generated by a computer-based “Learning Algorithm”Algorithm”

1515

Applying the “Ingredients” or “Resource Applying the “Ingredients” or “Resource Cost” method of cost analysis to School Cost” method of cost analysis to School

of Oneof One

Costs can be separated into Costs can be separated into

– Up front development costs Up front development costs

– Adoption costs that would be Adoption costs that would be experienced by sites adopting So1experienced by sites adopting So1Pre-requisitesPre-requisitesNew costs New costs

1616

Categories of ingredientsCategories of ingredients Personnel (often accounts for about 75% of any Personnel (often accounts for about 75% of any

educational intervention) including volunteerseducational intervention) including volunteers– Salaries, benefits (often approximated at 20% of salary)Salaries, benefits (often approximated at 20% of salary)

Facilities Facilities – rent, building costsrent, building costs

Equipment and materialsEquipment and materials– Technology, books etc.Technology, books etc.

Other program inputsOther program inputs– Insurance, electricity etc.Insurance, electricity etc.

Required client inputsRequired client inputs– transport coststransport costs

1717

Identifying the ingredientsIdentifying the ingredients

Review of program documentsReview of program documents Interviewing personnel involved in Interviewing personnel involved in

development and delivery of development and delivery of interventionintervention

Direct observation of the intervention Direct observation of the intervention in a typical field situationin a typical field situation

1818

Assigning costsAssigning costs Once the quantity and quality of each Once the quantity and quality of each

ingredient is identified, costs are assigned ingredient is identified, costs are assigned to eachto each– Market prices for salaries, equipmentMarket prices for salaries, equipment– Amortize facilities or equipmentAmortize facilities or equipment– Shadow prices for items where no market Shadow prices for items where no market

existsexists– Why we need to assign costs to volunteers or Why we need to assign costs to volunteers or

“free” resources“free” resources Replication elsewhere may not be able to depend on Replication elsewhere may not be able to depend on

thesethese– Concept of opportunity cost e.g. for high school Concept of opportunity cost e.g. for high school

mentorsmentors

1919

Sensitivity analysisSensitivity analysis

Costs may vary depending on scale and Costs may vary depending on scale and location location

Interventions with high fixed costs (as Interventions with high fixed costs (as opposed to variable costs) will look opposed to variable costs) will look cheaper as scale increases – up to a pointcheaper as scale increases – up to a point

Many educational interventions are added Many educational interventions are added onto the existing program so that a onto the existing program so that a marginal cost effectiveness analysis is marginal cost effectiveness analysis is appropriateappropriate

2020

So1 Development costs – So1 Development costs – estimated at $7mm over 2 yearsestimated at $7mm over 2 years

Technology development costs to build a system that Technology development costs to build a system that – houses 5000 math lessons that can be completed and assessed onlinehouses 5000 math lessons that can be completed and assessed online– Learning Algorithm that tracks individual student progress, determines what skills (s)he has Learning Algorithm that tracks individual student progress, determines what skills (s)he has

mastered and what still needs workmastered and what still needs work– presents a daily “playlist” of math activities for each individual studentpresents a daily “playlist” of math activities for each individual student

Currently outsourced to Wireless GenerationCurrently outsourced to Wireless Generation

Estimated cost $4mm over 2 Estimated cost $4mm over 2 yearsyears

Content: 5000 lessons purchased from 50 different vendors and adapted for So1 Content: 5000 lessons purchased from 50 different vendors and adapted for So1 systemsystem

Estimated cost $0.5mmEstimated cost $0.5mm

Panel of math experts to develop math skills map and review 25,000 possible lessonsPanel of math experts to develop math skills map and review 25,000 possible lessons

Estimated cost $150,000Estimated cost $150,000

Team of 12 education/technology professionals working with vendor to develop Team of 12 education/technology professionals working with vendor to develop system and interfacing with schoolssystem and interfacing with schools

Estimated cost $1.2mmEstimated cost $1.2mm

2121

Adoption cost assumptionsAdoption cost assumptions

Costs estimated based onCosts estimated based on– 480 middle school students480 middle school students– 4 fully certified teachers + 2 student 4 fully certified teachers + 2 student

teachersteachers– 4 groups of 120 students each working 4 groups of 120 students each working

with So1 for 70 mins/day, 5 days a with So1 for 70 mins/day, 5 days a week, 36 weeks a yearweek, 36 weeks a year

2222

Pre-requisite resources at adoption Pre-requisite resources at adoption sites: estimated costs per annumsites: estimated costs per annum

Math teachers – current model serves 4 math Math teachers – current model serves 4 math sections (25-33 students each) at once with 4 sections (25-33 students each) at once with 4 certified teachers plus 2 student teacherscertified teachers plus 2 student teachers

Estimated cost $380,000 Estimated cost $380,000 (expect to be lower outside of (expect to be lower outside of

NYC)NYC)

Wireless connectivityWireless connectivity– $50,000 for entire school amortized over 5 years$50,000 for entire school amortized over 5 years

Estimated cost $10,000Estimated cost $10,000

E-mail access for entire schoolE-mail access for entire school

Estimated cost $25,000Estimated cost $25,000

2323

New per annum costs of adoptionNew per annum costs of adoption

Construction costs of opening up suitable space in school building to Construction costs of opening up suitable space in school building to accommodate 120 students at once accommodate 120 students at once

Estimated cost $200,000 amortized over 5 years =$40,000Estimated cost $200,000 amortized over 5 years =$40,000

Initial teacher professional development – 1 week in summer (provided by So1)Initial teacher professional development – 1 week in summer (provided by So1)

Estimated cost for 4 teachers *5 days = $8000Estimated cost for 4 teachers *5 days = $8000

Full time in-house digital content manager to interface with So1 and provide Full time in-house digital content manager to interface with So1 and provide tech support/professional developmenttech support/professional development

Estimated cost $80,000Estimated cost $80,000

Approx. 100 individual PCs/laptops in classroom, two 48” LCD terminals Approx. 100 individual PCs/laptops in classroom, two 48” LCD terminals displaying each student’s daily playlist, 4 printers, 1 LCD projectordisplaying each student’s daily playlist, 4 printers, 1 LCD projector

Estimated cost $110,000 amortized over 3 years = $36,667Estimated cost $110,000 amortized over 3 years = $36,667

2424

New per annum costs of adoption New per annum costs of adoption (cont.)(cont.)

Licensing charge from So1 for access to Learning Algorithm, all Licensing charge from So1 for access to Learning Algorithm, all content and provision of daily playlistscontent and provision of daily playlists

Estimated cost $150,000Estimated cost $150,000

Ongoing professional development – 2 hours weekly with So1 Ongoing professional development – 2 hours weekly with So1 professional developer (assuming no extra pmt to teachers)professional developer (assuming no extra pmt to teachers)

Estimated cost of trainer $10,000Estimated cost of trainer $10,000

Virtual tutors (currently in the So1 model but due to high cost, Virtual tutors (currently in the So1 model but due to high cost, shown here as an option)shown here as an option)

Estimated for 15 tutors, 4 hours/day, 180 days/yr at Estimated for 15 tutors, 4 hours/day, 180 days/yr at $30/hr = $324,000$30/hr = $324,000

2525

Estimated adoption cost summary Estimated adoption cost summary (assuming 6 teachers, pre-existing wireless connectivity and e-mail for all (assuming 6 teachers, pre-existing wireless connectivity and e-mail for all students)students)* cost amortized over 5 years* cost amortized over 5 years** cost amortized over 3 years** cost amortized over 3 years

Cost per annum for Cost per annum for school of 480school of 480

Cost per Cost per studentstudent

Construction*Construction* $40,000$40,000 8383

Initial p.d.Initial p.d. $8,000$8,000 1717

Digital content Digital content manager/tech supportmanager/tech support

$80,000$80,000 167167

Hardware**Hardware** $36,667$36,667 7676

So1 licensingSo1 licensing $150,000$150,000 313313

Ongoing p.d. Ongoing p.d. $10,000$10,000 2121

TOTALTOTAL $300,000$300,000 $677$677

Virtual tutor optionVirtual tutor option $324,000$324,000 $540$540

TOTAL with virtual tutorsTOTAL with virtual tutors $619,000$619,000 $1,217$1,217

2626

Effectiveness data available for Effectiveness data available for So1 So1 http://www.schoolofone.org/research.htmlhttp://www.schoolofone.org/research.html

Summer ‘09 pilot 4hrs/day, 5 days /week for 5 weeks: rising Summer ‘09 pilot 4hrs/day, 5 days /week for 5 weeks: rising 77thth graders gained ave. of 28.2% from pre-test to post-test. graders gained ave. of 28.2% from pre-test to post-test. (EDC/CCT evaluation)(EDC/CCT evaluation)

Issues to consider:Issues to consider:

– 10 adult educators and 3 HS interns for 80 students 10 adult educators and 3 HS interns for 80 students → → very very high teacher/student ratiohigh teacher/student ratio

– 100 hrs of math is equivalent to instruction for over 4/5 of an 100 hrs of math is equivalent to instruction for over 4/5 of an entire regular school yearentire regular school year

– Lack of comparison group prevents determination of how Lack of comparison group prevents determination of how effective So1 is compared with traditional teaching or other effective So1 is compared with traditional teaching or other models models

2727

Afterschool and in-school pilots of Afterschool and in-school pilots of So1So1(NYCDoE Research and Policy Support Group, 2010)(NYCDoE Research and Policy Support Group, 2010)

Afterschool trial Feb-May 2010: 600 sixth Afterschool trial Feb-May 2010: 600 sixth graders across 3 schools graders across 3 schools – students opted instudents opted in– Evaluation positive in only one of the 3 schools Evaluation positive in only one of the 3 schools – only about half the students were included in only about half the students were included in

the analysisthe analysis Followed by in-school implementation Followed by in-school implementation

May-June 2010 with all 6May-June 2010 with all 6thth graders in 1 of graders in 1 of the schools. the schools. – No sig. impact on MAP scores vs. comparison No sig. impact on MAP scores vs. comparison

group.group.

2828

Another example: Roslyn High Another example: Roslyn High School iPad programSchool iPad program

Roslyn High School, NY serves 1100 Roslyn High School, NY serves 1100 students.students.

Last June the supt. leased 275 iPads for 4 Last June the supt. leased 275 iPads for 4 yrsyrs

Currently being used in 11Currently being used in 11thth and 12 and 12thth grade grade humanities classes in a 1-1 blended humanities classes in a 1-1 blended learning format learning format

Teacher uses class website to post Teacher uses class website to post articles, post assignments, hold online articles, post assignments, hold online discussions.discussions.

Paperless classroom and all homework Paperless classroom and all homework exchanged/graded online.exchanged/graded online.

2929

Marginal costs of 1-1 iPad Program (beyond Marginal costs of 1-1 iPad Program (beyond existing teacher and school costs) existing teacher and school costs)

* Note that in Roslyn the teachers volunteered their time and administrators provided the training* Note that in Roslyn the teachers volunteered their time and administrators provided the training

Total cost per annumTotal cost per annum Per student cost/yrPer student cost/yr

Wireless accessWireless access $78,000 for 1100 student $78,000 for 1100 student building amortized over 5 building amortized over 5 years= $15,600years= $15,600

$14 (spreading cost over $14 (spreading cost over 1100 students)1100 students)

Internet accessInternet access $50,000 for 5 schools (5400 $50,000 for 5 schools (5400 students)students)

$9 (spreading cost over $9 (spreading cost over 5400 students)5400 students)

Leasing 275 iPadsLeasing 275 iPads $180,000 over 4 years = $180,000 over 4 years = $45,000$45,000

$170 (assuming 265 $170 (assuming 265 students use iPads)students use iPads)

Tech supportTech support 5% of total district tech 5% of total district tech support cost of $400,000 = support cost of $400,000 = $20,000$20,000

$73$73

Teacher training (10 Teacher training (10 teachers)*teachers)*

$48,000 teacher payments$48,000 teacher payments

$30,000 trainer fees = $30,000 trainer fees = $78,000$78,000

$284$284

Additional iPad applicationsAdditional iPad applications $8800 (assuming 4 apps at $8800 (assuming 4 apps at $8 for 275 iPads)$8 for 275 iPads)

$32$32

TOTALTOTAL $217,400$217,400 $582 $582

3030

Effectiveness of Roslyn 1-1 iPad Effectiveness of Roslyn 1-1 iPad program?program?

No formal evaluation conducted, program No formal evaluation conducted, program only 3 months oldonly 3 months old

Early indications from teacher: Early indications from teacher: – Higher homework completion ratesHigher homework completion rates– Easier for students to make up work during Easier for students to make up work during

absences absences – Expanded learning time e.g. through after-Expanded learning time e.g. through after-

school online review sessions before testsschool online review sessions before tests– Paperless classroomPaperless classroom

3131

Credit recovery programs e.g. Credit recovery programs e.g. Cross High school, New Haven CTCross High school, New Haven CT

1111thth and 12 and 12thth graders can “retrieve” credits graders can “retrieve” credits through online courses taken at school or home. through online courses taken at school or home. 22 current users22 current users

Expenditures involved beyond existing costs of Expenditures involved beyond existing costs of schooling:schooling:– Content licenses $375 per actual userContent licenses $375 per actual user– Computer room proctor/tutor 3 hrs a day estimated at Computer room proctor/tutor 3 hrs a day estimated at

$25,000 per annum$25,000 per annum If all students earn 2 credits a year, expenditures If all students earn 2 credits a year, expenditures

are around $755 per creditare around $755 per credit This can be crudely compared to the cost of a This can be crudely compared to the cost of a

regular credit of approx. $2500regular credit of approx. $2500 Effectiveness?Effectiveness?

3232

Benefits of online credit recovery?Benefits of online credit recovery?

The student may graduate rather than drop out or repeat The student may graduate rather than drop out or repeat the year the year – Lower costs for districts due to fewer repeating studentsLower costs for districts due to fewer repeating students– Higher tax revenues from greater earnings of graduates (but Higher tax revenues from greater earnings of graduates (but

these accrue to Treasury, not schools directly)these accrue to Treasury, not schools directly)– If more HS graduates go to college, this will cost the state and If more HS graduates go to college, this will cost the state and

while earnings of these college graduates will eventually be while earnings of these college graduates will eventually be higher, they are deferred 2-4 years.higher, they are deferred 2-4 years.

However, is it realistic to learn in 30 hours online what However, is it realistic to learn in 30 hours online what might be learned in a semester long class (around 60 hours might be learned in a semester long class (around 60 hours of instruction)?of instruction)?

If remedial courses are required in college, these costs are If remedial courses are required in college, these costs are just being deferred, not savedjust being deferred, not saved– Around ¾ of 17,500 freshmen at CUNY community colleges Around ¾ of 17,500 freshmen at CUNY community colleges

this year have needed remedial education (NYTimes 3/4/11)this year have needed remedial education (NYTimes 3/4/11)

3333

Research on credit recovery Research on credit recovery effectiveness?effectiveness?

There are currently no peer reviewed There are currently no peer reviewed studies indicating effectiveness of credit studies indicating effectiveness of credit recovery programs in terms of academic recovery programs in terms of academic outcomesoutcomes

3434

ConclusionsConclusions

Significant cost savings are possible when Significant cost savings are possible when online learning is used to replace f2f online learning is used to replace f2f instruction, primarily due to instruction, primarily due to – increasing student/teacher ratiosincreasing student/teacher ratios– elimination of non-instructional services elimination of non-instructional services

In some cases the costs are just being In some cases the costs are just being deferred elsewhere e.g. families, colleges.deferred elsewhere e.g. families, colleges.

There is, as yet, little peer-reviewed There is, as yet, little peer-reviewed documentation of the effectiveness of K-12 documentation of the effectiveness of K-12 online learning wrt improving academic online learning wrt improving academic outcomes – we need to do more.outcomes – we need to do more.

3535

Online access to this presentationOnline access to this presentation

http://www.cbcse.orghttp://www.cbcse.org

(Publications)(Publications)

3636

Effect size – a standardized Effect size – a standardized measure for comparing measure for comparing

interventionsinterventions

Effect size = Effect size =

(mean score for Treatment group –(mean score for Treatment group –

mean score of Control group) / pooled mean score of Control group) / pooled standard deviation for both groupsstandard deviation for both groups

Conventionally, effect size of 0.9 is Conventionally, effect size of 0.9 is considered large, 0.45 medium and considered large, 0.45 medium and 0.15 small0.15 small

3737

Calculating effect size (Cohen’s d)Calculating effect size (Cohen’s d)

n = number of children in groupn = number of children in group

s = standard deviations = standard deviation

t = treatment groupt = treatment group

c = control groupc = control group

ct

cctt

ct

nnsnsn

xxd

22 )1()1(