A preliminary analysis of the first year of New Hampshires 1:1 laptop program Technology Promoting Student Excellence : A preliminary analysis of the first.

Technology Promoting Student Excellence:

A preliminary analysis of the first year A preliminary analysis of the first year of New Hampshire’s 1:1 laptop programof New Hampshire’s 1:1 laptop program

Damian Bebell

Lynch School of Education

Boston College

[email protected]

Full evaluation report and PowerPoint slides will be

available for download at :www.intasc.org

History of Educational Technology

Purported Benefits• Motivate and Engage students• Increase resources and information• Exposure to technology (21st century skills)• Improve teaching (lesson plans, communication)• Movement towards student centered classrooms• Streamline record keeping• Special Needs/Accommodations• Differentiate Instruction/learning

Trends in Educational Technology

• In the late 80’s a trend emerged that technology was good for students – so that they will be prepared for the technology jobs/skills of the future

• Current thinking is that technology is a vehicle for improved student learning of traditional curriculum

Student : Computer Ratios

• Standard Metric of technology access– National Student:Computer Ratios

• 125:1 = 1983• 9:1 = 1995• 6:1 = 1998• 4:1 = 2003Source: Market Data Retrieval, 1999; Education Week, 2004

Technology is sharedTechnology is shared

Use is sporadicUse is sporadic(lots of research)(lots of research)

Movement towards 1:1 technology

• Each teacher and student has full access to a computer (usually laptop)

• Previous research suggests:– Equity issues disappear– Technology becomes the relied upon tool students use

for research, writing, and presentation– Classroom management is simplified– Students are more engaged enthusiastic

Current 1:1 laptop programs

• Maine Learning Technology Initiative (2002)– All 7th and 8th grade students and teachers in 239 middle

schools – Apple iBooks, wireless classrooms,teacher training, support,

and professional development– 1,000 flowers blossoming philosophy

• Henrico County, VA (2002)– Apple iBooks in Grades 6th-12th– 20,000+ students– No systemic research or evaluation

• Massachusetts (3 Berkshire middle schools)• Andover, MA (Toshiba laptops--parent purchase)

Current 1:1 laptop programs (2)

•Michigan–Cross platforms–Looking into less expensive technology (i.e. Palms)

•Sedgwick, KS (2002)–Apple iBooks in middle school

•Texas•Florida•Georgia•Vermont

New Hampshire Question:

• Would the initial positive findings from Maine’s 1:1 laptop program generalize to six New Hampshire middle schools?

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Photo taken from: http://www.state.nh.us/governor/ (1/28/04)

History of TPSE

Fastest roll out of a 1:1 technology program ever!!!

History of TPSE (2)

"Technology must be heavily leveraged in a way to enhance the classroom experience and excite the student's passion for learning,”

In a perfect world, this would mean each of

the participating student would be excited each morning in anticipation of coming to school and no student would ever want to leave after school ends.”

Benson’s Philosophy:

Source:: http://www.state.nh.us/governor/

History of TPSE (3)

• Improve educational teaching, learning and achievement

• Use an interactive instructional practice

• Bridge the significant digital divide

• Create a highly educated, technology-savvy workforce

1:1 laptops would…

Source: Benson PowerPoint 9/2/03

TPSE Timeline (3)

9/2/03: Program announced10/15/03: Competitive material submitted10/03: 24 private organizations donate over

$1.2 million to fund the program (no public funds)

11/3/03: Participating schools announced12/03: Installation of technology

» Teachers receive laptops» Teacher training

January 5-6, 2004: iBooks distributed to 7th grade students

Participants

6 New Hampshire middle schools selected from approximately 20 applications

•Armand R. Dupont School (Allenstown, NH) •Indian River Middle School (Canaan, NH) •Haverhill Cooperative Middle School (N. Haverhill, NH) •Paul School (Sanbornville, NH) •Thornton Central School (Thornton, NH) •Winnisquam Regional Middle School (Tilton, NH)

400+ students400+ students 40 teachers40 teachers

Program Characteristics

• Apple iBook laptops for teachers and students• Software• Digital cameras and digital video cameras• Wireless school wide networks• Printers• Teacher training• Technology support

Study Design/Methodology

Try and capture the initial impact/effect of the the laptop program (Jan. 04)

Convince stakeholders that research and evaluation was an important component of the initiative

PRE/POST DesignPRE/POST DesignPre

Survey

Post Survey

Follow Up Web Survey

Teacher Survey

Jan. 04 May/June 04

Oct. 05

Student

Survey

Jan. 04 May/June 04

---

Study Design/MethodologySpring 2004 Student Surveys (n=862 both pre and post)1. Measures of technology use in school2. Technology use at home3. Across subject area4. Personal comfort level with technology

Spring 2004 Teacher Surveys (n=47 both pre and post)1. Measures of technology use in school2. Measures of technology use beyond the classroom3. Across subject area4. Personal comfort level with technology5. Demographic Information6. Attitude toward technology

Study Design/MethodologyFall 2004 Teacher Survey Follow Up (n=32)

Capture teachers perceptions about the specific impacts of the 1:1 program on students, teaching and learning

•Collected Online•http://corvus.bc.edu/nhteacherfollowup/NH_teacher_followup.cfm

•Survey items adapted from teacher survey measures created for use in Maine

•Specific focus on how technology has impacted different groups of students

•Traditional students•High Achieving Students•At-Risk or Low Achieving Students

http://corvus.bc.edu/nhteacherfollowup/NH_teacher_followup.cfm

Study Design/MethodologyNo systematic examination of achievement…yet

– First need to measure impact and level of us– No shared assessment in NH at the 7th grade– Need previous measures of student achievement– Individual student data– Difficulty in getting comparison groups to participate– Teachers perception of technology impact on

achievement

Today’s presentation focuses on Today’s presentation focuses on the immediate (1st six months) the immediate (1st six months) impact of the laptop programimpact of the laptop program

Student Level Results/Findings

Estimated # of 7th grade

Students

Pre (1/04) Responses

Post (5/04) Responses

Dupont 63 61 58

Haverhill 71 73 47

Indian River 118 100 96

Paul 80 64 61

Thornton 28 27 28

Winnisquam 175 114 134

Student survey response rate

Students used technology in school substantially more across all six 1:1 settings

How often do you use technology in your classroom? (Q1A)

1 2 3 4 5

Total

Wiinsquam

Thornton

Paul

Indian River

Haverhill

Dupont

Pre Post

Pre 2.28 1.67 3.93 2.17 1.93 2.4 3.22Post 4.68 4.71 4.52 4.85 4.38 4.85 4.9

Total WiinsquamThornton Paul Indian River HaverhillDupont

15 min or less

15 -60 minNever 1-2 hrs 2+ hrs

Students used technology in school substantially more in English/Language Arts

How often do you use a computer in Reading/ELA? (Q3A)

1 2 3 4 5

Total

Wiinsquam

Thornton

Paul

Indian River

Haverhill

Dupont

Pre Post

Pre 2.32 1.44 4.04 2.54 2.11 2.92 2.62Post 4.39 4.29 4.15 4.75 4.65 3.88 4.36

Total WiinsquamThornton Paul Indian River HaverhillDupont

15 min or less 15 -60 minNever 1-2 hrs 2+ hrs

Students used technology in school substantially more in Social Studies

How often do you use a computer in Social Studies? (Q3B)

1 2 3 4 5

Total

Wiinsquam

Thornton

Paul

Indian River

Haverhill

Dupont

Pre Post

Pre 1.74 1.32 1.26 1.19 1.97 1.77 2.62Post 4.12 4.15 4.08 4.68 3.36 4.16 4.36

Total WiinsquamThornton Paul Indian River HaverhillDupont15 min

or less15 -60 minNever 1-2 hrs 2+ hrs

Students used technology in school substantially more in Math

How often do you use a computer in Math? (Q3C)

1 2 3 4 5

Total

Wiinsquam

Thornton

Paul

Indian River

Haverhill

Dupont

Pre Post

Pre 1.24 1.07 1.89 1.08 1.27 1.26 1.38Post 3.3 3.02 3.19 3.54 4.38 4.59 4.09



Students used technology in school substantially more in Science

How often do you use a computer in Science? (Q3D)

1 2 3 4 5

Total

Wiinsquam

Thornton

Paul

Indian River

Haverhill

Dupont

Pre Post

Pre 1.68 1.27 1.96 1.27 1.34 2.31 2.6Post 4.11 4.05 3.19 3.34 4.6 4.24 4.16



Students used technology for wide variety of tasks

How often do you a computer in school to:

1 2 3 4 5

find information on theinternet

edit papers using acomputer

write first drafts

play computer games

create Hyperstudio orPowerpoint presentation

solve problems

create graphs or tables

analyze data

work withspreadhseets/databases

take a test or quiz

Sending and receivingemail

Pre Mean

Post Mean

15 min or less


How often does your teacher use a computer for each subject? (Q7)

0 1 2 3 4 5

Reading/ELA

Social Studies

Math

Science

Mean PreMean Post

Teachers use of technology as reported by the students


R

15 min or less

Students use of technology at home

How often do you use your home computer to: (Q12)

1 2 3 4 5

play games

search the 'net for school

chat/instant message

email

search the 'net for fun

write papers for school

download/play mp3/music

computer program

create/edit digital photos or movies

create/maintain web sites

Total PreTotal Post

15 min or less


How well are you able to:

1 2 3 4

find information on theInternet

write papers using acomputer

create multimediapresentations

record and analyze datausing a spreadhseet

communicate using email

Pre

Post

Not Well Very Well

Students reported technology skills

Pre/Post SY (03/04) Teacher Results

Teacher survey response rate less sure than student survey

Typically Math, Science, English/language Arts, or Social Studies teachers completed the survey, however around 10% of respondents were not primary classroom teachers

Teacher Pre/Post Beliefs and Confidence Measures

How important are computers in your teaching? (Q3)

1 2 3

Post

Pre

Series1 1.68 2.44Post Pre

How confident are you when using computers? (Q4)

1 2 3

Post

Pre

Series1 1.68 2.04

Post Pre

1=Very Important 2=Somewhat Important 3=Not Very Important

1= Very Confident2=Somewhat Confident3=Not Very Confident

Teachers Use of Technology

1 2 3 4 5

make handouts using computer

create test, quiz or assignment

perform research and lessonplanning using the web

communicate with teachers, parentsand administrators via email

adapt an activity to student specialneeds

deliver instruction with computer

use a computer to present info toclass

use a computer to help student betterundertstand a concept

creat WebQuest or build the webinto a lesson

access studnets via computer

use a computer to modelrelationships/functions

create/maintain web sites

prepare or maintain IEPS

PrePost

Pre 3.84 3.68 3.08 2.68 2.08 1.84 1.76 1.71 1.6 1.6 1.52 1.4 1.28Post 4.5 4.5 4.5 4 3.86 4.36 4.14 4.09 3.36 3.23 3.36 1.86 1.57

make handouts

using computer

create test, quiz or

assignment

perform research

and lesson planning

communicate with teachers, parents

adapt an activity to student special

deliver instruction

with computer

use a computer to present info to

use a computer to help student

creat WebQuest or build the web

access studnets

via computer

use a computer to model relationshi

create/maintain web

sites

prepare or maintain

IEPS

R

Once or twice a year

Several times per year

Never Several times per month

Several times per week

1 2 3 4 5

reports and termpapers

pictures/art work

graphs or charts

stories or books

multimedia projects

web pages, etc.

videos or movies

Pre

Post

Pre 1.88 1.88 1.56 1.48 1.46 1.16 1.16

Post 3.45 2.68 2.82 1.95 3.09 1.57 2.5

reports and term papers pictures/art work graphs or charts stories or books multimedia projects web pages, etc. videos or movies

How often do teachers assign students to create the following products using technology?

Once or twice a year

Several times per year

Never Several times per month

Several times per week

Also observed changes in teachers beliefs towards learning and technology?

Increase in the belief that students interact with each other more

while working with computers

Students work harder when using computers

Technology allows students to create better looking finished products

Overwhelming agreement among participating teachers that :

Students are more willing to write second drafts when using a computer

Students develop a deeper understanding of the material when using technology

Teacher Survey Follow Ups (Oct. 04)

32 teachers across the six schools responded to an online follow up survey (n=32 in all cases)

My goals for students have changed

18.8

37.5

25.0

15.6

3.1

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

Strongly Agree Agree Neutral Disagree StronglyDisagree


The school climate has changed

15.6

65.6

18.8

0.0 0.00.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0


My role in the classroom has changed

12.5

62.5

9.412.5

3.1

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0



Student acheivement has improved

15.6

50.0

31.3

3.10.0

0.0

10.0

20.0

30.0

40.0

50.0

60.0



My understanding of how people learn has changed

6.3

40.6

37.5

12.5

3.1

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0


My beliefs about teaching and learning have changed

6.3

56.3

18.815.6

3.1

0.0

10.0

20.0

30.0

40.0

50.0

60.0



My computer skills have improved

56.3

43.7

0.0 0.0 0.00.0

10.0

20.0

30.0

40.0

50.0

60.0



The delivery of curriculum in my classes has changed

21.9

65.6

12.5

0.0 0.00.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0


I have had adequate professional development opportunities

9.4

50.0

25.0

15.6

0.00.0

10.0

20.0

30.0

40.0

50.0

60.0



Students have greater freedom of choice with regard to their individual learning style

21.9

53.1

21.9

3.10.0

0.0

10.0

20.0

30.0

40.0

50.0

60.0


The school has developed effective policies and procedures for the laptop program

34.4

50.0

9.46.3

0.00.0

10.0

20.0

30.0

40.0

50.0

60.0



My ability to assess and evaluate student work has improved

15.6

21.9

40.6

18.8

3.1

0.0

5.0

10.0

15.0

20.0

25.0

30.0

35.0

40.0

45.0



Participation in class

3.10

3.1

12.56.3

34.4

84.4

93.8

62.5

0

10

20

30

40

50

60

70

80

90

100

Declined 3.1 0 3.1

No Effect 12.5 6.3 34.4

Improved 84.4 93.8 62.5

Traditional studentsAt-risk/low achieving

studentsHigh achieving

students


Motivation

3.16.3

3.13.1

15.6 15.6

93.8

81.3 81.3

0

10

20

30

40

50

60

70

80

90

100

Declined 3.1 6.3 3.1

No Effect 3.1 15.6 15.6

Improved 93.8 81.3 81.3



students


Engagement/interest level

3.1 0 3.16.3 3.1

15.6

90.696.9

81.3

0

20

40

60

80

100

120

Declined 3.1 0 3.1

No Effect 6.3 3.1 15.6

Improved 90.6 96.9 81.3



students


Ability to work independently

3.10

3.1

2528.1

43.8

71.9 71.9

53.1

0

10

20

30

40

50

60

70

80

Declined 3.1 0 3.1

No Effect 25 28.1 43.8

Improved 71.9 71.9 53.1



students


Ability to retain content material

3.10

3.1

46.9

40.6

5050

59.4

46.9

0

10

20

30

40

50

60

70

Declined 3.1 0 3.1

No Effect 46.9 40.6 50

Improved 50 59.4 46.9



students


Quality of work

3.10

3.1

21.9 21.9

34.4

7578.1

62.5

0

10

20

30

40

50

60

70

80

90

Declined 3.1 0 3.1

No Effect 21.9 21.9 34.4

Improved 75 78.1 62.5



students


Interaction with teacher

6.30

6.3

15.612.5

34.4

78.1

87.5

59.4

0

10

20

30

40

50

60

70

80

90

100

Declined 6.3 0 6.3

No Effect 15.6 12.5 34.4

Improved 78.1 87.5 59.4



students


Interaction with other students

3.10

3.19.4

15.621.9

87.584.4

75

0

10

20

30

40

50

60

70

80

90

100

Declined 3.1 0 3.1

No Effect 9.4 15.6 21.9

Improved 87.5 84.4 75



students


Quality of student writing

3.10

3.1

34.4 34.431.3

62.565.6 65.6

0

10

20

30

40

50

60

70

Declined 3.1 0 3.1

No Effect 34.4 34.4 31.3

Improved 62.5 65.6 65.6



students

The Rise and Fall of 1:1 Computing

•The cost of going 1:1 is great •$40 million in Maine, •$1.2+ million in NH•Over $25 million in VA

•There is a climate of great pressure on schools to demonstrate that money spent is directly and positively impacting students

•Current definition of impacting students is increased performance as measured by a standardized test (accountability at all levels, AYP, etc)

The Rise and Fall of 1:1 Computing (2)

It is estimated that the measurable impact of educational technology investments can take 4+ years

Stakeholders expect to see measurable differences in student scores given their investments (typically not very patient)

Rise and Fall of 1:1 Computing

Research/Evaluation is costly (time and money)

Methodological ChallengesGood accurate measures of technology “use” (STEP 1)Valid measurement of student achievement

Issues with paper based tests for high-tech students(Russell, 1999; 2001; 2002)Measures of prior achievement

IDEAL QUESTION:What kind of technology use leads to what kind of achievement gains (for x kind of student)?

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Laptop flop; Maine results should give pause to NH

MAINE'S two-year-old experiment with laptop computers in the classroom has flopped as measured by student test scores. Now maybe this nonsensical trend will fizzle out and we can get back to spending time and money on educating students instead of buying them expensive tools that don't help them learn the basics. Maine has laid out $37.2 million on a four-year experiment that provides laptop computers to all students in grades 7 and 8…Manchester Union Leader August 12, 2004 Page A16

A preliminary analysis of the first year of New Hampshires 1:1 laptop program Technology Promoting Student Excellence : A preliminary analysis of the first.

Documents

technology slide

technology program

laptop program technology

technology spring

expensive technology

technology jobsskills

measures of technology

instructionlearning