Closing the digital divide: evaluation of the World Links … · ARTICLE IN PRESS International Journal of Educational Development XX (2004) XXX–XXX Closing the digital divide:
Post on 21-Apr-2018
213 Views
Preview:
Transcript
ARTICLE IN PRESS
International Journal of Educational Development XX (2004) XXX–XXXwww.elsevier.com/locate/ijedudev
Closing the digital divide: evaluation of the World Linksprogram
Robert Kozma∗, Ray McGhee, Edys Quellmalz, Dan ZallesCenter for Technology in Learning, SRI International, 2151 Filbert Street, San Francisco, CA 94123, USA
Abstract
In response to this digital divide between developed and developing countries in their use of computers to preparestudents for the global economy, the World Bank and, subsequently, the World Links organization provided schoolsin developing countries with networked computers and training that supports integration of ICT into teaching. Thisarticle synthesizes findings from 3 years of evaluative research on the program. The findings are based on surveys ofteachers, students, administrators, and technology coordinators, as well as a field test assessment of student learning.The study examines the services provided, documents their impact, and draws implications for policy in developingcountries. 2003 Elsevier Ltd. All rights reserved.
Keywords: International development; Educational technology; Educational policy
1. Introduction
Information and communication technology(ICT) can make a tremendous contribution tohuman development—but only for those that haveaccess. Technological innovation affects humandevelopment in two ways (United Nations Devel-opment Programme, 2001). It can directly contrib-ute to human capabilities by increasing people’sability to participate more actively in the social,educational, economic and political life of a com-munity. It can also support economic growththrough the productivity gains that it generates.Conversely, human development—particularly the
∗ Corresponding author. Tel.:+1-415-292-2471; fax:+1-415-292-4343.
E-mail address: robert.kozma@sri.com (R. Kozma).
0738-0593/$ - see front matter 2003 Elsevier Ltd. All rights reserved.doi:10.1016/j.ijedudev.2003.11.014
development of a highly skilled workforce—can inturn contribute to technology development.Together, these two developments can create a“virtuous circle” that can reduce poverty andimprove the human condition.
Unfortunately, these same technologicalresources are currently creating significant dispari-ties between developed and developing countriesaccording to aUnited Nations Development Pro-gramme report (1999). For example, the reportpoints out that in South Africa, the best-connectedAfrican country, 75% of the schools have no tele-phone line. For the sake of comparison, there are99 telephones for every 100 people in Monaco, butin Uganda, there are only two for every 1000. Thereport concludes that, “The global gap betweenhave and have-nots, between know and know-nots,is widening” (p. 57).
This growing digital divide is further docu-
ARTICLE IN PRESS2 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
mented by several more recent reports. Bridges.org(2001) states that while there were 167 millionInternet users in North America and 113 millionin Europe, there were only 16 million in SouthAmerica and 3 million in all of Africa, accordingto data at the time of the report. While there hasbeen a dramatic increase in Internet users indeveloped countries—from an estimated 4% of theinhabitants in 1995 to 28% in 2000—the numberof users in developing countries has only inchedforward to only 3.6% in 2000. Similarly, a studyby the Organization for Economic Cooperation andDevelopment (OECD, 2001b) shows that there hasbeen a dramatic growth of Internet hosts—thesource of content on the Internet. But while thenumber of hosts in developed countries hasincreased from 23 per 1000 inhabitants in 1997 to82 per 1000 in 2000, the number in developingcountries has only increased from 0.21 per 1000 in1997 to 0.85 per 1000 in 2000. This lack of contentoriginating in and oriented to developing countriesis further compounded by the fact that 94% of theworld’s Internet content, as measured by the num-ber of links to pages on secured servers, is inEnglish.
The lack of access to technology reduces theprospect that citizens of developing countries willbe able to participate in the growing global econ-omy and minimizes the potential that technologyhas for improving their health, educational,governmental, and cultural institutions (UnitedNations Development Programme, 1999).
2. The World Links program
In response to this growing digital disparity andto requests from developing countries to assistthem in preparing their youth to enter an infor-mation age and participate in the global economy,the World Bank’s Economic Development Instituteconducted a pilot program, initially called WorldLinks for Development, from 1997 to 2002. Themission of the World Bank is to fight poverty andcontribute to sustained development (World Bank,1999). In its comprehensive development frame-work, the Bank posits that sustainable developmentrequires many social and structural elements in
addition to strong economic performance. Thedevelopment of educational capacity in clientcountries is a key element of the Bank’s strategy.While the primary mechanism for Bank support isthrough its program of strategic loans, the Bank’sEconomic Development Institute (now the WorldBank Institute, http://www.worldbank.org/) con-tributes to capacity building through training andoutreach programs (World Bank, 1997).
The Economic Development Institute initiatedthe World Links for Development in the belief thattechnology could be used to improve educationaloutcomes in client countries and facilitate culturalunderstanding across nations (World Bank, 1997).The program was started by connecting a singlehigh school in Uganda with one in the UnitedStates. By 1999, the program reached over 300schools in 15 countries. In 2000, the World Bankcreated an independent non-profit organizationcalled World Links (http://www.world-links.org/)to continue the program once the pilot was com-pleted. World Links has since grown to serve overa 1000 secondary schools in 26 developing coun-tries in Africa, Latin America, the Middle East, andSouth and Southeast Asia.
The World Links program aims to establish glo-bal, educational on-line communities for secondaryschool students and teachers around the world inorder to expand distance learning opportunities,enhance cultural understanding across nations,build broad support for economic and social devel-opment, and train teachers to integrate informationtechnology into the classroom. There are fivecomponents to the World Links program, eachelaborated below:
� Internet connectivity for secondary schools indeveloping countries.
� Training and educational content to promoteeconomic and social development.
� Regional and global partnerships with public,private, and non-governmental organizations.
� Telecommunications policy advice for the edu-cation sector.
� Monitoring and evaluation support.
Internet connectivity has been an essentialcomponent of the World Links program. For coun-
ARTICLE IN PRESS3R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
tries and schools with the most need, the programprovided up-to-date multimedia computer equip-ment and networking for the school and computerlaboratory. The program provided several alterna-tive networking models, depending on the net-working resources and bandwidth that were avail-able.
Training was also an essential component. Acentral tenet of the program is that the introductionof technology will alone not improve education.Teachers from participating schools received train-ing in the use of equipment and typical software.But more important was training on the integrationof ICT into classroom teaching. The programadopted a constructivist approach to teaching withICT that emphasized a shift away from teacher-centered lecture-based instruction towards student-centered, project-based learning (Carlson, 2002).Among the ultimate goals of the program was toencourage students’ development of higher-orderthinking and information-reasoning skills. Theseskills are cited as particularly important for partici-pation in the knowledge-based global economy(OECD, 1996, 2001a; 21st Century Partnership,2003). To accomplish this goal, World Links train-ing was to provide teachers with pedagogicalapproaches to incorporate ICT into their teachingand support student acquisition of advanced skillsinstructional goals. By 2000, the World Linksteacher training curriculum included the followingfour phases:
Phase I: Introduction to the Internet for teachingand learning.Phase II: Introduction to educational telecollabor-ative projects.Phase III: Integrating technology and curricula.Phase IV: Evaluating and diffusing innovativeclassroom practices.
The program also created a set of regional andglobal partnerships, such as I∗EARN, GLOBE,and ThinkQuest, that created digital content, activi-ties, and training materials for teachers and stu-dents. These partnerships and their services sup-ported teachers in their implementation of ICT andthe pedagogical strategies advocated by WorldLinks.
The program also provided ministries of edu-cation and other governmental agencies with con-sultation and policy materials that supported thedevelopment of ICT and its use to improve edu-cational systems. Among the advice was the advis-ability of drawing on World Bank lendingresources to support additional programs that useICT in light of the results of the pilot program intheir country.
The fifth component of the World Links pro-gram was monitoring and evaluation. To informthe decisions of national policy makers and pro-gram managers, an evaluation was designed toexamine the World Links program on theimplementation and effectiveness of the inter-vention and barriers that need to be overcome toassure its success. The evaluation was conductedby the Center for Technology in Learning at SRIInternational (http://www.sri.com/policy/ctl/).
This article summarizes the findings from evalu-ative research that was conducted over a 3 yearperiod (Kozma and McGhee, 1999; McGhee andKozma, 2000; Quellmalz and Zalles, 1999, 2000).The focus of the evaluation was on documentingthe services actually provided by the program,studying the classroom implementation of the pro-gram and the effect that it had on classroom prac-tice, examining initial indications of student andteacher outcomes and long-term impacts, and ana-lyzing the barriers that may limit the effectivenessof the program. Our findings indicate that the pro-gram provided a variety of intended services toschools and teachers and that these were effec-tively implemented in participating schools andclassrooms. The use of ICT in these schools wasassociated with student-centered, constructivistpedagogical practices. The data we were able toobtain indicate that students acquired the skillsthey would need to participate in the global knowl-edge economy. A range of barriers—from the needfor more reliable connections in schools, to theneed for more time in the curriculum for computer-based activities and national policies that integrateICT into education reform—limited the program’simplementation and impact. But the extent of theprogram’s success despite these barriers has impli-cations for the use of ICT to support education
ARTICLE IN PRESS4 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
improvement in developing countries and for thepolicies that can increase its impact.
3. Study design
As mentioned above, the goal of our evaluationwas to document the implementation of the WorldLinks program and assess its effectiveness. Thefocus of the evaluation was on the services pro-vided by the program, the extent to and ways inwhich these services were implemented in theclassroom, and student and teacher outcomes andthe extent to which they could be attributed to theprogram. In addition, we were concerned with thebarriers that teachers encountered in implementingthe program. Our purpose was to provide programstaff with information that could be used toimprove the program and to provide governmentswith information about the value of ICT invest-ments in education and polices that may be neededto increase the impact of these investments.
The evaluation of the World Links program wassituated within a conceptual framework in whichthe impact of educational improvement efforts isinfluenced by a number of variables, in addition tothe intervention itself, which mediate the impact ofthe program (see Fig. 1). The ultimate goal of the
Figure 1. Conceptual Framework for World Links Evaluation
World Links program was to contribute to suchoutcomes as improving the technological andinformation skills of students and teachers andlong-term impacts such as improved test scores,increased cross-cultural understanding, andimproved job prospects in the developing knowl-edge economy. However, these ultimate goals aremediated by a variety of intermediate factors, mostsalient being the extent to which the services pro-vided by the program are implemented and sup-ported by the participating schools. In addition,there are other “exogenous” variables unrelated tothe program which may influence the implemen-tation of the program and its outcomes andimpacts, factors over which the program and theBank may have very little influence. We used thisconceptual framework to guide the construction ofour research instruments. Within this framework,our focus was on the services of the program, theways their implementation affected classroompractice, and the impact of these practices, parti-cularly on the skills students would need to partici-pate in the knowledge economy.
Previous research suggests that the services pro-vided by the World Links program—specificallythe ICT infrastructure and teacher training—wouldhave an effect on classroom practice, particularlythe training of teachers to integrate ICT into their
ARTICLE IN PRESS5R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
classes. Classroom studies (Means and Olson,1995; Sandholtz et al., 1997; Means et al., 2001;Schofield and Davidson, 2002) have shown thatsuch interventions can have a direct effect on thepedagogical strategies used by teachers, the extentto which computers are used within these stra-tegies, the amount of time students are engaged inthe use of computers, and the kinds of projects thatstudents conduct with computers. The examinationof this relationship was a primary focus of theevaluation.
The classroom implementation, so influenced,may have a subsequent effect on short-term out-comes, such as the acquisition of certain knowl-edge, skills, and attitudes. Ultimately, the inter-vention may have longer-term impacts on studentretention rates or their preparation for the workworld. The goal often cited in the literature is toprovide students with the ability to access, analyze,evaluate, communicate, and use information tosolve problems and create new knowledge(Educational Testing Service (ETS), 2002; Inter-national Society for Technology in Education(ISTE), 2000; OECD, 2000; Quellmalz andKozma, 2003)—skills that are needed to participatein the global knowledge economy. However, therelationship between the use of ICT and variousoutcomes and impacts is an area where researchresults are less conclusive, with some studies find-ing negative results (Pelgrum and Plomp, 2002;Wenglinski, 1998) and some finding positiveresults (NCES, 2001; Wenglinski, 1998). The con-flicting results suggest that the mere introductionof computers is insufficient to affect outcomes; theway computers are used in the classroom matters.In our evaluation, we examined this relationshipbetween the use of computers, the use of specificpedagogical practices, and certain student out-comes. We designed this as a comparative studyso that some attributions could be made to the ser-vices provided by the program.
The evaluation was also situated within a practi-cal context of limited time and budget. As in manyevaluations, these pragmatic constraints influencedthe way in which we were able to assess the goalsand examine the relationships imbedded in ourconceptual framework. We had to rely on the self-report of participants for much of our data. How-
ever, we triangulated findings by the use of mul-tiple sources (students, teachers, technology coor-dinators, and administrators), periods (the programin 1999 and 2000), participating countries, andmethods (Tashakkori and Teddlie, 1998; Shavelsonand Towne, 2001). Survey instruments weredesigned to collect information on the services pro-vided by the program, teacher and student class-room practices and the extent to which computerscontributed to these, the effect of the program andthe use of computers on teacher and student knowl-edge, skills, and attitudes. We collected data fromstudents, teachers, technology coordinators, andadministrators. Also, sets of performance assess-ments were specifically designed to directly meas-ure the impact of the program on student learning.To address certain research questions related torelative effects of the program vs. the mere useof computers, we included comparison groups ofteachers and students who used computers but didnot participate in the program, as described below.
3.1. Surveys of students, teachers, andadministrators
The purpose of the surveys was to collect self-reported responses about the program servicesreceived, the implementation of these services inthe classroom, and the outcomes and impact of theprogram on students and teachers. Survey datawere collected during the 1998–1999 and 1999–2000 operational years. During 1998–1999, surveydata were collected from samples of adminis-trators, teachers, technology coordinators, and stu-dents from five countries. During 1999–2000, sur-vey data were collected from more countries (12in all) but only from teachers.
In 1998–1999, the evaluation was conducted infive of the 13 currently participating countries:Chile, Paraguay, Peru, Senegal, and Uganda. Ineach of the five countries in which the evaluationwas conducted, six World Links schools wereasked to participate in data collection. Because theresponses were self-reports, the data were triangu-lated with multiple types of respondents. At eachschool, samples of students and teachers were sur-veyed along with the headmaster or headmistressand the school’s technology coordinator. In total,
ARTICLE IN PRESS6 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
26 World Links schools participated in the evalu-ation. There were nearly 20,000 students served bythese schools, more than 12,000 girls and nearly8000 boys. There were over 1200 teachers in theseschools. A total of 661 World Links students and83 World Links teachers responded to the surveyfrom these schools. Also 25 administrators and 23technology coordinators responded. The responserate was at or above 80% for all schools. Becausewe wanted to assess the extent to which studentand teacher outcomes could be attributed to theWorld Links program, two other groups wereidentified for comparison purposes. One was agroup of non-participating or “ least participating”teachers and students within World Links schools.These were teachers and students in participatingschools who did not participate in the program orparticipated in a minimal way; respondents in thisgroup included 191 teachers and 441 students. Asecond comparison group was administrators, tea-chers, and students from nine schools that did notparticipate in the program but were asked to par-ticipate in the evaluation based on their compar-ability to World Links schools. Within theseschools, 90 teachers, 378 students, and nine admin-istrators responded to surveys. To assure parti-cularly useful comparisons, both computer usingand non-computer using teachers and studentswere surveyed in non-World Links schools. Of thetotal respondents in non-World Links schools, 32teachers and 247 students were computer users.1
Relating to pedagogical practices and outcomes,survey questions were constructed so that identicalresponses could be collected from both WorldLinks participants and non-participants and bothcomputer users and non-users. For example, ques-tions were asked of all respondents about the useof practices such as collaborating with other stu-dents on a project, gathering data for a researchproject, writing project reports, and so on. If the
1 The total number of respondents by country in 1998–1999was: Students—Chile (255), Paraguay (222), Peru (390), Sen-egal (262), and Uganda (350); teachers—Chile (48), Paraguay(47), Peru (109), Senegal (76), and Uganda (84); technologycoordinators—Chile (6), Paraguay (4), Peru (6), and Uganda(6); administrators—Chile (6), Paraguay (6), Peru (7), Senegal(8), and Uganda (6).
respondents used computers, they would then beasked what role computers played in these prac-tices. If they were participants in the World Linksprogram, they would then be asked the extent towhich their participation contributed to the prac-tices. Consequently, the occurrence of certain prac-tices could be compared between program parti-cipants and non-participants and between computerusers and non-users to see if there were differ-ences. Subsequent questions could provideadditional information about the extent to whichthe respondents felt that the use of computers orthe participation in the program contributed to anydifferences that existed.
In 1999–2000, 12 of 15 countries that were part-icipating in the World Links program at the timewere included in the study. These were: Brazil,Chile, Colombia, Ghana, Mauritania, Mozam-bique, Paraguay, Peru, Senegal, South Africa,Uganda, and Zimbabwe. The intent of this studywas to see if 1998–1999 findings held for newcountries that had subsequently joined the pro-gram. As mentioned above, data were collectedonly from teachers. The design included bothWorld Links and non-World Links schools: 98 and42, respectively. In World Links schools, four tea-chers were included for each school. To examinethe impact of the program on a broader range ofteachers in participating schools, two of the fourwere teachers who had received formal trainingfrom the program; one had received training froma colleague; and one teacher had not received train-ing. A total of 383 World Links teachers partici-pated in the survey and 158 teachers responded innon-World Links schools; the response rate was97%.2
3.2. Assessment of student learning
Because we wanted to go beyond self-report ofthe impact of the program on student outcomes,we devised a direct measurement of student
2 The total number of teacher respondents by country in1999–2002 was: Brazil (56), Chile (40), Columbia (42), Ghana(40), Mauritania (36), Mozambique (32), Paraguay (36), Peru(56), Senegal (56), South Africa (56), Uganda (55), and Zim-babwe (36).
ARTICLE IN PRESS7R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
achievement. There were some unique challengesin the design of student learning assessments forthis study. First, the World Links program wasimplemented in a wide range of subject areas,including science, social studies, language, andcomputer science. Furthermore, different countrieshad different curricula and expectations for studentlearning. Both these factors made it problematic todesign an instrument that would measure the learn-ing of specific school subject matter. However, anoverriding consideration in designing the studentassessment was the interest in measuring one ofthe most important outcome goals of the program:to prepare students for work in the knowledge-based global economy. Consequently, rather thanmeasure the learning within the multitude of sub-ject areas in which the program was implemented,the assessment of student learning focused on ICTskills and the use of ICT to search for, organize,and communicate information. These are the kindsof skills that are most likely to be influenced by theprogram, regardless of the subject area in which itis implemented, and they are the skills identifiedin policy documents (ISTE, 2000; OECD, 1996,2001a; World Bank, 2002) as those needed forworkers in the global knowledge economy.
With this goal in mind and drawing on the Edu-cational Technology Standards for Studentsdeveloped by the International Society for Edu-cational Technology (ISTE, 2000), the SRI teamidentified key skill components for three outcomeareas: reasoning with information, communication,and technology use. The SRI team then designedsets of performance assessments that engaged stu-dents in complex tasks of searching for, organiz-ing, and analyzing information and designed indi-cators that measured skill components, such asformulating a search query and communicating anargument (Quellmalz and Zalles, 1999, 2000). As aculminating task in one of the assessments, WorldLinks students were asked to write a newsletter forother students about the plight of two endangeredspecies. In response to this task, students gatheredinformation from Web pages about the twoendangered species, specified a line of inquiry forfurther research and a Web search query phrase,and then prepared a news article making evidence-based predictions about the likelihood that the two
species would survive. Students were also asked toinsert and annotate a relevant graphic. A parallel“paper-and-pencil” version of the task wasdesigned for students in the comparison group whodid not use ICT. The assessments were pilot testedin a World Links school in Uganda and in Parag-uay. Scoring rubrics were developed and scorerswere trained to assess the quality of studentsreasoning with information, communication, andtechnology skills.
The student assessment field test conducted in2000 involved six World Links schools and fourcomparable non-World Links schools in Uganda.There were a total of 200 students assessed; 121from World Links schools and 79 from non-WorldLinks schools. Students in non-World Linksschools were not assessed on their technologyskills.
4. Results
4.1. Services provided
Our evaluation began with an analysis of the ser-vices that the World Links program actually pro-vided to participating schools. The World Linksprogram is a distinctive blend of infrastructure andhuman capacity development. Perhaps, the mostvisible contribution of the World Links program isthe donation of computer hardware and software.Equipment was provided only to those schoolswhere the current lack of equipment limited theirparticipation in the program. At the time of the1998–1999 survey, the technology coordinators in60% of the schools reported that their schools hadreceived equipment from the program. The coordi-nators reported having received an average ofnearly nine computers for each school. The largemajority of these computers were located in a com-puter laboratory along with other computers thatthe school had previously acquired.
One of the outcomes of the World Links pro-gram—one that significantly increases itsefficiency—is the additional resources that thedonation of equipment and participation in the pro-gram fostered. The relatively modest donationsfrom the program were leveraged to get additional
ARTICLE IN PRESS8 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
resources, as they were often matched by signifi-cant local or national contributions. For example,the computer laboratories were often built withfunds from a country’s Education Ministry or fromlocal resources. In Senegal, the Ministry of Edu-cation built or remodeled laboratories at each ofthe World Links schools at a cost in excess of $20,000 for each school. Contributions of additionalequipment in Uganda were often made by old girlsand old boys clubs, as well. Other local funds, typi-cally student fees that ranged from $ 1 to $ 50 ayear, covered operating costs, such the cost ofpaper, ink, and dial up access. In most cases, stu-dents were able to use the resources even if theycould not afford to pay the fee. In some schools,parents and students supplemented computerresources through fund-raising events. Forexample, in a poor, all-girls school in Peru, stu-dents raised funds by performing musical and the-atrical shows for parents and neighbors.
While equipment donation was, perhaps, themost visible component of the program, it wasthrough the professional development of teachersthat the World Links program made its most sig-nificant contribution. In the 1998–1999 survey,World Links technology coordinators reported that336 of the teachers in their schools, or an estimated28% of their entire teaching staff, had receivedtraining as part of their school’s participation in theprogram. During the first year, most of the trainingtopics focused on the use of technology; 92% ofthe teachers said they received training on the useof the Internet and 75% received training on thedevelopment of Web pages. Teachers also saidthey received training on the use of computer hard-ware (70% of the participating teachers) and appli-cations software (64%). The workshops alsofocused on constructivist approaches to learningthat emphasized project-based learning and studentcollaboration. Nearly, 90% of the teachers saidthey received training on how to design and leadcollaborative student projects; 70% received train-ing on how to use student groups in their teaching;and 59% received training on how to collaboratewith other teachers on the development of instruc-tional materials.
These percentages were smaller for the largergroup of teachers in 12 countries that responded to
the survey in 1999–2000, due in part to theinclusion of a broader range of teachers in theWorld Links schools. In these schools, 62% of theteachers said they received training on designingand leading collaborative groups; 52% receivedtraining on using student groups in teaching; and45% received training on collaboration with teach-ers to develop instructional materials.
As a result of their experience, 93% of WorldLinks teachers in the five countries studied in1998–1999 expressed satisfaction with the way theprogram was implemented. In the 1999–2000 sur-vey, 90% the teachers in the 12 countriesexpressed satisfaction.
Administrators also received training as part oftheir school’s participation in the World Links pro-gram; only two of the 18 administrators whoresponded to the question in the 1998–1999 surveysaid they did not. Training in the use of Internetsoftware and in the design of collaborative studentprojects were mentioned most often as the topics ofthe training. Correspondingly, all 21 administratorswho responded to the question said they were sup-portive of the program and 91% said they weresatisfied with its implementation to date.
The school technology coordinator played a cru-cial role in teacher professional development.Technology coordinators were frequently teacherswho had some prior computer experience. Oftenthey taught mathematics or science courses. Thecoordinators served a large number and a wide var-iety of functions with 70% or more saying that theytrained teachers in the use of hardware, appli-cations software, the use of the Internet, and in thedesign of collaborative projects and ways to inte-grate technology into the curriculum. The coordi-nators also worked extensively with students. Morethan 80% of the coordinators said they trained stu-dents in the use of hardware, software, and theInternet. Many coordinators also serviced the hard-ware, with 57% saying that they maintained thecomputer equipment and 52% saying they adminis-tered the school’s network. These teachers oftenprovided these special services while continuingtheir teaching duties.
In summary, while many schools received com-puters from the World Links program and lever-aged these donations to obtain more resources, the
ARTICLE IN PRESS9R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
primary service provided by the program wasteacher training. Training covered both the use ofhardware, software, and the Internet, as well as theuse of computers within new, constructivist-ori-ented pedagogical strategies.
4.2. Implementation
The second focus of the evaluation was on theextent to which these services were used in theclassroom. We found that World Links training andthe availability of networked computers were asso-ciated with significant differences in what WorldLinks teachers and their students did in theirclassrooms, compared to computer-using teachersand students in non-World Links schools. In the1998–1999 survey, World Links students in fivecountries were much more likely than computer-using students in non-World Links to report thatthey engaged in a number of pedagogical practicesin their classrooms (see Table 1). Some of thesedifferences related to the use of various techno-logies. For example, while 52% of the World Linksstudents said they used the Internet and 40% saidthey used a search engine, only 16% of the com-puter-using non-World Links students said theyused the Internet and 19% said they used a searchengine. Similarly, while 34% of the World Linksstudents said they used email, only 21% of thecomputer-using non-World Links students saidthey did so. But some of the most dramatic differ-ences relate to other learning activities. Forexample, 38% of the World Links studentsreported that they gathered data for a research pro-ject, 31% collected information about anothercountry or culture, and 23% collaborated on a pro-ject with students from another country. On theother hand, 22% of computer-using non-WorldLinks students reported that they gathered data fora research project, 18% collected informationabout another country, and only 4% collaboratedon a project with students from another country.These differences held up when World Links tea-chers were asked parallel questions about the class-room practices of their students in the 12 countriesparticipating in the 1999–2000 study (see alsoTable 1).
As a result of the program’s emphasis on collab-
orative, project-based learning, World Links stu-dents frequently conducted projects with other stu-dents, often with students from other schools andother countries. The titles of some of the studentprojects in Uganda were Women in Mathematics,The Democracy Schools Project, The UtopianVision, Refugees, The Wetlands, Faces of War, andCulture and Technology. In Senegal, topicsincluded Technology and the Generation Gap,Oppression and Human Violence, Women andDevelopment, and Girls’ Education in Senegal.These projects were shared with students in Eur-ope, Canada, the United States, and South Koreafor their reaction. Box 1 describes a few of theWorld Links ecology-related student projects in abit more detail. In these projects, students workedtogether while using computers, software appli-cations, and the Internet to collect information,analyze it, write up reports, and post their resultson the Internet or communicate with students else-where.
In summary, participation in the World Linksprogram lead to the classroom use of a variety ofcomputer applications and resulted in significantdifferences in the use of collaborative, project-based pedagogical practices, relative to their useby non-participating schools.
4.3. Outcomes
The goal of the World Links program was toinfluence the learning outcomes of secondaryschool students and teachers in developing coun-tries and, ultimately, to have an impact on othervariables such as school attendance, examinationscores, graduation rates, and the employment ofstudents. While outcome and impact variables arethe most difficult to measure and most susceptibleto a range of influences apart from the program,we attempted to get an initial indication of the pro-gram’s effect by asking students, teachers, andadministrators questions about them. In addition,we compared reported outcomes in participatingschools to those in non-participating schools. Aswell, we directly measured student outcomes in asample of World Links and non-World Linksschools in one country. We used the triangulationof these responses as in indication of their veracity.
ARTICLE IN PRESS10 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Table 1Classroom practices as reported by students and teachers
How often did you use computers to do each of the following: Percentage of each group who responded “1–3 times amonth” or “once a week or more” a
World Links Non-World Links World Links teachersstudents 1999 computer-using 2000b
students 1999
Use applications software 52c 42 71Use CD-ROMs 34c 21 50Use e-mail 47c 25 62Use a search engine to find information on the Web 40c 19 56Use bulletin boards or listservs 26c 15 35Use the Internet 52c 16 59Use drill and practice materials 51 46 56Use spread sheets to analyze data 34c 31 28Produce a Web page 23c 9 16Collaborate on a project with other students in the same class 47c 30 53Collaborate on a project with students from another school in 25c 13 27your own countryCollaborate on a project with students from another country 23c 4 23Exchange information with another students from another country 28c 8 29Gather and analyze resource materials on a problem or topic 41c 25 44Gather data for a research project 38c 22 40Gather evidence to argue a position about an issue 34c 25 39Write project reports 35c 24 35Use graphics in a report 34 28 35Collect information about another country or culture 31c 18 26Draw conclusions or make predictions using data you gathered or 35c 28 34obtained from resource materialsCommunicate with your parents or other members of the 41c 31 28community about what you do in school
a The response options provided to respondents were: not at all, 1–5 times during the year, more than five times during the yearbut less than once a month, average of 1–3 times a month, and average of once a week or more.
b Teachers were asked about the extent of student practices and were provided with the same response option as students.c World Links and non-World Links students are statistically different (pair-wise t-test, p � 0.05) for full range of responses.
Nonetheless, these findings must be consideredpreliminary until direct measures of learning canbe systematically and widely collected.
4.3.1. Student outcomesIn the 1998–1999 survey, a majority of students
in the World Links program said that it had a sig-nificant effect on a variety of knowledge, skills,and attitudes (see Table 2). Perhaps, it is not sur-prising that 66% of students rated the program veryhighly for its impact on their attitudes toward tech-nology and over 60% said their technology skillsimproved. However, the most highly rated out-
come of the program was on students’ opinionabout their ability to get better jobs upon gradu-ation; 77% of the World Links students rated thisoutcome as very high. In addition, 71% rated theprogram very high on its effect on their communi-cation skills and 61% on its effect on their abilityto reason with information. A large majority saidthat the program improved their attitudes towardschool (66%) and their school attendance (64%).And 60% rated the program very highly in itseffect on their knowledge about other cultures asa result of their participation. In the 1998–1999survey, an overwhelming 92% of the students in
ARTICLE IN PRESS11R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Table 2Student outcomes attributed to World Links
How much impact the participation in the World Links Percentage of each group that responded “very much” a
program has on students:
World Links World Links World Links World Linksstudents 1999 teachers 1999 administrators 1999 teachers 2000
Improved technology skills 63 88 65 69Improved attitudes towards technology 70 86 71 78Improved ability to reason with information 61 71 65 63Improved communication skills 71 74 73 70Increased knowledge or awareness of other cultures 60 67 47 61Improved attitudes toward school 66 78 76 67Improved school attendance 64 31 50 44Improved ability to get bettor jobs upon graduation 77 48 25 59
a The response options provided to the respondent were: not at all, somewhat, and very much.
World Links schools expressed satisfaction withtheir participation in the program.
Teachers and administrators generally agreedwith students’ opinions of the program’s impact(see also Table 2). In the 1998–1999 survey, 88%of the teachers and 65% of the administrators ratedthe program very highly for its positive impact onstudent technology skills and 86% of the teachersand 71% of the administrators rated it highly onits impact on students’ attitudes toward technology.Similarly, these respondents rated the programvery high on its effect on students’ ability to reasonwith information (71% of the teachers and 64% ofthe administrators), on their communication skills(74% of the teachers and 73% of theadministrators), and on their attitudes towardschool (78% of the teachers and 76% of theadministrators). Two thirds of the teachers ratedthe program very highly on its impact on students’knowledge of other cultures and on their schoolattendance. Teachers were less enthusiastic thanstudents on their assessment of the program’simpact on student school attendance and on stu-dents’ ability to get better jobs. Administratorswere less optimistic about the program’s impact onstudents’ job prospects, on their school attendance,and on their knowledge of other cultures.
These positive assessments of the program’simpact were shared by the larger group of teachersfrom 12 countries that responded to the survey in
1999–2000 (Table 2). A large majority of the tea-chers rated the program very highly on its impacton all the outcomes, especially its impact on stu-dents’ attitudes toward technology, their communi-cation skills, their technology skills, and their atti-tudes toward school.
4.3.2. Student outcomes comparedWhile the questions reported above asked for
opinions about outcomes specifically linked to theWorld Links, the questions described in this sec-tion asked respondents to report on learning out-comes related to their use of computers but withoutreference to the program. Consequently, the ques-tions could be asked of students and teachers whodid not participate in the program as well as thosewho did.
Table 3 compares the responses of three othergroups of students from the 1998–1999 survey:students who participated in the World Links pro-gram, computer-using students who were in thesame schools but did not participate in the WorldLinks program, and computer-using students innon-World Links schools. On a wide range ofknowledge, skills, and abilities, World Links stu-dents rated their learning high as a result of usingcomputers and higher than did students in the sameschool who did not participate in the program. Amajority of World Links students rated their learn-ing highly and had higher end-of-year ratings on
ARTICLE IN PRESS12 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Table 3Student learning outcomes as reported by various groups of students
Use of computers has improved your knowledge, skill, and Percentage of each group that responded “very much” a
abilities in the following areas:
Participating World Non-participating Non-World LinksLinks students 1999 World Links computer-using
students 1999 students 1999
Knowledge about academic subjects 40b,c 33 36Planning, regulating and monitoring of your own learning 54b,c 43 35Finding, comparing, and evaluating information 53b,c 52 42Analyzing and interpreting information 57b,c 49 42Writing reports 67c 54 57Communicating with others 60c 60 55Using images, drawings, or graphics to represent ideas 52 54 51General knowledge about current events 53c 44 37Knowledge about other cultures, countries, or languages 53b,c 44 34Collaborating with others 63c 58 51Using computers 77b 67 73Using computers software 58c 52 49Using Internet 66b,c 58 45
a The response options provided to respondents were: not at all, somewhat, very much.b World Links and non-participating World Links students are statistically different (pair-wise t-test, p � 0.05) for full range
of responses.c World Links and non-World Links computer-using students are statistically different (pair-wise t-test, p � 0.05) for full range
of responses.
their ability to plan their own learning; their abili-ties to analyze and to interpret information; theirabilities to communicate, to write reports, and touse graphics to represent ideas; their ability to col-laborate with others; their knowledge of other cul-tures; and, of course, their skills in using com-puters, software, and the Internet. Only 40% of theWorld Links students gave a high rating to theimpact of computers on their knowledge of aca-demic subject matter, but this was still higher thanstudents in the two other groups. And only for theuse of graphics did World Links students report nomore learning than one or both of the other groups.
Similarly, teachers who participated in theWorld Links program were significantly morelikely than computer-using teachers in otherschools to report higher ratings for their students’learning and attitudes in a number of areas. Table4 shows that a majority of the World Links teach-ers from the five countries participating in the1998–1999 survey rated the use of computers ashaving a high impact on student learning in all but
two of the indicators—student planning of theirown learning and their analysis of information.They rated the impact of computers on studentlearning higher than teachers in one or both of theother groups for communication skills, knowledgeof other cultures, collaboration with others, andInternet skills. Similarly, World Links teachersfrom the 12 countries who responded to the 1999–2000 survey were more likely than non-WorldLinks teachers to highly rate the impact of com-puters on student communication skills, knowledgeof current events, knowledge of other cultures, col-laboration skills, and Internet skills.
4.3.3. Student outcomes assessedWhile the survey responses of students, teachers,
and administrators all attested to the impact ofcomputers and the World Links program on stud-ent learning, this is not as compelling as a directmeasure of student learning. In 2000, the perform-ance of 121 students in six World Links schoolsand 79 students in four comparable non-World
ARTICLE IN PRESS13R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Tab
le4
Stud
ent
lear
ning
outc
omes
asre
port
edby
vari
ous
grou
psof
teac
hers
Use
ofco
mpu
ters
has
help
edyo
urst
uden
tsim
prov
eth
eir
Perc
enta
geof
each
grou
pth
atre
spon
ded
“ver
ym
uch”
a
know
ledg
e,sk
ills,
and
abili
ties
inth
efo
llow
ing
area
s:
Part
icip
atin
gW
orld
Non
-par
ticip
atin
gN
on-W
orld
Lin
ksW
orld
Lin
ksN
on-W
orld
Lin
kste
ache
rs19
99W
orld
Lin
kste
ache
rsco
mpu
ter-
usin
gte
ache
rs20
00L
inks
teac
hers
1999
teac
hers
1999
2000
Kno
wle
dge
abou
tac
adem
icsu
bjec
tsb
5648
3739
34Pl
anni
ng,
regu
latin
g,an
dm
onito
ring
thei
row
nle
arni
ng47
5231
4330
Find
ing,
com
pari
ng,
and
eval
uatin
gin
form
atio
n53
5442
5054
Ana
lyzi
ngan
din
terp
retin
gin
form
atio
n47
5036
4855
Wri
ting
repo
rts
5654
7355
59C
omm
unic
atin
gw
ithot
hers
79c
5542
67d
46U
sing
imag
es,
draw
ings
,or
grap
hics
tore
pres
ent
idea
s53
6174
5656
Gen
eral
know
ledg
eof
curr
ent
even
ts65
5948
62d
36K
now
ledg
eab
out
othe
rcu
lture
s,co
untr
ies,
orla
ngua
ges
75c,
e46
2253
d38
Col
labo
ratin
gw
ithot
hers
80c,
e51
4865
d35
Usi
ngco
mpu
ter
soft
war
e81
7177
6458
Usi
ngth
eIn
tern
et75
c,e
4435
67d
35
aT
here
spon
seop
tions
prov
ided
tore
spon
dent
sw
ere:
not
atal
l,so
mew
hat,
very
muc
h.b
The
wor
ding
ofth
isst
atem
ent
was
slig
htly
diff
eren
tfo
rth
e19
99–2
000
surv
ey.
Itst
ated
“per
form
ance
onte
sts
ofac
adem
icsu
bjec
ts”.
cW
orld
Lin
ksan
dno
n-W
orld
Lin
ksco
mpu
ter-
usin
gte
ache
rsar
est
atis
tical
lydi
ffer
ent
(pai
r-w
ise
t-te
st,
p�
0.5)
for
full
rang
eof
resp
onse
s.d
Wor
ldL
inks
and
non-
Wor
ldL
inks
teac
hers
are
stat
istic
ally
diff
eren
t(p
air-
wis
et-
test
,p
�0.
5)fo
rfu
llra
nge
ofre
spon
ses.
eW
orld
Lin
ksan
dno
n-pa
rtic
ipat
ing
Wor
ldL
inks
teac
hers
are
stat
istic
ally
diff
eren
t(p
air-
wis
et-
test
,p
�0.
05)
for
full
rang
eof
resp
onse
s.
ARTICLE IN PRESS14 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Links schools in Uganda were measured, scored,analyzed, and compared (Quellmalz and Zalles,2000). Table 5 shows the number of performancesof each component that were exhibited in theseopen-ended tasks and the percent that was rated as“adequate” or “above adequate” by raters.
The World Links schools out-performed thenon-World Links schools on all components ofreasoning with information and communicationthat were measured by the assessment. The differ-ences were greatest in the components of reasoningwith information that involved finding and categor-izing relevant information (85% for World Links,59% for non-World Links), making comparisons(63% for World Links, 46% for non-World Links),and using information to make supported predic-tions (62% for World Links, 50% for non-WorldLinks). Overall, the percentage of “adequate” or“above adequate” responses by students in WorldLinks schools on reasoning with information was14% higher than the ratings of student responsesfrom non-World Links schools and 5% higher oncommunication.
The lowest scores for both World Links andnon-World Links schools were for ratings of howwell the students’ news articles presented and sup-ported their predictions. Consequently, the differ-ences between the two groups were least pro-nounced for this outcome area. While slightlyhigher than the non-World Links group, only 37%of the World Links students’ news articles wererated as presenting an “adequate” or “aboveadequate” predictions.
World Links school students out-performed non-World Links students on ratings of the logicaldevelopment of their news articles. Seventy-fourpercent of the news articles written by World Linksstudents were rated as “adequately organized” or“well organized” , in comparison to 65% of thenews articles written by students in the non-WorldLinks schools.
The students in World Links schools did quitewell in their display of technology skills, as scoredby raters. Seventy-four percent of the World Linksstudents received ratings of “adequate” or “above”on tasks requiring searching for information on theWeb. Sixty-three percent received “adequate” or“above adequate” ratings on either formulating or
identifying an appropriate phrase for a Web searchquery on a specific topic.
4.3.4. Teacher outcomesStudents were not the only ones to benefit from
participation in the World Links program; teachersbenefited in a number of ways. As mentioned earl-ier, a major component of the World Links pro-gram is its emphasis on the professional develop-ment of teachers and on improved pedagogy.
As a result of their participation in the WorldLinks program, a large majority of the World Linksteachers in the 1998–1999 survey felt that they gre-atly improved not only their ability to use computerhardware (70%), applications software (67%), andthe Internet (73%) but their ability to use studentgroups in their teaching (63%) and to design andlead collaborative student projects (68%), as shownin Table 6. A majority of teachers (52%) indicatedthat the program increased their ability to collabor-ate with other teachers on the development ofmaterials.
A very large percentage of administrators feltthat the World Links program greatly affected tea-chers’ computer skills (82%), their ability to usethe Internet (80%), their attitudes about technology(85%), and their attitudes about their own teaching(90%). A majority felt that the program improvedteachers’ software skills (67%) and their ability touse student groups in their teaching (60%). Bothteachers and administrators in the 1998–1999 sur-vey agreed that program had limited impact on tea-chers’ ability to integrate computers into curricu-lum integration and assessment.
Teachers from the 12 countries participating inthe 1999–2000 survey agreed with their colleaguesin the earlier survey on most questions (also inTable 6). These teachers rated the program’simpact as higher on integration of computers intothe curriculum but they also felt that impact onassessment design was limited.
4.3.5. SummaryIn summary, data from administrators, teachers,
and students indicate that students who participatedin the World Links program learned a range of newskills, more so than those in non-participatingschools. These findings were confirmed by a larger
ARTICLE IN PRESS15R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Tab
le5
Sum
mar
yby
outc
ome
area
and
skill
com
pone
nts
for
the
stud
ent
asse
ssm
ent
inU
gand
a
Out
com
ear
eaSk
illco
mpo
nent
Wor
LD
scho
ols
Non
-Wor
LD
scho
ols
Num
ber
ofre
spon
ses
Perc
enta
geof
resp
onse
sN
umbe
rof
resp
onse
sPe
rcen
tage
ofre
spon
ses
“ade
quat
e”or
“abo
ve”
“ade
quat
e”or
“abo
ve”
“ade
quat
e”or
“abo
ve”
“ade
quat
e”or
“abo
ve”
(tot
alre
spon
ses)
(tot
alre
spon
ses)
Rea
soni
ngw
ithFi
ndin
gan
dca
tego
rizi
ngre
leva
nt14
2(1
67)
8567
(114
)59
info
rmat
ion
info
rmat
ion
Com
pari
sons
178
(282
)63
89(1
93)
46Pr
edic
tions
135
(217
)62
76(1
53)
50Fo
rmul
atin
gor
iden
tifyi
nga
rese
arch
83(1
07)
7851
(71)
72qu
estio
nSu
ppor
ting
apr
edic
tion
43(1
15)
3728
(77)
36T
otal
s58
1(8
88)
6531
1(6
08)
51C
omm
unic
atio
nO
rgan
izat
ion
85(1
15)
7450
(77)
65A
rgum
ent
43(1
15)
3728
(77)
36T
otal
s12
8(2
30)
5678
(154
)51
Tec
hnol
ogy
use
Sear
chin
gfo
rin
form
atio
non
the
Web
147
(198
)74
–a–a
Form
ulat
ing
orid
entif
ying
ase
arch
quer
y64
(102
)63
–a–a
Tot
als
211
(300
)70
–a–a
aT
echn
olog
yus
eou
tcom
eda
taw
ere
not
gath
ered
from
non-
Wor
ldsc
hool
s.
ARTICLE IN PRESS16 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Table 6Impact of the World Links program on teachers as reported by various groups
Impact of the program on teachers’ skills, abilities, knowledge, and Percentage of each group that responded “very much” a
attitudes:
World Links World Links World Linksteachers 1999 administrators 1999 teachers 2000
How to use computer hardware 70 82 76How to use software applications 67 67 71How to use the Internet 73 80 72How to develop Web pages 47 53 50How to use student groups in teaching 63 60 63How to design and lead collaborative student projects 68 58 63How to design and use student assessment materials 46 35 45How to collaborate with other teachers in developing materials 52 44 46How to integrate computers into the curriculum 39 40 53Attitudes about technology 72 85 84Attitudes about teaching 75 90 82
a The response options provided to respondents were: not at all, somewhat, very much.
sample of teachers in many more countries thatparticipated in the program during the second yearof the evaluation. Even more important, WorldLinks students performed better than non-WorldLinks students on direct assessments of ICT skillsand their use to reason with information and com-municate ideas. Teachers too acquired significantICT and pedagogical skills as a result of their par-ticipation in the program. All these results, takentogether, affirm the positive—although still pre-liminary—effects of the World Links program.
4.4. Barriers and differences between countries
While the primary focus of the evaluation wason program practices and outcomes, we felt thatan assessment of the problems and barriers thatconfronted teachers would provide information forWorld Links program improvement and for the for-mulation of national policies that would increasethe effectiveness and impact of such ICT-basedprograms and investments.
Teachers were asked about the reasons that theymay have used computers in their teaching lessthan they had planned. Of the 83 WorLD teacherswho responded to the survey in 1998–1999, 16 (ornearly 20%) indicated that they had not yet
implemented computer-related activities with theirstudents, and 53 (or 64%) mentioned that they hadencountered one or more problems that reducedtheir ability to use computers in their classes.
Among the 16 teachers who said they were notable to implement computer-related activities, themost often cited major barrier was not the lack ofequipment but the lack of time; 62% of these tea-chers indicated that, given curriculum and testingrequirements, difficulty in finding time for com-puter activities was a major barrier (Table 7). Simi-larly, many of these teachers said that finding timeto complete computer activities within the school’sdaily schedule (59%) and finding time in their ownschedule to prepare for implementing technologyin their classes (56%) were major barriers. Relatedto these issues, 42% said that a barrier was the lackof a national policy on the use of computers inschools. Overall, relatively few of these teachersidentified infrastructure problems, such as the lackof computers in working condition, unreliable elec-tricity, or lack of access to the Internet, althoughthese varied by country, as described below.
The 53 teachers who said they were able toimplement computer-related activities butimplemented fewer than planned responded simi-larly. Of these teachers, 51% indicated as a major
ARTICLE IN PRESS17R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Tab
le7
Bar
rier
sto
impl
emen
ting
com
pute
r-re
late
dcl
assr
oom
prac
tices
To
wha
tex
tent
wer
eth
efo
llow
ing
barr
iers
:Pe
rcen
tage
ofea
chgr
oup
who
resp
onde
d“m
ajor
barr
ier”
a
Wor
ldL
inks
teac
hers
who
Wor
ldL
inks
teac
hers
Wor
ldL
inks
did
not
yet
impl
emen
tw
hodi
dim
plem
ent
teac
hers
2000
com
pute
rs19
99co
mpu
ters
1999
Lac
kof
com
pute
rha
rdw
are/
soft
war
e32
2960
Lac
kof
com
pute
rsin
good
wor
king
cond
ition
3632
56L
ack
ofre
liabl
eel
ectr
icity
2418
18L
ack
ofad
equa
tete
leph
one
line
orse
rvic
esfo
rac
cess
toth
eIn
tern
et38
5352
Lac
kof
inte
rnet
acce
ssfo
rot
her
reas
ons
3124
52L
ack
ofth
ein
suffi
cien
ttr
aini
ngfo
rus
ing
com
pute
rsan
dso
ftw
are
3224
23L
ack
ofor
insu
ffici
ent
tech
nica
lsu
ppor
tfo
rus
ing
com
pute
rsan
dso
ftw
are
3124
23L
ack
ofor
insu
ffici
ent
trai
ning
orsu
ppor
tfo
rin
tegr
atin
gco
mpu
ters
into
the
curr
icul
um54
3938
Lac
kof
orin
suffi
cien
ttr
aini
ngor
supp
ort
for
usin
gco
mpu
ters
inco
llabo
rativ
est
uden
t38
2235
proj
ects
Dif
ficul
tlyfin
ding
time
topr
epar
efo
rim
plem
entin
gte
chno
logy
inyo
urcl
assr
oom
orat
your
5643
44sc
hool
Dif
ficul
tyco
mpl
etin
gco
mpu
ter
activ
ities
with
inth
esc
hool
’sda
ilysc
hedu
le59
4251
Dif
ficul
tyfin
ding
time
for
com
pute
rac
tiviti
es,
give
not
her
curr
icul
uman
dte
stin
g61
5149
requ
irem
ents
Com
pute
rac
tiviti
esdo
not
mat
chw
ell
with
your
inst
ruct
iona
lgo
als
and
met
hods
1510
10C
once
rnab
out
whe
ther
tech
nolo
gyw
ould
beva
luab
leto
your
stud
ents
210
7C
once
rnth
atus
ing
tech
nolo
gyin
your
scho
olw
illaf
fect
the
stab
ility
ofyo
urcu
rren
tjo
b4
04
Lac
kof
acce
ssto
exis
ting
com
pute
rs12
1220
Lac
kof
supp
ort
from
othe
rco
lleag
ues
2220
15L
ack
ofad
min
istr
ativ
esu
ppor
tin
scho
ol37
1715
Lac
kof
ana
tiona
lpo
licy
onth
eus
eof
com
pute
rsin
scho
ols
4229
36
aT
here
spon
seop
tions
prov
ided
tore
spon
dent
sw
ere:
not
aba
rrie
r,m
inor
barr
ier,
and
maj
orba
rrie
r.
ARTICLE IN PRESS18 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
barrier the difficulty in finding time for computeractivities, given other curriculum and testingrequirements; 43% cited the difficulty in findingtime to prepare for implementing technology intheir classes; and 41% mentioned the difficulty infinding time within the school’s daily schedule. Aswith WorLD teachers who were not able to usecomputers at all, relatively few computer-usingteachers identified infrastructure problems, such asthe lack of computers in working condition, unre-liable electricity, or lack of access to the Internet,per se. However, 53% did say that a major barrierto using the computer as much as they had plannedwas the lack of adequate telephone lines for usewith the Internet. Indeed, this was the most fre-quently cited major barrier for this group of teach-ers.
There were differences among the five countriesin the kinds of barriers that teachers encounteredin the 1998–1999 study. Teachers in Senegal(53%) and Chile (90%) most often identified thelack of time to integrate ICT into the curriculumas a barrier. Teachers in Senegal (53%) also mostfrequently identified lack of computers in goodworking order. On the other hand, teachers inUganda (78%) and Paraguay (60%) most oftenidentified inadequate telephone or Internet serviceas a barrier. Finally, 67% of the teachers in Peruidentified the lack of training and support for inte-grating computers into the curriculum, the barriermost often mentioned by teachers in this country.
In contrast with the 1998–1999 teachers, WorldLinks teachers from 12 countries who respondedto 1999–2000 survey were most likely to identifythe lack of computer hardware (60%), software(56%), and reliable Internet connections (52%) asmajor barriers to their use of ICT in their classes.These findings suggest that as the World Linksscaled up the program, the organization wasinvolving countries and schools within countriesthat were less technologically prepared and in moreneed of assistance. But even in these schools, theteachers agreed with their colleagues in the 1998–1999 study that a major barrier was finding time inthe school’ daily schedule to implement computer-related activities in their classes (51%) and thatthey had difficulty finding time given curriculumand other testing requirements (49%).
There were also differences among countries inthe 1999–2000 study. Teachers in Mauritania(100%), Zimbabwe (67%), Ghana (74%), Chile(59%), and Peru (53%) most often mentioned thelack of hardware and software among the barrierslisted in Table 7. All the teachers in Mauritaniaalso mentioned the lack of computers in workingorder as a barrier. Inadequate phone lines or otherInternet connection problems were most oftenmentioned by teachers in Brazil (88%), Colombia(65%), Mozambique (62%), and Paraguay (62%).Teachers in Senegal mentioned the lack of trainingand support for integration into the curriculum asa barrier; 78% of them mentioned this problem.And teachers in South Africa (62%) and Uganda(55%) were most likely to mention lack of time tointegrate computers into the curriculum as a bar-rier.
The barriers to teachers’ use of ICT were some-times significant. However, it is also important toidentify those factors that teachers did not consideras barriers. As important as those things that teach-ers considered to be barriers are those they did not.Regardless of country or year of the survey, fewof the World Links teachers felt that computers didnot match their instructional goals or methods (ascontrasted to curricular goals). Few were con-cerned that the use of computers would have novalue for their students. And very few were con-cerned that the introduction of computers inschools would affect their jobs status or stability.
5. Summary and conclusions
The findings from this study indicate that whentechnological infrastructure is developed in con-junction with appropriate teacher training, signifi-cant educational change can be achieved indeveloping countries. Despite the identified bar-riers and problems encountered by World Linksteachers, we can infer from our data that studentsin World Links schools were more likely thancomputer-using students in non-World Linksschools to use a variety of technologies, such asemail, search engines, and the Internet. They werealso more likely to engage in classroom practicesthat are often cited as important for preparing stu-
ARTICLE IN PRESS19R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
dents for the global knowledge economy, activitiessuch as gathering data for a research project, col-lecting information about another country or cul-ture, and collaborating on a project with studentsfrom another country.
By all indications, students not only engaged inthese practices but acquired the skills that they willneed to enter the global workforce. Students, teach-ers, and administrators agreed that the World Linksprogram contributed to student outcomes such asimproved skills in reasoning with information,communication skills, knowledge of other cultures,better attitudes toward school and technology, and,of course, improved technology skills. Further-more, teachers and students (in the case of the1998–1999 study) in the World Links programwere more likely than those in non-World Linksclassrooms to say that students improved theircommunication skills, knowledge of currentevents, knowledge of other cultures, collaborationskills, and Internet skills. Finally, and mostimportantly, World Links students scored higherthan non-World Links students in direct assess-ments of reasoning with information and com-munication skills.
Our findings support the argument thatdeveloping countries can benefit from investmentsin equipment, software, and network infrastructure.The findings also demonstrate the benefit of train-ing teachers not only in the operation of hardwareand software but in the integration of technologyinto classroom instruction. Finally, the findingsdemonstrate that they payoff of these investmentsis increased skills of both students and teachers.Taken together, these findings support the value ofthe World Links program and suggest that WorldLinks and other programs which aim to build thetechnological infrastructure and teacher skills indeveloping countries can contribute to theimprovement of education and reduction of thedigital and perhaps economic divide betweendeveloping and developed countries.
But the barriers cited by World Links teachers,as well as findings from other studies (Means andOlson, 1995; Sandholtz et al., 1997; Means et al.,2001; Schofield and Davidson, 2002; Kozma,2003) suggest that there are other actions and poli-cies that if employed could increase the impact of
World Links and other such programs. On the tech-nology side, better ways are needed to connectschools in developing countries to the Internet,especially low-cost, high bandwidth solutions thatmight benefit schools in remote areas that do notcurrently have reliable phone lines.
But the most-often cited barriers were not tech-nological. World Links teachers consistently citedthe lack of time for computer activities, given cur-riculum and examination requirements, the lack oftime in the school day, and the lack of time forpreparation. These findings suggest that teachers indeveloping countries are encountering the samebarriers as teachers in developed countries are(Means and Olson, 1995). In turn, these time-related barriers suggest an underlying lack of pri-ority for the use of ICT in current national andlocal policies. The lack of national ICT policy wasalso a barrier often cited by teachers. This barrieris most vividly illustrated by an incident during afield site visit by one of the authors (Kozma) to aWorld Links school in Uganda. When asked whya computer lab was empty during the school day,even though it was packed after classes, the teachersaid that there was little fit between the use of com-puters and the national curriculum and examinationsystem in Uganda. The use of computers was notin the curriculum nor would it be tested on thenational exam. Consequently, it was difficult forthe teachers of this school to justify using com-puters during regular school time.
Developing countries can draw on the experi-ences of other countries where ICT has come tohave a more important role in education. In a grow-ing number of developed countries, national poli-cies identify a high priority for the use of ICT ineducation; see for example Singapore (Ministry ofEducation and Singapore, 2002), Finland (Ministryof Education and Finland, 1999), and US (USDepartment of Education, 2000). These policiesand priorities are often expressed as an educationtechnology master plan. These plans provide avision for the use of technology in education andinstitute programs to support the realization of thisvision (Jones, 2003). To maximize the impact ofinvestments in educational ICT, the ICT policyneeds to be coordinated with the policies of otherministries (such as telecommunications, human
ARTICLE IN PRESS20 R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
resources, and science and technology) and withother policies within the ministries of education onmatters related to teacher training, curriculum,and assessment.
Teacher training—both in-service and pre-ser-vice—needs to incorporate the use of ICT in class-room teaching. World Links teachers identified thelack of sufficient training and support for the inte-gration of computers into the curriculum as anotherbarrier. If computers are going to have a significantimpact on the learning of students, it will bebecause they are integrated into the curriculum, notjust as a separate subject such as computer literacyor keyboarding, but as tools to support the learningof science, mathematics, social studies, language,and creative arts. In the work world, technology isdramatically changing the way these school sub-
Box 1. Selected Examples of WorLD School Projects on Environmental Issues
Ciencia a Conciencia (Conscientious Science)Participating schools in: Paraguay, MexicoDescription: Students learn collaboratively about various environmental issues including: defor-estation; pollution; ozone layers; toxic wastes. They share results of findings and explore sol-utions to these problems. Findings from the collaborative learning activities are posted on theWeb site.Web site:
http://www.enlaces.edu.py/cndelm/131198/circulos.htm
Wetlands Project
Participating schools in: Uganda, Australia, USADescription: The project brings together students around the theme of learning about and pro-tecting local wetlands and waterways. Reports are prepared by students on: wetland conditions;legislation to protect wetlands; public awareness raising on the impact of human activities onthe ecology; water levels and water quality of wetlands/waterways; and steps to take to ensurefuture protection of wetlands. Findings are shared among the participating students on the web-sites.Web sites:
http://www.viser.net/gs21/ugdepletion.htm (school)
http://www.camperdowncoll.vic.edu.au/global/wateryworlds/wetlands.htm (project)
Flora y Fauna
Participating schools in: Peru, Chile, Brazil, Argentina, U.S.A.Description: An interdisciplinary project where students learn about “Flora and Fauna” fromvarious approaches including: mathematics (statistical/geometric/measurement aspects); litera-ture (metaphoric use of plants & animals in literature); language (translation).Web site:
http://www.geocities.com/Athens/Atlantis/6126/proyecto.htm
jects are applied to solve everyday problems. Theintegration of ICT into the learning of these schoolsubjects can make a significant contribution to thepreparation of students in developing counties forparticipation in the global knowledge economy andinformation society (21st Century Partnership,2003; ISTE, 2000).
Classroom integration will depend on the inte-gration of ICT into the formal national curriculumand testing program. Policies in an increasing num-ber of developed countries (Jones, 2003) areincorporating the use of ICT throughout the cur-riculum as part of an explicit effort to reform edu-cation and improve student learning. Often thesechanges are coordinated with student-centeredpedagogies. Some countries, specifically Singaporeand Norway, are beginning to review their
ARTICLE IN PRESS21R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
approach to national assessments and examinationsso as to incorporate the use of and learning fromICT.
If developing countries set their national pri-orities to emphasize the use of ICT in educationand these are coordinated with other policiesrelated to teacher training, curriculum, and assess-ment, the foundation will exist for teachers to inte-grate computer activities into their daily practice.This in turn will prepare students for the globalknowledge economy and contribute significantly toclosing the digital divide between developed anddeveloping countries. Without such national poli-cies, priorities, and resources, in all likelihood thedigital divide will continue to grow.
References
21st Century Partnership 2003. Learning for the 21st Century.21st Century Partnership, Washington, DC.
Bridges.org. 2001. Spanning the Digital Divide: Understandingand Tackling the Issues. Bridges.org, Washington, DC.
Carlson, S., 2002. The missing link in educational technology:trained teachers. TechKnowLogia 4 (6), 7–11.
Educational Testing Service (ETS) 2002. Digital Transform-ation: A Framework for ICT Literacy. ETS, Princeton, NJ.
International Society for Technology in Education (ISTE) 2000.National Educational Technology Standards for Students:Connecting Curriculum and Technology. InternationalSociety for Technology in Education, Eugene, OR.
Jones, R., 2003. National and local policies. In: Kozma, R.(Ed.), Technology, Innovation, and Educational Change: AGlobal Perspective. International Society for Technology inEducation, Eugene, OR, pp. 163–194.
Kozma, R., 2003. Technology, Innovation, and EducationalChange: A Global Perspective. International Society forTechnology in Education, Eugene, OR.
Kozma, R., McGhee, R., 1999. World Links for Development:Accomplishments and challenges. Monitoring and evalu-ation annual report: 1998–1999. SRI International, MenloPark, CA.
Means, B., Olson, K., 1995. Technology’s Role in EducationReform: Findings from a National Study of InnovatingSchools. Washington, DC, US Department of Education,Office of Educational Research and Improvement.
Means, B., Penuel, W., Padilla, C., 2001. The ConnectedSchool: Technology and Learning in High School. Jossey-Bass, San Francisco.
McGhee, R., Kozma, R., 2000. World links for Development:Accomplishments and challenges. Monitoring and evalu-ation annual report: 1999–2000. SRI International, MenloPark, CA.
Ministry of Education, Finland 1999. Education, Training andResearch in the Information Society: A National Strategyfor 2000–2004. Ministry of Education, Olso, NO.
Ministry of Education, Singapore, 2002. Masterplan 2(http://www.moe.edu.sg/edumall/mp2/mp2—overview.htm).
National Center for Educational Statistics (NCES) 2001. TheNation’s Report Card: Science 2000. National Center forEducational Statistics, Washington, DC.
Organization for Economic Co-operation and Development(OECD) 1996. The Knowledge-Based Economy. OECD,Paris.
Organization for Economic Co-operation and Development(OECD) 2000. Literacy in the Information Age. OECD, Paris.
Organization for Economic Co-operation and Development(OECD) 2001a. Education Policy Analysis. OECD, Paris.
Organization for Economic Co-operation and Development(OECD) 2001b. Understanding the Digital Divide. OECD,Paris.
Pelgrum, W., Plomp, T., 2002. Indicators of ICT in mathemat-ics: status and covariation with achievement measures. In:Beaton, A.E., Robitaille, D.F. (Eds.), Secondary Analysis ofthe TIMSS Data. Kluwer Academic Press, Dordrecht.
Quellmalz, E., Kozma, R., 2003. Designing assessments oflearning with technology. Assessment in Education 10 (3),389–407.
Quellmalz, E.S., Zalles, D., 1999. World Links Student Assess-ment Report, 1998–1999. SRI International, Menlo Park, CA.
Quellmalz, E., Zalles, D., 2000. World Links for Development:Student Assessment Uganda Field Test, 2000. SRI Inter-national, Menlo Park, CA.
Sandholtz, H., Ringstaff, C., Dwyer, D., 1997. Teaching withTechnology: Creating Student-Centered Classrooms. Teach-ers College Press, New York.
Shavelson, R., Towne, L., 2001. Scientific Inquiry in Education.National Academy Press, Washington, DC.
Schofield, J., Davidson, A., 2002. Bringing the Internet toSchool: Lessons from an Urban District. Wiley, New York.
Tashakkori, A., Teddlie, C., 1998. Mixed Methodology: Com-bining Qualitative and Quantitative Approaches. Sage,Thousand Oaks, CA.
United Nations Development Programme 2001. Human Devel-opment Report 2001. UNDP, Paris.
United Nations Development Programme 1999. Human Devel-opment Report 1999. UNDP, Paris.
US Department of Education 2000. E-learning: Putting aWorld-Class Education at the Fingertips of all Children. USGovernment Printing Office, Washington, DC.
Wenglinski, H., 1998. Does it Compute? The RelationshipBetween Educational Technology and Student Achievementin Mathematics. ETS, Princeton, NJ.
World Bank 2002. The Education Sector Strategy Report.World Bank, Washington, DC.
World Bank 1999. The Education Sector Strategy Report.World Bank, Washington, DC.
World Bank 1997. The Economic Development InstituteAnnual Report. World Bank, Washington, DC.
top related