Report on Research Methods Module - University of StirlingReport on Research Methods Module Trial of Basic Training Module in Applied Social Research ... .uk/schema/ The publication

Educational Multimedia Taskforce

SCHEMA

Report on Research MethodsModule

Trial of Basic Training Module in Applied Social Research

Duncan Timms and Simon Booth

More information about the Schema project can be found underhttp://www.stir.ac.uk/schema/

The publication of this package has been supported bythe European Commission (TEN-Telecom, TSER andLeonardo da Vinci) under the Joint Call orchestrated bythe Educational Multimedia Taskforce

.Centre for Research and

Development in LearningTechnology

Project Number: 1024

Project Title: SCHEMA; Social Cohesion through Higher Education in Marginal Areas

Document Type: (PU/LI/X)* PU

Deliverable Number: D 3.1

Contractual Date of Delivery: 31.1.1999

Title of Document: WP1: Report on Research Methods ModuleInternal Document Number: 1/99 (Stirling)

Nature of the Deliverable: RE

Author(s): DUNCAN TIMMS; SIMON BOOTH;

Contact Details: [email protected]

*Type: PU–public, LI–limited, X–internal

**Nature: PR–Prototype, RE–Report, SP-Specification, TO–Tool, OT-Other

Copyright © ESCS - EEC - EAEC, Luxembourg and Brussels [1996]Copyright © 1998 by the European CommissionAll rights reserved. No part of this publication shall be reproduced, stored in a retrieval system, ortransmitted by any means, including electronic, mechanical, photocopying, recording or otherwise,without written permission from the copyright holder. No patent liability is assumed with repeat of theinformation contained herein.'

'All terms mentioned in this publication that are known to be trademarks or service marks have beenappropriately capitalised. However, the European Commission cannot attest to the accuracy of thisinformation, and the use of a term in this publication should not be regarded as affecting the validity ofany trademark or service mark.'

The publication of this package has been supportedby European Commission (TEN-Telecom, TSER andLeonardo da Vinci) under the Joint Call orchestrated bythe Educational Multimedia Taskforce

.

Centre for Research andDevelopment in Learning

Technology

Executive Summary

This report describes the initial running of the research methods module developedby Dr Nils-Magnus Björkman (University of Stockholm) and delivered as part of theM.Sc in Applied Social Research at the University of Stirling in the Autumn of 1998.The report gives the background to the development of the module and its learningmodel. Examples of the materials presented in the module are also given.This initial presentation of the module demonstrated that ProTo (the Web learningenvironment used to deliver the course) was easy-to-use and effective for thestudents as a learning delivery mechanism. We were unable to loan NCs to thestudents for this presentation of the module— the reasons for this are given in thetext. An affect of the failure to exploit NC technology was that the plannedcollaborative features of the course could not be undertaken.A number of appendices are included. The first three are course materials, includingexamples of the course assessments and the “take-home” examination. The finalappendix is the handout that accompanied the initial training session in ProTo.

Contents

1. REPORT ON TRIAL OF BASIC TRAINING MODULE IN APPLIED SOCIAL RESEARCH.... 1

INTRODUCTION ............................................................................................................................... 1WORK PACKAGE DESCRIPTION......................................................................................................... 1DELIVERABLES ............................................................................................................................... 2TIMING........................................................................................................................................... 2

2. MODIFICATIONS TO ORIGINAL PROPOSAL...................................................................... 3

STAFFING ....................................................................................................................................... 3DELIVERY MECHANISM ................................................................................................................... 3COURSE CONTENT........................................................................................................................... 3

3. LEARNING MODELS................................................................................................................. 5

4. THE QUANTITATIVE DATA ANALYSIS MODULE.............................................................. 7

OVERVIEW OF COURSE.................................................................................................................... 7LEARNING ENVIRONMENT............................................................................................................... 8RESOURCES .................................................................................................................................... 8

course notes............................................................................................................................... 8on-line texts and other materials ................................................................................................ 9on-line applications (Durkheim example and Luddersby Hall) .................................................. 10self-assessment tests (QUTAL material).................................................................................... 11

ASSESSMENT ................................................................................................................................ 11

5. THE LEARNERS....................................................................................................................... 13

6. STUDENT EVALUATION OF THE UNIT AND SUGGESTIONS FOR THE FUTURE....... 15

7. THE NEXT PHASE ................................................................................................................... 17

APPENDIX A: DURKHEIM EXERCISE..................................................................................... 21

APPENDIX B: MSC (ASR) DATA ANALYSIS: TEST 3............................................................. 25

APPENDIX C: TAKE-HOME TEST............................................................................................ 29

APPENDIX D: A SHORT INTRODUCTION TO PROTO.......................................................... 33

INTRODUCTION ............................................................................................................................. 33STARTING PROTO ......................................................................................................................... 33PROJECT OFFICE............................................................................................................................ 35

Message Board ........................................................................................................................ 35Course Info.............................................................................................................................. 35Student Card............................................................................................................................ 35Team folder ............................................................................................................................. 35Tutor folder ............................................................................................................................. 35

WORKSHOP .................................................................................................................................. 35COMMUNICATION ......................................................................................................................... 35LIBRARY ...................................................................................................................................... 36

USING PROTO FEATURES......................................................................................................... 37

PROJECT OFFICE............................................................................................................................ 37LIBRARY ...................................................................................................................................... 39COMMUNICATIONS ....................................................................................................................... 43LOGGING OFF ............................................................................................................................... 44USEFUL INFORMATION .................................................................................................................. 46

NETSCAPE BOOKMARKS ......................................................................................................... 47

SCHEMA 1

1. Report on Trial of Basic Training Module in Applied Social Research

Introduction

The first course materials produced in SCHEMA consist of a basic training module inapplied social research, stressing quantitative methods of data analysis. The modulewas described in the initial work plan as follows:Work package identifier: WP03Work package title: Basic training module in applied social researchPackage leader: University of StirlingDates: 98-04-01 - 2000-03-31Person-months: 18Tasks addressed: LdVObjectives: Development, verification and demonstration of a

validated training module on introductory appliedsocial science research methods for health, educationand welfare workers tested in a real-life setting.

Work package description

The work package is designed to produce sets of training materials on applied socialresearch, accessible over the WWW using a common protocol, with associatedsupport facilities. The material will go on-line for verification in September 1998 andwill take the form of a single semester unit occupying a 15-20 week slot.The material will be developed for delivery across the net with a particular emphasison the needs of health, education and welfare workers in the community. Themodule addresses a number of the objectives identified in the Leonardo da VinciCommon Framework, notably a) improving the quality and innovation capacity ofvocational training arrangements, c) promoting life-long learning; i) promotingequality of access, and r) fostering the development of methods of self-training.A common interface, according to standards developed in WP2, will be used for thepresentation of course material. Delivery will be via NCs with machines being madeavailable to learners throughout the regions being served (as specified in WP 2).Electronic tutorials will be set up using standard email and computer conferencingprotocols. Local tutorial support will be provided by graduate assistants. Courseadministration, also carried out over the network, will be the responsibility of theconsortium member serving the geographical area in which the learner resides.Course content will be developed collaboratively by lecturing staff in the partnerinstitutions, led by Stirling. and will be trialled in autumn 1998. Course content willbe based on units already given on-campus in Stirling as part of the programmes foreducation, social work and nursing students engaged in post-qualifying courses andwill be guided by consultation with relevant professional bodies (e.g. the BritishAssociation of Social Workers and the Nursing Board for Scotland). Localcustomization, including the provision of illustrative material, and the provision oftutorial support will be carried out by each partner. The basic language of instructionwill be English. The module will examine the relationship between researchquestions and research design in the social sciences and will pay particular attentionto methods of data gathering. Both the design of research projects and the

SCHEMA 2

application of specific research instruments will be discussed. There will be aconcentration on issues relating to research in relation to health, education andwelfare in the community. The module will include an introduction to the use ofstandard computer-based packages for data collection and analysis, includingquestionnaire and statistics programs (e.g. PinPoint and SPSS). Integrated assessmentand self-testing facilities will be included.Evaluation of the module will be in terms of both its educational content and interms of the functionality of client-server NC approach for the delivery and supportof educational multi-media in a real-life situation. The evaluation will involveexternal expert assessors as well as the feedback from the learners themselves.Following formative evaluation by learners and professional agencies during theautumn, a revised version will be mounted the following year. The evaluation willdraw upon questionnaires administered before and after learners have taken themodule, ethnographic studies of learner sessions and intensive interviews with asample of participants. The course will include a number of tasks to be completed bylearners and submitted electronically, including an overall examination. Learnerscompleting the unit satisfactorily will be able to include it in portfolios to besubmitted for post-qualifying recognition.Deliverables

Two sets of deliverables are planned:

• a transnational credit-bearing module on social research• Reports evaluating the success of the unit in meeting its objectives (Interim

Report May 1999; Final Report March 2000).The evaluation will consider both technical and pedagogical aspects. Followingcompletion of the project the training module will be incorporated in the on-goingODL programmes of the consortium partners (WP 9).

Timing

Specification and development: April 1998— August 1998Verification: September 1998— December 1998 (trial delivery)Evaluation and revision: January 1999— August 1999

(Interim Report May)Demonstration: September 1999— December 1999 (delivery)Evaluation: January 2000— March 2000 (Final Report)Product March 2000(applied social research module)

SCHEMA 3

2. Modifications to Original Proposal

Staffing

Staffing changes meant that the plan to use “in-house” expertise had to besupplemented by buying-in help. Dr Nils-Magnus Björkman, of the Department ofSociology in the University of Stockholm, was sub-contracted to produce an on-lineversion of a course in research methods for applied social science which had beengiven to students in sociology and social work in both Stockholm and Stirling. DrBjörkman has been the recipient of several teaching awards for his work in this area.

Delivery mechanism

As indicated in SCHEMA Deliverable 2.2, problems relating to the support ofJavaScript and to connection using modems have been experienced with theNetwork Computers provided by Acorn. It is hoped that the problems will beovercome in time for the demonstration phase of the research training module inautumn 1999, but the effect in 1998 was to severely curtail the extent of the trial. Inthe event the module was offered to students using computer laboratories on thecampus of the University of Stirling, connected via ethernet to the standardUniversity LAN. The problem relating to JavaScript, required for the version ofProTo used to mount the learning resources, was solved through the use of NetscapeNavigator V4. NC access was via Citrix ICA clients, supplied by Acorn. Although ithad been hoped that this solution would enable the modem version of the NC to beemployed this turned out to be impossible and plans to supplement the computerlaboratory provision with NCs in student homes had to be abandoned.

Course content

Discussions with Dr Björkman and with the Director of the MSc in Applied SocialResearch suggested that the original goal of providing a comprehensive introductionto research methods, including both qualitative and quantitative elements, wasoverly-ambitious. Instead it was decided to concentrate on quantitative methods,emphasising the role of statistics in survey research.

SCHEMA 4

SCHEMA 5

3. Learning Models

SCHEMA has overtly adopted a social constructivist approach, seeing learning asbeing the outcome of a process in which students engage with material through aseries of stages involving both the acquisition of new knowledge and its modificationand testing in social interaction. Different stages of learning require different inputsand instructional design needs to be adapted to meet the requirements of eachspecific task. The nature of the learning to be accomplished also impacts on thedesign criteria.A simple model which can be used to help the design process has been outlined byMayes (1995). Three major stages in the learning process are identified:conceptualization, construction and dialogue.The conceptualization phase is essentially concerned with knowledge transfer,presenting students with an established body of material. It has sometimes beenargued that this approach, with its assumption that a consensual body of knowledgeexists, is inconsistent with the perspective of the social sciences, in which disputationis a common feature. Lack of agreement may well be characteristic of some areas inthe social sciences but is not a general phenomenon. In applied social research thereis a clear body of knowledge concerning the techniques and rationale of statisticaltests and a clear set of skills which need to be learned. This information lends itself tothe knowledge-transfer approach. Lectures and text-books are the traditional way ofcarrying out this task. In this context the use of the Web may simply be seen as aconvenient way of distributing relevant information. Even in this case, however, theability of the learner to rerun material, to attach their own notes and illustrations topresented material, to follow alternative pathways and to seek clarification throughthe use of linked resources, may give the Web-based learning package an advantage.According to Mayes the incorporation of learnt materials into the repertoire ofknowledge and skills which makes the learning useful constitutes the constructionphase of the learning process. In this phase the newly-acquired skills are applied topractical situations. In the classroom this objective is commonly carried out inworkshops or practical sessions: in teaching applied social research methods thismight involve the construction and carrying-out of a questionnaire survey and itssubsequent analysis. Within a telematics-based learning environment this task cantake place either in an entirely simulated “virtual” environment or more simplythrough the use of standard packages which are accessed on the Web but used inreal-life situations. The major problem with this latter approach is probably one oftime. There are likely to be less severe problems involved in the application ofstandard packages to existing bodies of material and the ability of the Web toprovide access to data collected in real-life studies (e.g. as made available from theESRC Data Archive) makes it possible to give students the chance of working withreal data rather than the artificial material frequently encountered in text-books.It is a basic tenet of the social constructivist approach that final mastery of newknowledge involves learning from and with other people. Mayes calls this thedialogue phase and originally saw it as occupying the final stage in the learningprocess. More generally, however, dialogue can be seen as an underlying principlethroughout the learning process (see Fowler and Mayes, 1997): effective knowledgetransfer is likely to be enhanced through questioning, the construction andapplication of new knowledge is likely to be aided through collaboration and

SCHEMA 6

comparison. Final mastery, however, is likely to involve self-conscious commitmentand a willingness to defend the new paradigm. The newly-acquired knowledge isverified and demonstrated through a process of social comparison involving facing-up to the criticism of peers, teachers and experts. In face-to-face learning situations,this stage of the learning process is carried out in tutorials, seminars and, perhapsmost importantly, in coffee-room chats. To provide opportunities for this level ofmastery to be displayed in a learning environment delivered through C&IT facilitieshave to be made available for conferencing and for shared working spaces andcourse design has to stimulate learners to engage in debate.Both learning environments being investigated by SCHEMA, ProTo and TELSIpro,have been designed to accommodate the learning model outlined above (for detailssee SCHEMA deliverables 5.1 (Pulkkinen and Ruotsalainen 1998) and 7.1 (Rousselleand Pärkkä 1998). The module on quantitative data analysis was mounted in theProTo environment.

SCHEMA 7

4. The Quantitative Data Analysis Module

Overview of Course

The course outline is given in figure 1:

In addition to the course notes and other learning materials supplied on-line,students were recommended to purchase two standard texts:Rose, D. and Sullivan, O. (1996): Introducing Data Analysis for Social Scientists.Buckingham: Open University Press.Loether, H.J. and McTavish, D.G. (1993): Descriptive and Inferential Statistics: AnIntroduction. Boston: Allyn and Bacon.Statistical analyses were carried out within SPSS and Excel, both mounted on Stirlingservers; questionnaire design and analysis was carried out within the PinPointpackage, also mounted in Stirling. Other packages used and mounted in Stirlingincluded material developed by the Teaching and Learning Technology ProgrammeStatistics Consortium (STEPS and Luddersby Hall) and by Queen’s University,Belfast (QUTAL). Links on Web pages were provided to three sets of resourcesdeveloped by Australian teams: Introductory Statistics On-line developed by theDepartment of Statistics in the University of Newcastle, N.S.W., Resources forStatistics Teaching, developed by the Queensland Department of Education(http://curriculum.qed.qld.gov.au/kla/eda/resource.htm)and a set of resourcesexploring the topic of “Chance and Data in the News” extracted from the pages of

Figure 1: Course Outline

SCHEMA 8

the Hobart Mercury and presented by the University of Tasmania and the AustralianAssociation of Mathematics Teachers Inc.(http://www.ni.com.au/mercury/mathguys/mercury.htm). The text by Rose andSullivan includes a disc of data in SPSS format extracted from the files of the BritishHousehold Panel Survey and this was used following the courses for additionaltraining in aspects of longitudinal data analysis.

Learning Environment

The unit was offered within the ProTo environment, described in SCHEMAdeliverable 5.1. In the introductory session students were allocated log-in IDs andpasswords and shown a simple introduction to ProTo (Appendix D). The programitself was maintained on a server in the Institute of Education in Oulu. All coursematerial was accessed via ProTo, though some of it was kept on servers elsewhere(e.g. the Durkheim correlation exercise, the QUTAL self-assessment materials and theLuddersby Hall exercise were mounted on a server in Stirling, material on samplingfrom the Hobart Mercury was mounted on a server in Australia). Students quicklybecame familiar with the interface, appreciating the simplicity of the Web-basedenvironment. The level of service was high, only one serious break occurring as theresult of a fault in the Edinburgh connection between the EaSTMAN supplyingStirling and the external world.

Resources

course notes

A series of “pages” of notes, prepared by Dr Björkman, provided the mainframework for the course. An example of the pages, taken from the section ondescribing bivariate relationships is shown in figure 2.

Figure 2: Describing Bivariate Relationships

SCHEMA 9

on-line texts and other materials

Links to other resources were available within the course. An example is the link tothe Glossary of Statistical Terms produced by a consortium funded under the UKTeaching and Learning Technology Programme (Figure 3).

Figure 3: Resources

SCHEMA 10

on-line applications (Durkheim example and Luddersby Hall)

Links to applications running on a Stirling server (e.g. Pinpoint, SPSS and Excel)were also available. An example is an exercise on the use of correlation analysis,using the data which lead Durkheim to conclude that there was a relationshipbetween modernization and rates of suicide in the countries of Western Europe at theend of the nineteenth century (Figure 4).

Full instructions for the Durkheim exercise are given in Appendix A.

Figure 4: Durkheim Revisited

SCHEMA 11

self-assessment tests (QUTAL material)

Self-assessement tests on statistics for social scientists developed by Queen'sUniversity, Belfast (Figure 5), were available (but little used)

Assessment

During the course a number of tests and exercises were administered. An example isshown in Appendix B. At the end of the unit a “take-home” paper was administered(Appendix C).

Figure 5: Question Bank

SCHEMA 13

5. The Learners

As intended, the course was trialled with students enrolled in the Master’s Degreeprogramme in Applied Social Science offered by the University of Stirling.Unfortunately, the number of students recruited to the course (seven) was smallerthan anticipated. Three of the students were full-time, four part-time. Theirbackgrounds were in nursing studies (three), social work (two), sociology (one) andsocial policy (one); all but one were female. Prior knowledge of research methodswas varied, as was experience of using computers and the InternetAt the end of the unit all had achieved a satisfactory standard of knowledge. Post-course discussions suggested that they had appreciated the availability of the coursenotes as revision aids. They also suggested that although interactive aspects of thecourse such as the Durkheim exercise on correlation and the Luddersby Hall exercisein the use of Chi-square were valuable, they took more time to complete than hadbeen allocated and they were doubtful whether a greater use of such interactivematerial could be accommodated within the time constraints of the unit.Little use was made of the On-line Cafe, the discussion areas or the workgroup areasavailable within the ProTo environment. Email was used primarily to makeappointments and to check the timing of course assignments. Part of this relativefailure to use the communication facilities built into ProTo undoubtedly reflected theready physical availability of the course instructors and the fact that for much of thetime students were seated adjacent to each other; part also reflected the nature of thedefined learning task. The skills being focussed on have a rather linear form and,especially at an introductory level, leave relatively little room for discussion. Giventhe unfamiliarity of most of the students with quantitative research greater effortwould have been required on the part of the course instructors to engage them indiscussions. Within the resource library, material was available which dealt with alarge number of “frequently asked questions” and relatively few areas were left forspecific reference to the course tutors or for discussion among the learners. Theindividualistic approach was further emphasised by the nature of the exercises set upwithin the course. Each of these points will need attending to in future developmentsof the module for delivery at a distance.

SCHEMA 15

6. Student Evaluation of the Unit and Suggestions for the Future

In the final session of the unit a general discussion with students took place aboutways in which the unit could be changed in order to be mounted successfully as afully on-line module for distance delivery in the future. Not surprisingly, most of thepoints raised can be found in other accounts of computer-based distance-learningcourses (e.g. Hansson et al., 1997; Paulsen, 1997).More advance information is required on the techniques and the pedagogy to beused in an on-line course. Few students taking post-qualifying courses in appliedsocial research have much familiarity with computer-based instruction or withcollaborative patterns of learning. In the trial group, the level of knowledge of theInternet was extremely varied and only three of the students had access to their ownmachines elsewhere (and one of these, by her own admission, was still a “novice”user). Students were initially excited at the prospect of accessing the course throughan NC at home and were disappointed when this turned out not to be possible.Students need a high degree of motivation and self-discipline. Although the use of aweb browser simplifies the task of accessing computer-based material it still imposesan additional obstacle between the student and the learning resource. In order toensure continued involvement a regular programme of exercises is helpful, but caremust be taken that these do not take up too much time. It was suggested that theinstructors tended to underestimate the amount of time required to completeexercises and the general point was made that Web-based courses appear to makeheavier time demands on participants than is believed at the outset.In order to engage the students material presented should be relevant to theirinterests. It follows that the instructor should have good knowledge about thebackground of the students and the reason for their taking the course. There wassomething of a split in the pilot group between those with experience and interests insocial research and those coming from a practicing nursing background. Examplesused in the course were largely taken from the health and welfare area in order toprovide what was hoped would be a common meeting-ground, but some of theancillary material used made specific assumptions concerning knowledge ofsociological theory which put some of the students at a disadvantage.Good help facilities must be available to deal with both the technical aspects ofcourse provision (e.g. to give assistance when there is a problem with the computeror network) and with those concerning the material being taught (e.g. in relation torunning a questionnaire design package such as PinPoint). Although the provision ofstandard reference materials can help with common queries, students thought it wasstill essential to have more personalised assistance available as and when required.Asynchronous contact via email can help with individual enquiries, but studentswere of the opinion that in the fully-distanced system these would needsupplementation with synchronous meetings, preferably involving video-conferencing (though they were unable to come up with any clear rationale for thisbelief).Discipline is required in the use of e-mails if the instructor is not to be “drowned” inmessages. Students should be encouraged to use on-line communication facilities forsharing problems and solutions among themselves. The use of a common “questionboard”, a specialised form of “bulletin board”, would enable students to learn from

SCHEMA 16

each other. In the classroom there is a degree of embarrassment about askingquestions which might be thought to be too simple; this is thought to be less likely tooccur when the communication is electronic. It might be possible to buildquestioning into the course by specifying that each student should post a number ofquestions during each phase of the course.Face-to-face meetings are very helpful in developing group identity and solidarity. Afully on-line version of the course will need to provide other avenues (e.g. the on-linechat area) for informal communication. The presentation of photographs and shortstatements of personal characteristics is also likely to be helpful. A number of grouptasks should be specified to encourage students to get in the habit of workingtogether. Social research is generally taught in a rather individualistic way, stressingskills to be acquired by the individual student. By contrast, most research projectsinvolve teams of researchers and it was suggested that this should be reflected moredirectly in the design of the course. Students felt that ProTo provided a suitableenvironment for the development of group tasks but also thought that little of thispotential had actually been exploited during the pilot.Faith in the ability of C&IT to deliver and support learning is easily shaken. Studentanxieties are easily roused when they are “glitches” in the system (e.g. making anerror in typing in a password; a network crash). One of the promised advantages ofthe NC approach is that it should be as transparent as possible, with set-up tasksbeing the responsibility of the course administrator rather than being left to theindividual student. Post-qualifying health and welfare students taking an on-linemodule in quantitative research methods are likely to be anxious about both C&ITand the contents of the course itself. The instructor must be sympathetic to theirconcerns on both counts.Formal debriefing meeting(s) at the end of the course are essential.

SCHEMA 17

7. The Next Phase

Evaluation of the quantitative data analysis module in terms of the learning modeloutlined in section 3 suggests that while it was reasonably successful in terms ofdelivering accepted knowledge concerning the techniques and rationale of researchmethods it was not as successful in encouraging learner involvement in either theapplication of new skills or developing their social confidence in relation to themethods. The less successful aspects reflect the relative under-use of thecommunication and collaboration facilities of the learning environment. Revisions tothe unit will need to address this problem directlyThe biggest disappointment with the unit was that it proved impossible for thestudents to use the Acorn NCs. It had originally been planned to provide all thestudents who could benefit (i.e. those that had access to a telephone connection) withan NC. The reasons for this failure are fully detailed in D2.2 (Booth 1998). Thestudents were keen to use the NCs as most had no access to computing or theInternet from their homes. A major knock-on effect of the NCs not working was thatthe on-line collaborative work that was planned as part of the unit was renderedinappropriate as the computing tended to be done as a group activity.Despite the failure to get the NCs working with ProTo, ProTo itself proved verypopular with the students taking this course. They appreciated that they could accessthe materials in ProTo at times that suited them---this was especially true during therevision period. They also found the ProTo interface straight-forward to use andencountered few problems accessing the materials.The materials that were mounted in ProTo, in this first running of the unit, werelargely textual/numerical. Whilst there are many ways in which this course could be"improved" by the judicious use of multimedia, the materials were appreciated bythe students and they made heavy use of them during their revision.A revised version of the unit will be offered on-line in autumn 1999, involvingstudents from other sites in the SCHEMA Consortium. The various points raised bythe students who took part in the pilot run will be attended to in the redesign of theunit, with particular attention being paid to the desirability of collaborative working.A further Report will be produced in Spring 2000 and it is planned that a verifiedversion of the unit will then be made available to students throughout theConsortium.

SCHEMA 19

References

Booth, S.P. 1998. Report on the implementation of Network Computers. Initial Evaluationof System. SCHEMA deliverable D2.2http://www.stir.ac.uk/schema/deliverables/D2.2.pdfFowler, C.J.H. and Mayes, T. 1997. Applying telepresence to education. BT TechnologyJournal, Vol.15, no.3, p189, October 1997.Hansson, H. et al. 1997. Utvärdering av distanskurs I Signalteori för hörselrehabilitering.Stockholm: DUKOM. ISBN 91-630-6495-2Mayes, T. 1995. Learning Technology and Groundhog Day. In W. Strang, V. Simpson, &D. Slater (Eds) Hypermedia at work: Practice and Theory in Higher Education.Canterbury: University of Kent PressPope, W. 1976. Durkheim’s ‘Suicide’: A Classic Analysed. London: University ofChicago Press.

Paulsen, 1997. The Third Generation NKI Electronic College: A Survey of StudentExperiences and Attitudes.http://www.nettskolen.com/alle/forskning/33/evaluati.htmPulkkinen, J. and Ruotsalainen, M. 1998. CMC techniques as applicable to the support ofopen and distance learning and the specification of appropriate tools for use in SCHEMA.“State-of-the-art” review specification. SCHEMA deliverable D5.1.http://www.stir.ac.uk/schema/deliverables/D5.1.pdfRousselle, E. and Pärkkä, T. 1998. The specification of simulation techniques, originallydeveloped for language teaching purposes, for the provision of training to health, educationand welfare workers. SCHEMA deliverable D7.1.http://www.stir.ac.uk/schema/deliverables/D7.1.pdf

SCHEMA 21

Appendix A: Durkheim Exercise

UNIVERSITY OF STIRLING/SCHEMA

Autumn Semester 1998

MSc (ASR) Data Analysis: Exercise 4

Durkheim and the Art of Correlation

In one of the classics of sociological investigation, Suicide (originally published in1897), Emile Durkheim explored the relationship between the occurrence of suicideand the modernization of society. Much later sociological theorising concerninganomie and the problems of the "mass society" is a direct descendant of Durkheim'smonograph.One of the planks in the argument advanced by Durkheim is that the decline ofRoman Catholicism reflects the emergence of the individualistic values whichaccompany the move from mechanical to organic solidarity. The argument isillustrated through the use of data on the distribution of suicides and the percentageof Roman Catholics in the countries of Western Europe. Durkheim claims that thedecline of Roman Catholicism will be accompanied by an increase in anomie and,hence, of suicide. The impact of the decline in mechanical solidarity is aided andabetted by the rise of Protestantism:

“Thus, everywhere without exception, Protestants show far more suicide thanthe followers of other confessions.”

The data which Durkheim used was extracted from a study by Henry Morselli,published in 1882, and purports to show suicide rates and percent Roman Catholicaround 1870. One difference from the original data concerns that for Germany.Durkheim made use of an unweighted average which was dominated by oneextreme case (Saxony), resulting in figures for both variables which were far toohigh. The figures for Germany in the spreadsheet are based on wiighted data. (Nbthe question of weighting is complex, especially when ecological or areal data isbeing used). According to one commentator (Pope, W. (1976): Durkheim's Suicide):

“a single German state, populated by fewer than three million persons,consituted the empirical base on which Durkheim associated Protestatntismwith high suicide rates”.

A century later, data extracted from OECD Reports, provide additional figures. Inthis case, as well as data on percent Roman Catholic and the suicide rate we can alsouse information on three other measures often used as indicators of“modernization”: the percent of people attending church weekly (a measure ofsecularization), the newspaper circulation rate (a measure of knowledgedissemination) and GNP per capita (a measure of economic develop-ment). Thepassage of a hundred years may be expected to have seen a considerable increase inthe extent of modernization and it may also be anticipated that the pace of changewill have varied considerably from country to country, leading to new patterns in therelationship between suicide and the various measures of modernization.

SCHEMA 22

EXPLORATORY ANALYSIS: STAGE ONE

Open the Durkheim file by clicking on the link in the ProTo page (the data is athttp://www.stir.ac.uk/schema/teaching). The data is in an EXCEL spreadsheet; theapplication should start immediately. The data for the 1870s is in cells B7:C20 and forthe 1980s in E7:I20. The data is ordered alphabetically by country.

a) Examination of Distributions:To begin with we shall use the SORT facility in EXCEL to examine the 1870s data,beginning with the patterning of Roman Catholicism. Select A7:I20 (the data of theentire table); choose Data=>Sort and sort by Column B. Note that there is no headerrow included (check that the right box is ticked).

What does the distribution look like? Do most countries bunch towards the middle ofthe percentage scale or is there evidence of polarisation?

Now repeat the analysis for the Suicide Rate. Remember to ensure that the whole setof data A7:I20 is included in the sort routine and sort by Column c (descending).

What does the distribution look like: is there a regular gradient in the rates or aresome values extreme?

b) Graphical Analysis:To examine Durkheim’s assertion that the higher the percentage of Roman Catholicsthe lower the suicide rate we shall draw a scatterplot of the two variables. This hasalready been done for you but the procedure is as follows: select cells B7:C20(making sure that you do not include the labels in your selection) and click on theEureka graph icon. Choose the scatter graph option. Tidy up the graph using thetools provided (mainly shown at the bottom of the window).

Note:1. the direction of the line. Does it go from high left to low right, indicating a

negative relationship between the two variables, as predicted by Durkheim?2. the steepness of the line. This illustrates the power of the relationship, with a

steep line indicating that the dependent variable (suicide) is greatly affected byvariation in the independent variable (%RC).

3. the degree of scatter around the line. The more scatter the weaker therelationship; a perfect relationship is shown by all the points being on the line.

SCHEMA 23

c) Correlation Analysis:

Pearson’s correlation coefficient was invented a few years after Suicide was written. Ifit had been available to Durkheim what would it have shown?

To calculate a correlation coefficient in EXCEL use the formula=correl(array1,array2). For the 1870s data this is represented as=correl(B7:B20,C7:C20).

Examination of the scatterplot and the data produced by sorting on the suicide ratesuggests that Denmark is something of an outlier and may well be affecting thecorrelation calculation. To check for this repeat the analysis, deleting Denmark (i.e.=correl(B8:B20,C8:C20) (n.b. make sure Denmark is the country in row 7!).

What does the analysis suggest? Was Durkheim right?

The 1870s data is not terribly convincing; how about the material for the 1980s?

Produce a report showing the results of your analysis. What do the two sets of datasuggest about the validity of Durkheim’s argument?

Further stages of analysis will make use of multi-variate techniques. You might liketo look these up in the ProTo library.

Duncan Timms20th October 1998

SCHEMA 25

Appendix B: MSc (ASR) Data Analysis: Test 3

Name: … … … … … … … …

1.a Several measures of correlation have been constructed for different scale levels.Mark with a + for each measure listed below the level they are constructed for.Leave other alternatives without any marks.Measure Nominal Ordinal Interval Ratio

Lambda or Guttmancoefficient of correlation … … … … … … … … … … … … … … … …Gamma … … … … … … … … … … … … … … … …Spearman’s rho

… … … … … … … … … … … … … … … …Pearson’s (theproduct correlationcoefficient) … … … … … … … … … … … … … … … …

1.b Two measures of correlation, the percentage difference and Yule’s Q, areconstructed for a specific kind of variables. Describe what is specific aboutthese two correlation coefficients (e.g. what kinds of variables can be analysedthis way).… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

2. In a small group of children, age and height (cm) was registered for the sixmembers as presented below. Compute the appropriate correlation coefficientfor the association between the two variables and give a verbal description ofthe outcome. Each pair of values for each child is presented with age first,followed by height.

10 139, 11 142, 11 145, 12 155, 14 161, 14 158

Correlation coefficient : … … … … … … … … … … … … … … . = … … … …Verbal description of the outcome.… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

SCHEMA 26

3. In a fictitious example, attitudes towards local politicians were measured. Oneof the aims was to study if these attitudes were associated with social class. Theresulting data is as follows:

_________________________________________________Attitudes Social statustowardslocal Low MiddleHighpoliticians n n n

_________________________________________________Positive 12 5 5Neutral 8 5 6Negative 0 6 9_________________________________________________Total 20 16 20_________________________________________________

Compute the appropriate measure of correlation and give a verbal descriptionof the outcome.

Correlation coefficient : … … … … … … … … … … … … … … . = … … … …Verbal description of the outcome.… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … .… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … .

4. In a country with three political parties a researcher wanted to find out if therewas any relation between marital status and voting behaviour. A sample studywas carried out and the result is presented below.

___________________________________________________Voting Marital statusbehaviour Single Married Divorced

n n n___________________________________________________Conservative 12 10 8Socialist 8 9 7Liberal 5 6 10___________________________________________________Total 25 25 25___________________________________________________

SCHEMA 27

Compute the appropriate measure of correlation and give a verbal descriptionof the outcome.Correlation coefficient : … … … … … … … … … … … … … … . = … … … …

Verbal description of the outcome.… … … … … … … … … … … … … … … … … … … … … … … … … … … … … …… … … … … … … … … … … … … … … … … … … … … … … … … … … … … …Nb for the time being we shall ignore the fact that this is sample data.

5. A teacher in sociology wanted to find out if there was any relationship betweenstudents' ability in theory and skill in quantitative methods. She ranked herstudents according to the two variables and the data hereby collected ispresented below. The person with the highest level of skill is placed first, withthe person with the lowest skill placed last.

Ability in theory: John, Eliza, Franz, Vera, Jonathan, SaraSkill in quantitative methods: Vera, Franz, Sara, John, Eliza, JonathanCompute the appropriate measure of correlation and give a verbal descriptionof the outcome.Correlation coefficient : … … … … … … … … … … … … … … . = … … … …

Verbal description of the outcome.… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

SCHEMA 28

6. A survey presents the following set of data on gender and wish to get married inthe next five years. 35 women and 20 men wanted to get married within fiveyears and 25 women and 40 men did not want to marry within five years.Set up a table for the data presented, compute the percentage difference (d%) andYule’s Q and give a verbal description of the outcome.

____________________________________________________________Females Males Totaln % n % n %

_____________________________________________________________Want to marryDo not want to marry

_____________________________________________________________Total

_____________________________________________________________d% = … … … … Q = … … … …

Verbal description of the outcome.… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …… … … … … … … … … … … … … … … … … … … … … … … … … … … … … … …

SCHEMA 29

Appendix C: Take-Home Test

UNIVERSITY OF STIRLING SCHEMA/Department of Applied Social Science

MSc (ASR) – Quantitative Data Analysis

Take-Home Test: ALL Questions to be answered

1. What do you consider to be the most appropriate measures of central tendencyand dispersion for the following sets of data? In some cases you might like tosuggest an alternative. In addition please state whether you consider the datato be measured on nominal (N), ordinal (O), interval (I), or ratio (R) scales

a) The distribution of respondents on a permissiveness scale which rangesfrom 0 (“Very low permissiveness”) at one end to 5 (“Very highpermissiveness”) at the other.

b) The distribution of household incomes in Scotland.

c) The distribution of body weight in a sample of lecturers.

d) The distribution of the percent Asian across British cities.

2. What does the graph below show?

SCHEMA 30

3. An analysis of social mobility includes the following raw data. Chi-square (χ2)is quoted as 73 with 1 d.f.; (p<.0001). What does this mean? Is χ2 anappropriate measure? How would you describe the relationship shown in thetable?

Son working class Son middle classFather middle class 20 40Father working class 180 25

4. Multiple Choice Questions (more than one answer may be correct)

a) Which of the following is the standard way of writing the equation for astraight line as used in the simple linear model:

(i) b + aX?(ii) aX + bX?(iii) a + bX?(iv) aY + bX?

b) What is r2

(i) the (squared) correlation coefficient?(ii) the residual variance in a regression analysis?(iii) the (squared) regression coefficient?(iv) the proportion of variance explained in a regression analysis?

c) What does a partial regression coefficient represent?

(i) the overall fit of a model(ii) the correlation between the independent and dependent variables

when controlling for other variables?(iii) the effect of a change in the independent variable on the dependent

variable when controlling for the other variables?

d) c2 is a measure

(i) of the strength of association between two nominal variables(ii) of the fit between an observed distribution and one predicted by a

specified model(iii) of the significance of a set of sample data(iv) of the direction of association in a table

SCHEMA 31

e) What is the standard deviation of the distribution of repeated sample means?

(i) the standard deviation of the sample(ii) the mean error(iii) the sample error(iv) the standard error of the mean

f) Why are inferential statistics used?

(i) to analyse patterns of association(ii) to prove the research hypothesis(iii) to test if the normal distribution applies to the sample(iv) to make claims about population parameters based on a sample

5. The following table is contained in a report on mental health and familybackground among a cohort of men in Stockholm. What does the table show?

TABLE 1: MEASURES OF COPING ABILITY AT AGE 20 YEARS BYPARENTAL MARITAL STATUS WHEN COHORTMEMBERS WERE AGED TEN YEARS

%High %Low Pop‘n

Married 10.4 12.7 6219Never-married 9.2 17.2 87Widowed 7.8 15.5 103Sep/Divorced 9.0 20.4 455

Total 10.2 13.3 6928

Key:%High % of cohort rated as being of high coping ability%Low % of cohort rated as being of low coping abilityPop‘n population at risk

SCHEMA 32

6. The data below is taken from an article which investigates the relationshipbetween well-being (measured on a scale combining subjective expressions ofmorale and a number of health variables) and social participation (measured by ascale containing items on friendship, kin relations, and neighbourly help) amonga sample of elderly people living in social housing. The investigation was alsoconcerned with the extent to which the relationship between the two variablesmight be affected by neighbourhood characteristics, notably the degree of agesegregation. Two sets of sample material are presented, one obtained in an age-concentrated environment (where being 65 years and older is a condition ofresidence) and one in a “normal” mixed-age environment. The samples weredesigned to control for as many other variables as possible

Well-being by Social Participation among Elderly People in Social HousingWell-being Social Participation

Low High TotalHigh 17 35 52Medium 69 62 131Low 33 27 60Total 119 124243

Distribution of Well-being by Social Participation in an Age-concentrated EnvironmentWell-being Social Participation

Low High TotalHigh 8 15 23Medium 18 29 47Low 4 14 18Total 30 58 88

Percentage Distribution of Well-being by Social Participation in a Mixed-ageEnvironment

Well-being Social ParticipationLow High Total

High 9 20 29Medium 51 33 84Low 29 13 42Total 89 66 156

What does the data show? (hint: to summarise your findings calculate gammas anduse percentages to describe your findings). Assuming that the data is generalizablewhat recommendations might you make to, say, a housing association?

SCHEMA 33

Appendix D: A Short Introduction to Proto

Introduction

ProTo (Project Tools for Learning) is a web-based tool for students and their tutors to planand carry out learning projects. ProTo is produced by the Unit for New Learning Media &Educational Technology at the Faculty of Education, University of Oulu, Finland. The URLor Internet address for ProTo is: http://edtech.oulu.fi/proto_v2/index.cfm. Use of ProTorequires a username and password.What you can do with ProTo:• Edit documents either on your own or in a team (only one member of a team can edit a

document at any one time; if you cannot access a document it is almost certainly becauseanother member of the team is editing it)

• When you are working on a document you can choose not to show it until you are ready• You can publish documents for different reader groups: either your team or the whole

course• To encourage discussion, all the documents you produce are automatically listed

discussion• You can refer to other texts by using the linking tool.

Starting ProTo

When you first start ProTo (http://edtech.oulu.fi/proto_v2/index.cfm), you will see thelogin screen (you should bookmark this page in Netscape— see final section on how to onhow to do this):You must enter your ProTo username and password in the appropriate boxes. Beforepressing the “Log in” button remember to change the language to English!

Enter your username here

Enter your password here

SCHEMA 34

Having done this but not pressed the button, you should see (with your username instead ofspb001):

Now press “Log in”.The main ProTo environment will shortly display:

Note the four “buttons” across the bottom: “project office”, “workshop”, “communications”and “library”. Each button accesses a different group of ProTo’s features.

SCHEMA 35

Project office

This is for course administration. You will occasionally visit this area to deal withadministrative matters.

Message Board

This is where your instructors, tutors and administrators write important/urgent messagesabout the course. Student messages only stay visible whilst they are on-line. Have a look atthe “Message Board” every time you log in for course information.

Course Info

Here you will find general information on your course: duration, number of participantsand milestones to pace your project work. Read the milestone information that details thedeadlines of specific subtasks in ProTo. ProTo informs you about the time left before thenext milestone.

Student Card

Fill in personal information on your student card when you first log in. You can changeyour password here.

Team folder

In here you can create your own team by picking members from the list of participants. Youcan only form one team. However, you could be selected as a member of many.

Tutor folder

This folder is for tutors. The tutors can send messages to the students in their group via thistool.

Workshop

This is the area for students to edit and publish documents: study plans, project plans orreports and other texts. You can also read and comment upon other student’s texts. Privatenotes (for personal use only) can also be stored here.

Communication

There are several different ways to communicate using ProTo. The main mechanism is viathe discussion groups. There are groups for the whole course and for your group(s). Thereare also tools to comment on the course materials and the “on-line Café”— a space tosocialise.

SCHEMA 36

There is a mailbox too.

Library

All the course materials are to be found here. A search engine is present to help quickly finddocuments (if you can recall a key word or phrase). The library is the part of ProTo that youwill use most.

SCHEMA 37

Using Proto Features

Project office

Click on the button and the project office reveals itself

Remember to start by checking the message board. To do this click on the “Message board”link.

SCHEMA 38

The messages are read by clicking on the underlined part that is visible. Having done this asecond, smaller window opens revealing the full message.

Adding your own message is easy: a new link appeared when “message board” was clicked.This new link (Write) is for adding messages. Click “Write” and another window opens:

Simply type your message in the Text box and press the “Post” button when you are readyfor it to be posted. Alternatively, select “Discard” and it will not be posted.The only other feature that interests us at this stage is the “Student Card”— these yourpersonal details. Details such as email address, full name and interests. Check that yourstudent card is correct. Apologies if this is wrong but you can change.This is where your password can be changed. Do this if you wish to change from the default(your username). Change it to your university password (or some other choice). Note thatyour Proto password is entirely independent of your University one. If you change yourUniversity of Stirling password your Proto password will remain unchanged.

SCHEMA 39

Library

The library area looks like:

There are several ways to use the library to access course materials. One is to examine thematerial in sequence as the course progresses. The other, probably more useful for revisionpurposes, is to search to material based on keywords.Courses in Proto are organised into what are called “Material Summaries”. To see thesesummaries select “Materials” and then “Browse”

The summaries then appear along with the name of which instructor placed them into Proto.

SCHEMA 40

To select a particular summary, click on the link:The summary can then be navigated in two ways. Firstly, there is a list of the pages in thesummary in the “Index” window that opens:

Next button

SCHEMA 41

Each item is a link to a page. Sometimes Netscape hides this window behind the mainwindow— simply move the main window to reveal or click on the Windows taskbar.Alternatively, each page has links to the previous and next pages. The page above is the firstin the summary and so has no previous link (i.e. there is no small left arrow icon next to thetitle) but does have a next.Clicking on the next button, reveals the next page:

SCHEMA 42

That’s how simple navigation in Proto is! Note that the page shown above has both previousand next buttons.To read through a particular summary, either work through it using the page links or usethe next button to turn to the next page.

SCHEMA 43

Communications

The communications area looks like:

and supports a number of ways of interacting with fellow course members. Proto is alearning environment designed primarily for students learning at a distance and thiselectronic meeting space is a powerful tool to facilitate interaction between these individualsat a distance.Firsly, we’ll examine the on-line Café

SCHEMA 44

To place a message into the On-line Café is very simple. Firsly, click the link (On-line Café):Type your contribution into the box at the bottom of the screen and when you wish to add itto the discussion press the submit button (it will appear at the very top of the list ofmessages).The Comments feature allows comments to be made on each of the course summaries andthe Discussion contains discussion topics. Some discussion topics will be added over theduration of the course.

Logging Off

When you have finished your Proto session, you should logout. To do this select “Session”and “Logout”

SCHEMA 45

You will then see the screen below indicating that you have left Proto.On some courses a different page will appear. The course instructor decides upon the page.The above is the default

SCHEMA 46

Useful Information

If you any queries about Proto 2, contact Simon either via email: [email protected] or phone7247 (note that most computer laboratories have phones).

SCHEMA 47

Netscape Bookmarks

Bookmarks, as the name implies, mark useful or interesting Web sites for futurereference. Once a Web site is book marked there is no need to remember“http://www.bbc.co.uk/” or whatever, simply use the bookmark.

To mark a site is simplicity itself. When at the site, select Bookmarks and then “AddBookmark”. There after the site is in your bookmarks and can be accessed (fromwherever) by selecting bookmarks and the site.

SCHEMA 48

Try this! You can add as many bookmarks as you like. They will be available fromany PC on campus (sorry but from a Mac you must launch Netscape using yourbookmark file and then select the appropriate link).

Simon Booth26th August 1998