Enhancing Teachers' Technological Knowledge and Assessment Practices to Enhance Student Learning in Technology: A Two-year Classroom Study

Research in Science Education 31: 155–176, 2001.© 2001 Kluwer Academic Publishers. Printed in the Netherlands.

Enhancing Teachers’ Technological Knowledge and Assessment Practicesto Enhance Student Learning in Technology: A Two-year Classroom Study

Judy Moreland, Alister Jones and Ann NorthoverCSTER, University of Waikato

Abstract

This paper reports on a two-year classroom investigation of primary school (Years 1–8) technologyeducation. The first year of the project explored emerging classroom practices in technology. In thesecond year intervention strategies were developed to enhance teaching, learning and assessmentpractices. Findings from the first year revealed that assessment was often seen in terms of social andmanagerial aspects, such as teamwork, turn taking and co-operative skills, rather than procedural andconceptual technological aspects. Existing formative interactions with students distorted the learningaway from the procedural and conceptual aspects of the subject. The second year explored the devel-opment of teachers’ technological knowledge in order to enhance formative assessment practices intechnology, to inform classroom practice in technology, and to enhance student learning. Interventionstrategies were designed to enhance the development of procedural, conceptual, societal and technicalaspects of technology for teachers and students. The results from this intervention were very positive.This paper highlights the importance of developing teacher expertise pertaining to broad concepts oftechnology, detailed concepts in different technological areas and general pedagogical knowledge.The findings from this research therefore have implications for thinking about teaching, learning andassessment in technology.

Key Words: formative assessment, teacher knowledge and student learning

Introduction

The New Zealand Ministry of Education gazetted Technology in the New ZealandCurriculum (Ministry of Education, 1995) in February 1999. Schools are now re-quired to implement this curriculum and teachers are required to plan, teach andassess technology programs that will allow children to gain technological knowledgeand understanding, technological capability and understanding and awareness of therelationship between technology and society, in seven technological areas. To carryout effective assessment of student learning in technology education teachers neednot only an understanding of technological practice, of technology and technologyeducation; and student learning and progression; but also appropriate formative andsummative assessment strategies. Research carried out in the course of teacher de-velopment programs in technology education (Moreland, 1999; Northover, 1997)would suggest that technology assessment is not yet well understood by teachers. Acoherence is yet to develop between the technology curriculum documents learningoutcomes and assessment procedures.

156 J. MORELAND, A. JONES AND A. NORTHOVER

This paper reports on a two-year classroom based investigation of primary school(Years 1–8) teaching, learning and assessment in technology. The first year of theproject explored emerging classroom assessment practices in technology, a new sub-ject area for primary teachers. In the second year, intervention strategies were negoti-ated, developed and refined by the research team and the teachers to enhance assess-ment procedures. The intervention strategies assisted teachers to focus on technolog-ical knowledge, including conceptual, procedural, societal and technical outcomes intheir planning in their subsequent formative interactions with students.

Classroom-Based Assessment

Classroom-based assessment, and in particular assessment that has as its goal theenhancement of student learning, is the focus for this paper. How we see learningis of prime importance in examining the role of assessment in enhancing children’slearning and achievement. This research project is based on a sociocultural view oflearning where human mental processes are situated within their historical, culturaland institutional setting (Wertsch, 1991, cited in Gipps, in press), and learning is aform of cognitive apprenticeship (Brown, Collins, & Duguid, 1989). Such a perspec-tive requires that assessment of the processes of learning occur within a social setting.Hattie and Jaeger (1998) argue strongly that assessment needs to be an integral partof a framework of teaching and learning rather than as an adjunct to learning. Hencethe focus of our research is on formative interactions and summative judgementswithin the classroom.

Clearly, classroom-based assessment does not exist in isolation. It informs and isinformed by school-wide assessment policy and practices, which in turn are influ-enced by national standards and curriculum. However, the central role of classroom-based assessment in the promotion and enhancement of student learning and achieve-ment is widely accepted both within New Zealand (Crooks, 1988; Ministry of Ed-ucation, 1993, 1994; Bell & Cowie, 1997) and internationally (Gipps, 1994; Black,1998; Black & Wiliam, 1998).

Formative assessment relates to finding out about student learning and “the prin-ciple that feedback is essential to the conduct of effective teaching” (Black, 1998,p. 25). Expected, is that assessment practices will be able to provide short-termfeedback so that obstacles to students’ learning can be identified and tackled. Forma-tive assessment, as the means to provide feedback for assisting in making learningadaptive and thereby more effective, cannot be added as an extra to a program ofwork. Instead, the procedures need to be built into the program of work.

The core of the activity of formative assessment lies in the sequence of two actions.The first is the perception by the learner of a gap between a desired goal and his orher present state (of knowledge, and/or understanding, and/or skill). The second isthe action taken by the learner to close that gap in order to attain the desired goal(Ramaprasad, 1983, cited in Black & Wiliam, 1998). Clearly, formative assessmentwhich guides learners towards valued learning goals can only be generated with tasks

ENHANCING TEACHERS’ TECHNOLOGICAL KNOWLEDGE 157

that work to those goals and are open in their structure to the generation and displayof relevant evidence, both from student to teacher and to students themselves. Actionwill be inhibited if this gap is seen as impracticably wide. Given this, the orientationby a student to his or her work can only be productive if that student comes to sharethe teacher’s vision of the subject matter.

Students do not make progress through what they can not do, but through extend-ing what they can already do to include what they can not yet do (Hurst & Lally,1992). Assessment which solely identifies what the learner can not do, as opposedto what they currently can do, assesses for failure. Such a deficit framework has noplace in quality classroom-based assessment. Rather, gap analysis should provide theopportunity to extend and enhance current knowledge and capability by establishingappropriate learning goals.

Teacher feedback is a key element to effective formative assessment and is usuallydefined in terms of information that gives the learner the opportunity to see howwell they are doing or have done (Sadler, 1989). It therefore follows that effectivefeedback must be specific and focussed on the identified learning outcomes. Sadler(1998) identifies “expertise in framing feedback statements for students” (p. 82)as one of the resources that highly competent teachers bring to the evaluative act.Quality of feedback is essential in supporting and enhancing learning. Black andWiliam (1998) report that repeated explanation of techniques that have previouslyled to failure are less effective than the use of alternative strategies.

Researchers (Mavromattis, 1997; Butler, 1987) identified that teacher feedbackfocussed on praise and promotion of social and managerial aspects of tasks drawsstudent attention away from the task and may have a negative effect on learning.In contrast, students have been shown to benefit from feedback that identifies thestrengths and weaknesses of their work, enabling them to take control of their ownlearning. The cueing effect of feedback on students cannot be underestimated. Sadler(1998) makes the point that the more complex the learning situation, the more so-phisticated the feedback needs to be. Therefore in technology education, where pro-gression in learning may be thought to consist of dealing with a greater number anda more complex array of variables, the development of sophisticated feedback skillsby teachers is critical to the enhancement of student learning.

Effective assessment is thus dependent on informed assessors who are able tointerpret observations and student outcomes. The focus is on processes, concepts andproducts. In building up a comprehensive and holistic picture of student progress intechnology several strategies will need to be adopted. Assessment strategies need tobe both focussed and detailed, and yet able to be conducted on a day to day basis aspart of normal classroom routines (Blenkin, 1992).

If assessment is to be effective, information has to be formulated with a structureand a language that reflects a shared understanding between those who are commu-nicating. This has not yet been fully developed in technology in New Zealand (Jones& Moreland, 1998). Like Black and Wiliam (1998), we believe that information hasto be adequately detailed with common criteria for grading. There also should be ashared procedure for determining achievement, and with clear and agreed documen-tation. The need is for multi-dimensional data and for those using the information


to study and understand its structure and its terminology. Assessment judgementsbecome all the more difficult to make in a new subject area such as technology,where there is a lack of a shared subculture on the nature of the subject, insufficientaccumulated practical classroom experience and a limited assessment structure.

It would appear therefore that to carry out effective teaching, learning and assess-ment strategies in the classroom a strong teacher knowledge base is required.

Teacher Knowledges

The construction of a knowledge base for teachers is pivotal for effective technol-ogy teaching and for expecting teachers to add technology teaching to the existinglearning areas that they are required to teach. Shulman (1987) strongly emphasisesthe need for teachers to build a knowledge base for teaching. He suggests that teach-ing begins with an understanding of what is to be learned and what is to be taught.His comprehensive, minimal framework includes knowledge of content, general ped-agogy, curriculum, pedagogy content, learners, educational contexts and educationalends. Pedagogical content knowledge is acknowledged as important as it identi-fies the distinctive bodies of knowledge for teaching. A blending of content andpedagogy represents how particular topics are organised for learners. For a newcurriculum area such as technology, this presents particular challenges for teachersas they search to construct a coherent, technological content base and appropriateassessment practices.

Sadler (1998) outlines six resources that competent teachers bring to assessment.These include knowledge about the content or substance of what is to be learned,attitudes towards learners and learning, skill in devising tasks, knowledge of criteriaand appropriate standards, skill and expertise in previous similar tasks, and expertisein giving appropriate, targeted feedback. Black and Wiliam (1998) also show thatthere is a very close link between teachers’ formative assessment practice, the com-ponents of a teacher’s personal pedagogy and their conception of their role. Theyhave identified that the implementation of effective formative assessment practice inclassrooms will often require significant changes both in the way teachers perceivetheir role in relation to their students and in their classroom practice.

Formative interactions with students become distorted if there is a lack of subjectknowledge and how the subject knowledge is constructed. Therefore it is arguedin this paper, that to be effective in technology teachers will need to develop threedimensions of knowledge. These dimensions are the knowledge about the nature oftechnology and technological practice, knowledge in technology, such as the techno-logical concepts and procedures, and general technological pedagogical knowledge.

Subcultural Influences

Teachers have a subjective view of the practice of teaching within their conceptsof subject areas (Goodson, 1985). This subject subculture leads to a consensus view


about the nature of the subject, the way it should be taught, the role of the teacher, andwhat might be expected of the student (Paechter, 1992). Given that teachers are inthe beginning stages of ‘being technology teachers’ and that there is little collective,coherent technology practice, the subject subcultural impact on technological peda-gogy becomes very complex. There is instead a multitude of subcultures impactingon technology in a variety of ways. Influential are teachers’ concepts of technologyand their concepts of learning and teaching, both within technology and generally.

Jones (1999) noted that the strategies developed by the teachers in their classroomswhen implementing technological activities were often positioned within that partic-ular teacher’s teaching and subject subculture. The subcultures had a direct influ-ence on the way teachers structured the lessons and developed classroom strategies.Teachers developed strategies to allow for learning outcomes which were often moreclosely related to their particular subject subculture than to technological outcomes.Teachers entering areas of uncertainty in their planned activities often reverted totheir traditional teaching and subject subculture. In addition, teachers are positionedwithin a particular social environment that comes complete with community expecta-tion of their practice. These can include views of learning environments, expectationsof artefacts to be taken home and value being placed on books full of notes. These,along with teachers’ own expectations of their students’ characteristics, ability andprior experience, impact on the classroom strategies. For example, activities in thecourse of a unit were changed so students could include project type reports, or theywere given science notes, to satisfy outside demands (Jones, 1999).

Methods

Year One

The first year of the research (1998) explored the emerging assessment practices inan emergent subject, technology. Nine teachers (2 male, 7 female) from two primary(Years 1–6) schools were involved. The teachers’ classroom experience ranged froma first year teacher to a teacher with 16 years experience. In terms of professionaldevelopment in technology, three had had minimal involvement; two moderate andfour had had extensive involvement over a whole year.

The research particularly focussed on teachers’ concepts of technology and class-room practices in technology. A case study approach was utilised to gain an under-standing of assessment practices in technology education in the classroom. The the-oretical framework within which the project was set is that technological knowledgeand assessment knowledge are socially constructed and context dependent.

The researcher took the role of a participant observer in the classroom during thetechnology education sessions and this contact involved approximately one hundredhours. Several different methods of data collection were used including classroomobservations, field notes, individual and group interviews (both teachers and stu-dents), teacher written documentation, and student work. The classroom observations


allowed for the general context of the environment to be explored as well as thevariety of influences that might be affecting student performance in the technologytasks. Individual and group interviews provided an opportunity to explore student in-teraction with the tasks throughout the process. Group interviews were valuable sincemany of the students worked collaboratively. The teacher observation and commentof student work allowed for further consideration of the context, and the teacher’sviews of their teaching and student performance. An analysis of student outcomeswas undertaken to assess student performance.

The classroom discussions between the teacher and students and the studentsthemselves were taped and analysed. Students’ written work and teachers’ writtenmaterial, including planning and assessment, were collected and analysed. All of theanalysed data were then used to write individual case studies for each of the nineteachers involved in the research. The case studies are presented in Case Studies ofClassroom Practice in Technology (Moreland & Jones, 1999).

The case studies provided an overview of the technology units taught by the re-search teachers during 1998 as well as a description of the sequence of tasks and ananalysis. Also examined were individual sessions within the technology units. Thisdetailed analysis of key sessions assisted in highlighting some prominent issues inteaching, learning and assessment in technology and employed a three-perspectivestructure in relation to the views of the key players: the teacher, the students and theresearcher.

Year Two

The second year of the research marked the beginning of the intervention phase,and was expanded to involve working in five primary schools (years 1–8) with 14teachers (3 male, 11 female). The teachers’ classroom experience now ranged froma second year teacher to a teacher with 26 years experience. In terms of technologyteacher development, it ranged from minimal involvement to extensive experience,such as a technology facilitator. In expanding the number of teachers involved, thenumber of researchers was also increased.

The increased classroom involvement was extensive involving nearly four hun-dred hours of contact time. This contact included classroom observations, individual(teacher and student) and student group interviews, and teacher observation and com-ment. Workshops were an essential part of the process with the teachers attendingseven days of workshops. These days consisted of three sets of workshops, withan initial 3 days and two day workshops spread through the year. The first threeday workshop focussed on discussions of the findings from the 1998 research andthe implications. As well, the initial framework developed by the contract team wasintroduced and trialing began. The second two-day workshop offered the teachersopportunities for reflection and enhancement of the use of the frameworks. The fi-nal two day workshop focussed on assessing the use of the frameworks and their


further enhancement. Teacher reflective comments were collected throughout theworkshops.

The First Year

Teachers were endeavouring to teach in the seven technological areas of biotech-nology, electronics and control technology, food technology, information and com-munication technology, materials technology, structures and mechanisms technol-ogy, and production and process technology. They planned their teaching so thateach unit of technology attempted to cover the three strands of the curriculum: tech-nological knowledge, technological capability, and technology and society. However,though teachers attempted to introduce tasks reflective of these strands (Moreland &Jones, 1999), the capability strand had the most weighting in their actual classroompractice where technology was taught as designing, making and testing. All teachersacknowledged they had difficulty in understanding specific technological concepts.

Though technology held a less tenuous position in terms of allocated teachingtime compared with earlier studies of technology education in New Zealand (Jones& Carr, 1992; Jones, Mather, & Carr, 1995), teacher knowledge of technology lackeddetail and teacher talk remained transient. Technology was not spoken of with easeor familiarity and continued to remain outside the parameters of usual teacher talk.

Teachers had developed reasonably consistent concepts of technology, though theyhad been involved with different levels of teacher professional development. Theydemonstrated that they had a sound understanding of primary school pedagogicalknowledge. A sophisticated, complex technological knowledge base from which toteach had yet to develop. Doubt was a constant companion. In essence, the imbalancebetween the interplay of a detailed technological knowledge base and a pedagogicalknowledge base led to the development of technology teaching touchstones. Thesetouchstones permeated technology teaching and became the underlying substructurefor teaching technology. A subculture was beginning to form related to the nature oftechnology, the way it should be taught, the role of the teacher and what might beexpected of students.

The touchstones included the following aspects: technology being about meetingneeds and opportunities; students designing, constructing and testing; students visit-ing ‘real’ technological workplaces; technologists visiting students; the expectationthat students would take risks to solve problems; the practical, hands-on involvementof students; students working in teams; and, the expectation that students would de-velop a range of creative solutions. While this was broader than the range found in thesurveys of 1992 (Jones & Carr, 1992) and 1994 (Jones, Mather, & Carr, 1995), it stillrepresented a restricted view of technology and technology education. The teacherswere able to discuss technology in a broader way than similar teachers in previousyears, but this had not translated into broader assessment practices. Technologicalprinciples and processes received minimal attention, whereas some other aspects,such as working in teams, became dominant in assessment.


The teachers were attempting to build the foundations for teaching a new subject.To do so, they relied upon much of what they did in other curriculum areas, theirprimary school teaching community of practice, their prior knowledge and experi-ence and the touchstones that had become part of the rhetoric of teaching technologyin New Zealand. Due to technologies tenuous position and its lack of a definitiveform and structure, the teachers’ relatively unsophisticated, technological conceptualframework distorted and limited their technology teaching.

Additionally, the learning outcomes that were identified were often not techno-logical learning outcomes. Technology had yet to become an integral part of thetalk of classroom teachers and the community. This meant that a shared languageof technology had not developed to any degree of specificity, which Black (1998)argued, is vital for assessment.

The following points highlight the aspects that teachers believed were crucial toteaching technology. Alongside these aspects, some of the difficulties they encoun-tered are described.

Practical subject with multiple solutions

All teachers viewed technology as a subject requiring the practical involvement ofstudents. As a consequence of this, all of their tasks included actual drawing, makingand testing components. There was an expectation that for students to be involved intechnology they would experience these aspects.

When teachers involved their students in creating artefacts, they were preparedfor a variety of solutions. They anticipated that no two artefacts students producedwould be alike. They believed that this search for divergence was reflective of cre-ativity, an aspect that they understood was inherent in the technology curriculum. Toencourage students to produce divergent solutions, they often set technology tasks sothat students negotiated aspects of them believing that this encouraged ownership ofthe task and thus enhanced student learning.

I often leave the particular design process up to the students. What is important is the divergence which iscrucial . . . . Children need to experience success with design. In all situations it does have to be satisfyingfor kids who make it or otherwise it’s incredibly frustrating . . . how far do we intervene with youngchildren and the design process.

An emphasis on creativity and praise

Though there were formative interactions at the level of specific task detail, in-teractions at the organising principle level were marginalised. Sometimes teachersdid not wish to interact with the students during the technological activity about theprocess students were undertaking, as they believed that this could stifle students’creativity. Students were expected to independently utilise their existing skills orknowledge, or acquire whatever they could from around them, to solve technological


problems. The idea that teacher intervention or guidance might hinder students’ cre-ativity was tied to ideas that open-ended, student-driven problem solving was crucialfor students to realise innovative technological solutions.

Many of the teachers were unable to identify the procedural and conceptual tech-nological learning outcomes of the tasks to any degree of detail. Therefore, interac-tions were frequently praise-based and related to task completion, rather than relatedto enhancing students’ understandings with regard to conceptual and procedural as-pects. This lack of critique of student understanding meant that detailed guidancefor ongoing work was minimal. When all attempts were accepted without discrim-ination, students were not required to reflect on their work as to whether they hadmet the objectives of the task, or how well the objectives had been met, or what theymight do next. The opportunities to build on relevant conceptual, procedural andsocietal technological aspects were lost.

Task Selection

The selection of technology tasks for the students was based on ensuring thatstudents were involved with experiences in a number of different technological areas.Task selection was related to achieving a balance across the technological areas,rather than developing progression in student technological understanding. Thoughteachers were able to organise tasks reflective of technology, they were somewhatisolated experiences rather than cumulative and purposeful. A teacher articulatedthus:

I can’t see progress in technology. I don’t know what to look for the same . . . . It is so much harder. Iwould hope that the methods I am using are the right ones for technology . . . . I’m sort of trialing thingsthat are right for me, but do they mean anything? So it is difficult, difficult to know what is right.

As teachers’ concepts in relation to technological knowledge, procedural knowl-edge and technical skills and knowledge were not detailed, progression, in terms ofincreasing the complexity and number of variables that students might consider in atechnology task, was not considered. Teachers developing assessment procedures intechnology were influenced by the current school focus, including the emphasis onsummative outcomes. Teachers commented that the school-wide assessment policies(school requirements for assessment) influenced what they assessed in technology.

A Focus on Social and Managerial Aspects

A lack of a detailed understanding of technological practice, including conceptsin technology, meant that the teachers focussed on aspects other than technologicalprocedures and concepts when assessing. They tended to focus on social and man-agerial aspects and these were detached from the technological process, for example:“Did they enjoy it?” or “Working in-groups, turn taking, sharing.”


Intensifying this emphasis was the strong subcultural influence in primary schoolsof expected ways of working. The classroom climate was one where students wereencouraged to work in collaborative as well as independent ways. The result wasthat many formative interactions with students were focussed on these aspects atthe expense of technological aspects. Students often became confused as to the mainpurpose of the task, believing that social and managerial aspects were more importantthan technological aspects. For instance, students thought it more important to drawco-operatively with their partner than to create a drawing that would show their initialproduct ideas.

Also impacting on teachers’ assessment practices in technology were the existingsubcultures in schools, from other experiences, subject expertise and school widepolicies. What teachers relied on for assessing in technology therefore became de-pendent on what they already knew and did in other curriculum areas. All teachers inprimary schools have common understandings of teamwork, leadership, turn-taking,discussing, depicting ideas, gathering information, describing, reflecting, etc., andthese common understandings of social and managerial skills became the focus ofassessment in technology. Therefore in terms of the technology curriculum, teach-ers focussed on aspects of the achievement objectives that aligned with social andmanagerial aspects.

Thematic Approaches to Teaching Technology and Difficulties with Transfer

Most teachers used a thematic approach to their technology teaching wherebythey carried a unifying theme through from another curriculum area to technology,such as people with disabilities. Their reasoning was that the students would be ableto transfer knowledge and understanding gained in one situation to the next. Everyteacher underestimated the difficulties associated with students accomplishing this.

Teachers had assumed that the conceptual knowledge acquired by the studentsin other parts of the curriculum would be transferred during the technology task.Leaving the problem of transfer up to the learner became problematic. Using knowl-edge from one domain to another is not easily achieved. For example, teaching aboutnets in mathematics does not automatically mean that students will independentlyapply nets to packaging problems in technology. Also, some teachers assumed thatwithin the technology task students would effortlessly transfer knowledge articulatedin former technology sessions to later technology sessions. However, they did notprovide the means for this to happen. Though the teachers organised the task sessionsin an order that could enhance learning, often conditions for transfer were not takeninto account and so opportunities for the students to make links were lost.

The Technological Task as the Learning Outcome

Teachers were able to identify suitable technological tasks for their students buthad difficulty identifying suitable technological learning outcomes. An aspect they


focussed on as the learning outcome was the task definition. With the focus on thetask it became almost impossible for teachers to provide detailed feedback to enhancestudent performance at the conceptual and procedural level. Whether students hadcompleted the task was the assessment made. This resulted in teachers having diffi-culties making statements about student learning that was useful for future teachingand learning. Absent were detailed concepts of technics (techniques associated withpractice), technological praxis (how the knowledge base is used) and technologicalknowledge. Assessing at the level of organising principles was not apparent.

Summative Assessment

All teachers attempted to make appropriate summative assessments in technology.However, they indicated that it was difficult to make an overall judgement of studentperformance when assessing in technology.

I wondered if my assessment was right. Like I wondered if I should have been assessing them all the waythrough the process. They did so much work in the pet house unit and I just evaluated them on the finalfeeding machine . . . maybe I should have done some more work . . . what they added and what they didn’tadd and how they changed their original ideas.

Assessment difficulties were highlighted in interviews, when teachers were askedto compare how they assessed reading with how they assessed technology. Nearly allteachers commented that it was easier to assess in reading. They believed that thiswas because they taught reading every day, they used established tests, they knew alot more about reading, they knew where their students were going in reading andthe schools had strong, school-wide practices in place. In technology, all they hadwas the curriculum document that they believed did not give them enough guidanceand detail for assessment. Additionally, they had little teacher experience in teachingtechnology. They were not sure what the goals were in technology and had no senseof technological progression throughout the school. Consequently they did not assesswith any degree of confidence. For example:

I think that assessment in reading is probably easier because teachers are much more confident withthe subject, there are very clear benchmarks in assessing reading . . . . You can’t really go wrong. Withtechnology, there is nothing. Absolutely nothing, so it’s really going on probably teaching background,background knowledge of things, I think. Your confidence, your understanding of the document, expe-rience with using the document. I don’t know, but on the day it is down to your gut feeling and ownprofessional teaching.

Like formative assessment, teachers’ summative assessment reflected a bias to-wards social and managerial aspects. Teacher comments describing student progresswere largely confined to these social and managerial aspects. Additionally, teachersviewed each achievement objective in the curriculum as one item of assessmentfor summative purposes. These broad objectives in the New Zealand Curriculumrepresent approximately two years of learning. As student performance was assessed


Task definition: Tech Area/s:

Overall dimensions of technology:

Conceptual Procedural Societal Technical

Learning Outcomes Learning Outcomes Learning Outcomes Learning Outcomes

Figure 1: Planning for learning and assessment in technology.

summatively against these, no detailed analysis of student performance could be ob-tained. Little sense of progression was indicated. These broad statements meant thatinformation about student performance in technology was not useful to the teacher,to other teachers in subsequent years, or indeed the students. In essence it meant thatthe statements about each student’s learning were ineffective.

The Second Year: Intervention Year

From this background, it was decided to focus the intervention on moving theteachers away from thinking about technology as a series of tasks and defined solelyby the broad curriculum achievement objectives. The intervention centred on de-veloping planning strategies that compelled teachers to articulate intended learningoutcomes in concise technological terms. The development of a planning format wasdirectly related to our 1998 research (Moreland & Jones, 2000), and teachers’ identi-fication of the importance of developing a more detailed, complex and sophisticatedknowledge base in the different technological areas. The construction of a knowledgebase is pivotal for effective technology teaching. What we endeavoured to do was toprovide the means for teachers to begin to develop a mental framework for makingdecisions about what needs to be included when teaching technology.

By undergoing a process of articulating concise intended learning outcomes, theteachers were able to deduce what they personally needed to know to teach technol-ogy, as well as providing a guide for clearly defining what they wanted the studentsto learn. Technology education is concerned with complex and interrelated problemsthat involve multiple conceptual, procedural, societal and technical variables (Jones,1999). The research team therefore devised a planning format (see Figure 1) thathad the following features: specific task definition, overall dimensions of technology(e.g., knowledge, capability and societal) and specific learning outcomes in terms oftechnological concepts, procedures, societal aspects, and technical skills.

The planning format was based on a framework that included five domains. Thefive domains being:1. conceptual – knowledge and understanding of relevant technological concepts

and procedures;


2. procedural – knowing how to do something, what to do and when to do it;3. societal – aspects related to the interrelationship between technology and groups

of people;4. technical – skills related to manual/practical techniques; and,5. the operationalisation of the conceptual, procedural, societal and technical as-

pects in student technological practice – integrating all four domains in under-taking and completing the technology task.

With students working across the five domains the development of student tech-nological literacy is enhanced. All five domains are critical in assisting teachers toprovide a comprehensive and balanced approach to teaching technology. Comple-tion of the planning format requires an iteration process between task definition,dimensions of technology and specific conceptual, procedural and societal aspects intechnology.

The change strategy used by the researchers with the teachers was that of negoti-ated intervention (Jones & Simon, 1991). This was used to gradually move teachersfrom existing practice to a new negotiated position. The intervention program wasdesigned to provide intensive workshops followed by opportunities for classroompractice with the research team providing feedback on the teachers’ developing class-room practice. Each workshop and intervening practice in the classroom was usedas a stepping-stone to the next phase. This provided the teachers with the oppor-tunities to build stronger foundations for enhancing their teaching of technology.On-going facilitation provided vital support during the workshops and once teachershad returned to their classrooms.

Workshops

The first three-day workshop held early in the year focused upon earlier researchfindings and implications for classroom practice, moving from task description to aconceptual and procedural focus, and enhancing formative and summative interac-tions. The second intervention workshop held in the middle of the school year hadan overall focus on the enhancement of the strategies developed in phase one. Thefeatures of the intervention were as follows: increased precision in thinking aboutprogression; stronger links between detailed learning outcomes and technologicalpractice; overall judgements more reflective of the dimensions of technology; and,the iteration between task definition, dimensions of technology and specific concep-tual, procedural and societal aspects in technology. The third and final workshop atthe end of school year had a focus on teachers sharing and evaluating their classroomteaching programs and student performance. As well, they attempted to identifythe key elements of guidance for other teachers in planning, teaching and assessingtechnology in primary school classrooms.


Work in Classroom

In working in classrooms the research team needed to continually stress to theteachers the need to identify and focus on the articulation of specific learning out-comes. The teachers required continuing advice and direction related to this identifi-cation and articulation. The research team provided support material, demonstrationof suitable learning outcomes and clear instruction about the desired outcomes. Ap-propriate learning outcomes were made known to the teachers on an ongoing basis.This continual insistence by the research team on the identification of learning out-comes, in both workshops and the classroom, compelled the teachers to move awayfrom thinking about technology just in terms of suitable activities for their students.

The struggle teachers demonstrated meant that support strategies became a crucialfeature of the research and development. One-to-one, ongoing support in the class-room and the collaborative atmosphere built during the workshops were important.When the teachers’ foundations on which they had come to rely for teaching tech-nology were shaken, when doubt became a companion once again and when feelingsof uncertainty surfaced, it was crucial that the research team was understanding,supportive and appreciative of the efforts that the teachers were making. Also ofsignificance was the support that the teachers gave each other, during both workshopsessions and in schools. The teachers’ commitment to the research was a key factor,as was their unfaltering enthusiasm for the project.

Results from Intervention Year

As discussed in the preceding section, the conceptual and procedural aspects oflearning in technology were highlighted as the means to enhance the formative in-teractions of the teachers and the learning outcomes for the students. This resultedin teachers moving from using general concepts about technology to more specificconcepts within different technological areas. For the first time teachers were ableto identify the specific technological learning outcomes they wished to assess. Theteachers’ developing conceptual and procedural knowledge enabled them to writespecific learning outcomes, and they began to move with more confidence betweenthe global dimensions of the nature of technology and the specific technologicallearning outcomes.

Task selection

The teachers were able to choose more suitable tasks that had the potential todevelop student learning in technology. This shift in focus from providing a tech-nology experience to providing opportunities for students to develop technologicallearning outcomes was significant. By investigating a wide range of learning out-come possibilities and then selecting particular learning outcomes teachers pursueda more appropriate approach to technological learning. They became focussed on the


technological learning of their students. Teachers were also increasingly cognisant ofunexpected and negotiated learning outcomes and were better prepared to allow stu-dents to pursue such outcomes. Teacher talk about technology education had a higherprofile and was increasingly embedded in teacher conversations. Teacher talk alsodeveloped related to progression aspects, with aspects about linking and enhancingtechnological learning outcomes from one unit to the next being discussed.

Conditions for transfer

Transfer of learning from one subject to another was no longer taken for granted.The conditions required for successful transfer to occur were more likely to be takeninto account in that cueing by teachers was more obvious, questions were more fo-cussed and situations were planned for. Concerns for students’ cognitive engagementand in-depth understanding became increasingly important. Where teachers had ear-lier withdrawn from the learning process for fear of closing down students’ creativity,there was now an awareness of the need to provide sufficient guidance for sustainedlearning and understanding to occur.

Enhanced formative assessment

Some difficulties with assessment in technology continued but teachers demon-strated greater confidence with formative assessment, particularly in relation to pro-viding appropriate technology feedback to the learners. Considered direction wasgiven where deemed appropriate, which led to more considered and purposeful in-teractions. Not only was there more emphasis on providing feedback and assistanceto students to develop particular technical skills, there was also more emphasis onconceptual and procedural aspects rather than social and managerial aspects. Ad-ditionally, there was less emphasis on praise as the sole formative interaction andmore emphasis on assisting students to move on, to reflect, and to assess their ownprogress.

Use of the framework

The framework that was developed in the project had a key role in enhancing theteachers’ planning and classroom strategies. An example of ways in which the formatenhanced the detailed planning of the teachers is shown in Table 1.

In 1998, this teacher defined the learning outcomes in terms of the broad curricu-lum level objectives. He did not attempt to define specific technological conceptualand procedural knowledge, nor did he try to develop any sense of progression instudent learning. His prime concern was to provide students with the opportunities toexperience different technological areas rather than to develop any degree of sophis-tication or complexity in technology. The difference between 1998 and these 1999learning outcomes is the degree of specificity. He no longer described technology interms of an activity, rather in terms of technological learning.


Table 1A Comparison of 1998 Planning with 1999 Planning.

1998: Time measuringdevices

1999: Hand-held home technology for ArthriticsufferersKey aspects of technological practice:• Investigate and describe the use and operation

of home technologies by sufferers of arthritisfocussing on ergonomics;

• Explore and modify existing technical aids forarthritic sufferers through drawing;

• Identify the positive and negative effects ofexisting, modified and adapted technologiesfor arthritic sufferers

Technological knowledge:• Investigate and

describe the use andoperation of timemeasuring devices

Conceptual:• Explore and understand what is technology• Understand that interview schedules, surveys

and open questioning are methods ofcollecting information

• Understand ergonomics-the need for theproduct to fit the user

• Understand the use of and drawing ofexploded diagrams

• Understand that modified, adapted andfail-safeare technical terms

Technological capability:• Prepare a plan of

action for creatingspecific timemeasuring devices

• Collect resourcesand produce theselected designto meet the specifiedcriteria

• Test, adapt andreflect on timedevices produced

Procedural:• Conduct an interview to find out about

problems people face with arthritis• Complete a needs analysis• Define the target group needs• Negotiate the criteria for designs• Explore existing modifications to technologies

employed• Complete a materials analysis• Draw 2D, 3D and exploded drawings of

existingand modified technologies forsufferers


Table 1(Continued.)

Technology and society:• Identify and discuss

the importance andimpacts of time inour lives

Societal:• Cost, the user and materials will affect technical

developments• The traits of a group will affect the ergonomics

of a product

Technical:• Orthographic drawing• 3D drawings• Interviews – develop open-ended questions

He moved from thinking of technology as broadly described by the achievementobjectives to thinking about technology in detailed, conceptual, procedural, societaland technical terms. Additionally, he began to bring this detail together to form anoverall statement. Together these provided a base for his teaching, enhancing bothhis student interactions and his assessment practices. He commented that:

The whole process has become more refined. I knew that when I was planning, I knew exactly what Iwas going to assess and how I was going to assess it, so I was very specific . . . . The planning modelshelped me to have a better look at exactly what technology is being taught and what the technology is indifferent activities. The activities that I am giving my children are more clearly targeted and identified. Mychildren have a fair idea of where they are going to go . . . . But they have still gone in a thousand differentdirections and they’ve really enjoyed it. You can start to see the divergence coming out in children andthat’s neat too.

The gains in planning was also reflected in other teacher comments, for exam-ple, “Very focussed in terms of planning and technological learning outcomes” and“Formative interactions were more purposeful and productive which resulted in chil-dren being more focussed and tuned into the technology involved in the learningactivities.”

The teachers valued the following intervention strategies: identifying specific andoverall learning outcomes rather than just activities; identifying procedural, con-ceptual, societal and technical learning outcomes; summative assessment during theunit as well as at the end; questioning using technological vocabulary and concepts;an iterative use of the framework; and, allowing for multiple outcomes. These areillustrated in some of the teachers’ comments below:

Thinking about the learning I wanted to take place enabled very focussed activities to meet the learningoutcomes.

Dividing planning into conceptual, procedural, societal and technical allowed me to more effectively honein on the technology involved.


To focus on specific learning outcomes first then plan activities to match allow me to better cater for theneeds of students.

The identification of possible and planned learning outcomes made me more aware of the questioning thatwould be required.

The planning and on-going formative and summative assessments allowed for accommodation of negoti-ated outcomes.

Also increased were more appropriate pedagogical approaches. A variety of meth-ods were employed by the teachers including student interviewing, conferencing, ob-servation, use of considered portfolios and analysis of appropriate learning outcomes.There was continued encouragement for students to seek divergent solutions.

Evident was the development of initial teacher understanding of progression instudent learning in technology. This was reflected in task selection and development.Tasks were identified to develop particular technological conceptual and proceduralaspects rather than just providing a variety of experiences in different technologicalareas. The use of the frameworks also enabled the teachers to differentiate betweenthe different levels of effectiveness of student learning and to justify the differen-tiation. The teachers also noticed enhanced student learning in technology. Theircomments were illustrative of this:

Children’s differences in learning can be better identified with specific learning outcomes with moreeffective children coping with more variables.

Have had quality opportunities to show what they can do with improved vocabulary, language and skills.

The more effective children were engaged all of the time, they had the vocabulary and could use itappropriately. This was evidenced in their mock up and drawing.

Summative assessment

When teachers assessed summatively they placed less emphasis on social andmanagerial aspects, as technological learning outcomes were identified. A furthershift was identified in the timing and scope of summative assessment. Attempts weremade to assess summatively at different points in the learning process. Negotiatedand unexpected learning outcomes were taken into account alongside planned sum-mative assessment. Attempts were made by the teachers to make overall judgementsabout the effectiveness of students’ learning and these judgements were more ef-fective in assisting teachers to make appropriate decisions to guide students’ futurelearning.


Impact on other curriculum areas

The teaching, learning and assessment strategies that have been developed in thisintervention year also impacted on the teaching and learning in other curriculumareas. All teachers made comment on this, for example:

I am looking at making my learning outcomes very focused for other curriculum areas to develop morepurposeful and structured formative and summative assessment practices. I am thinking more carefullyabout what I want the children to learn.

The planning framework has helped the curriculum documents fall into place.

I am now probing, in-depth questioning and constantly challenging. I am thinking about how my learningactivities link and how I can help children transfer ideas and skills.

This research project has developed intervention strategies that encourage teachersto identify the conceptual, procedural, societal and technical aspects, task definitionand aspects of holistic assessment. The results are very encouraging with the focusat the conceptual and procedural level rather than in terms of an activity. Many of theteachers commented that the intervention had a direct influence on other subjects,especially with their planning and formative interactions. They have moved fromthinking about progression in terms of a series of activities to examining the concep-tual and procedural aspects of student learning. The focus on more precise formativeinteractions has enhanced student learning.

General comments

General comments made by the teachers at the conclusion of the year includedsome of the following.

My technology teaching has made huge leaps forward because of my involvement. It has been verydemanding but the risks have been worthwhile.

The framework has helped immensely and it has been particularly rewarding to see the quality of workthat is being produced by the children as a result of the research.

Intervention strategies based on the frameworks produced accelerated learning.

Conclusion

To enhance and sustain learning in technology there needs to be a focus on teacherknowledge of specific and detailed technological learning outcomes in conjunctionwith appropriate pedagogical approaches to realise these. In order to enhance the


teachers’ knowledge base, a well-developed framework was used to focus the teach-ers’ attention on the conceptual, procedural, societal and technical aspects of studentlearning in technology. Overall aspects of the dimensions of technology were alsoencouraged, as was the iterative process between task definition, learning outcomesand the dimensions of technology. In conclusion, the framework that was developed,coupled with the intervention by the research team, had a major impact on improvingteachers’ formative interactions and understanding of summative outcomes. As aconsequence student learning has been significantly enhanced in technology. Therehave been spin-offs into other curriculum areas such as science, with teachers com-menting on a renewed emphasis on learning outcomes. However, this is only thebeginning of this process and more work is required to develop sustaining classroompractice in technology consistent with the New Zealand technology curriculum.

Acknowledgment

We are appreciative of the teachers’ involvement and the challenges they under-took in their classrooms. We are grateful to Megan Chambers for her comments onan earlier draft of this paper. The Ministry of Education, New Zealand, funded theintervention research program.

Correspondence: Alister Jones, CSTER, University of Waikato, PO Box 3105,Hamilton, New ZealandE-mail: [email protected]

References

Bell, B., & Cowie, B. (1997). Formative assessment and science education. ResearchReport, Learning in Science Project (Assessment). Hamilton, New Zealand: Cen-tre for Science Mathematics and Technology Education Research, University ofWaikato.

Black, P. J. (1998). Testing: Friend or foe? Theory and practice of assessment andtesting. London: The Falmer Press.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment inEducation, 5(1), 7–74.

Blenkin, G. (1992). Progression, observation and assessment in early education: Thecontext. In G. V. Blenkin, & A. V. Kelly (Eds.), Assessment in early childhoodeducation (pp. 24–45). London: Paul Chapman Publishing.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture oflearning. Educational Researcher, 18(1), 32–42.

Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Ef-fects of different feedback conditions on motivational perceptions, interest andperformance. Journal of Educational Psychology, 79(4), 474–482.


Crooks, T. J. (1988). The impact of classroom evaluation practices on students.Review of Educational Research, 58(4), 438–481.

Gipps, C. (1994). Beyond testing: Towards a theory of educational assessment.London: Falmer Press.

Gipps, C. (1999). Socio-cultural aspects of assessment. Review of Research inEducation, 24, 355–392.

Goodson, I. F. (1985). Subjects for study. In I. F. Goodson (Ed.), Social histories ofthe secondary curriculum. London, Falmer Press.

Hattie, J., & Jaeger, R. (1998). Assessment and classroom learning: A deductiveapproach. Assessment in Education, 5(1), 111–122.

Hurst, V., & Lally, M. (1992). Assessment and the nursery curriculum. In G.V. Blenkin, & A. V. Kelly (Eds.), Assessment in early childhood education(pp. 46–68). London: Paul Chapman Publishing.

Jones, A. (1999). Teachers’ subject subcultures and curriculum innovation: the exam-ple of technology education. In J. Loughran (Ed.), Research teaching: Method-ologies and practices for understanding pedagogy (pp. 155–171). London:Falmer Press.

Jones, A., & Carr, M. (1992). Teachers’ perceptions of technology education:Implications for curriculum innovation. Research in Science Education, 22,230–239.

Jones, A. T., Mather, V., & Carr, M. D. (1995). Issues in the practice of technol-ogy education. Hamilton, NZ: Centre for Science and Mathematics EducationResearch, University of Waikato.

Jones, A., & Moreland, J. (1998). Assessment in technology education: Developinga framework for research (Working Paper 522, Research in Assessment of Pri-mary Technology Project). Hamilton, NZ: Centre for Science, Mathematics andTechnology Education Research, University of Waikato.

Jones, A., & Simon, S. (1991). Strategies for educational change: The work of theOPENS project in the context of a new national curriculum for science. Auckland,NZ: Longman Paul (SAMEpapers 1991).

Mavromattis, Y. (1997). Understanding assessment in the classroom: Phases of theassessment process – the assessment episode. Assessment in Education, 4(3),381–399.

Ministry of Education. (1993). The New Zealand curriculum framework. Wellington,NZ: Learning Media.

Ministry of Education. (1994). Assessment: Policy to practice. Wellington, NZ:Learning Media.

Ministry of Education. (1995). Technology in the New Zealand curriculum. Welling-ton, NZ: Learning Media.

Moreland, J. P. (1998). Technology education teacher development: The importanceof experiences in technological practice. Unpublished M.Ed Thesis, Universityof Waikato, Hamilton, New Zealand.

Moreland, J., & Jones, A. (1999). Case studies of classroom practice in technology(Working Paper 523, Research in Assessment of Primary Technology Project).


Hamilton, NZ: Centre for Science, Mathematics and Technology EducationResearch, University of Waikato.

Moreland, J., & Jones, A. (2000). Emerging assessment practices in an emergentcurriculum: Implications for technology. International Journal of Technology andDesign Education, 10, 283–305.

Northover, A. B. (1997). Teacher development in biotechnology: Teachers’ percep-tions and practice. Unpublished M.Ed Thesis, University of Waikato, Hamilton,New Zealand.

Paechter, C. (1992). Subject subcultures and the negotiation of open work: Conflictand co-operation in cross-curricular coursework. In R. McCormick, P. Mur-phy, & M. Harrison (Eds.), Teaching and learning technology (pp. 279–288).Wokingham, UK: Addison-Wesley, Open University Press.

Sadler, D. R. (1989). Formative assessment and the design of instructional systems.Instructional Science, 18, 119–144.

Sadler, D. R. (1998). Formative Assessment: Revisiting the territory. Assessment inEducation, 5(1), 77–84.

Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform.Harvard Educational Review, 57(1), 1–22.

Enhancing Teachers' Technological Knowledge and Assessment Practices to Enhance Student Learning in Technology: A Two-year Classroom Study

Documents