The social network of peer appraisal in an undergraduate design studio

The social network of peer appraisal in an

undergraduate design studio

Sian Joel

Napier University, Edinburgh, Scotland, EH10 5DT

[email protected]

AbstractAppraisal is a key feature within the design process as it allows a designer to refine and

progress their work. This paper seeks to understand an undergraduate peer appraisal

network from a final year design course. This paper considers go-between, vulnerable and

core-member students and the implications that these roles have upon overall grade. It is

hypothesised that go-between students are in a better position to gain appraisal from a broad

range of student designers and are continually improving their work. In contrast vulnerable

students are only refining their work with a limited number of their peers, whilst core-members

are held within a set grade threshold. It is also proposed that go-between students are in a

very influential position to impart their own subjective opinion about “good” design work to an

entire design studio, which has important connotations on the success of the course as a

whole.

IntroductionThe design studio has often been the focus of research into design education (Argyris and

Schön, 1974; Porter and Kilbridge, 1981; Shih et al., 2006; Oxman 2004). The design studio

is very social in nature and it is argued that its social atmosphere is integral to the design

process, as designers communicate ideas, appraisal and reflection between each other

(Ashton and Durling 2000).

The concept of appraisal and peer review is widely recognised within the design community

(Craig, 2000, Teasley, 1997, and Bruckman, 1998). Appraisal specifically, can be understood

as the sharing of communication to evaluate design work, and can be seen as social

reflection. Throughout the design process, appraisal is a phenomenon that is perceived as

being integral to the development of design work, from both an educational and professional

standpoint (Boyer and Mitgang, 1996; Goldschmidt, 2002: Schön, 1984). For more

experienced designers, appraisal allows the designer to constructively give their opinion in a

formal critiquing process (Mirochnik, 2000). Similarly in an educational setting, formal

critiquing sessions are timetabled for the duration of most projects (Uluoğlu, 2000). Informal

peer review, on the other hand, is more serendipitous in nature. It is less structured, for

instance, and as such the times when people seek appraisal may occur at varying points in

the design process. Informal appraisal allows the designer to reflect on their own work with

their peers and by reflecting on the work of course mates, peer based learning can be

encouraged. A consideration that is particularly pertinent in light of continuing emphasis within

higher education for a student-based paradigm (Trigwell, 2001). Recent pedagogical literature

has encouraged educators to facilitate student peer appraisal in order to reflect on each

other’s design experiences as well as interpreting the social dynamic of their work (Nicol and

Pilling, 2000). Issues have further been examined which look at how students react to the

review process (Fredrickson, 1990), and importantly how design students approach learning

(Davies and Reid, 2001).

The importance of appraisal within the design process can be understood by looking at why it

is sought. Dong (2006), analysed student peer appraisal via blogs, and categorised the

appraisal exhibited in three ways: rational decision, kinship support, and muse. From this it

can be seen that people seek appraisal because it fulfils a certain need. For example,

appraisal can give a designer emotional support for their work, which is particularly useful for

novice designers. Similarly, Ashton and Durling (2000) proposed that students needed to

know whether they were “doing the right thing”. They categorised the concept of “doing the

right thing”, as students fulfilling creative uncertainty by referring to past experience and

learning, assessing user needs and comparing their work socially. Ashton and Durling (2000)

maintained that student designers sought appraisal to ascertain whether they were doing their

work correctly, and following the right process to produce results of an adequate standard.

Ashton and Durling’s (2000) work shows the importance of appraisal on the student and

novice designer and how appraisal can overcome insecurities about design ideas. They

explored the impact that peer assessment had upon the student, and suggested that if design

students did not operate in the social setting of the studio this assessment would be lost. To

some extent their findings are re-examined in this paper. They proposed that isolated and

vulnerable students are at a considerable disadvantage when producing design work and this

question is again re-evaluated. This paper also seeks to understand the impact of go-between

students and core-member students. To do this two types of appraisal networks are

considered, namely the seeking appraisal network and the sought appraisal network.

In addition to both appraisal networks other networks are compared such as communication

within the studio. Similarly the research has looked at the wider appraisal network, as

students have a wide range of people that they rely upon to support them during their design

course. This includes their peers, their tutors, visiting lecturers, and friends outside of

university, amongst others. Although there is some reference and comparison to these other

networks, this paper focuses on the social relationships between students in their final year

design course. This paper discusses the appraisal network as a whole, but in particular seeks

to understand the following four areas:

That network vulnerable students produce poorer design work (lower grades)

Go between students produce better design work (higher grades)

Reciprocated appraisal has no bearing on design work

Core network members produce average design work (average grades)

Data and MethodIn June 2006, a group of final year undergraduate students from the Design Futures course at

Napier University, Scotland were asked to complete a questionnaire that would provide social

network data. This same group had been studied the previous year using ethnographically

oriented techniques (Joel et al., 2005). The student group consisted of 20 individuals, who

ranged in age from 21 to 25. Of these, 7 were male and 13 female, with 3 students born

outside Scotland and 17 born within Scotland.

This paper concentrates on the findings elicited from the survey data however it is worth

referring to the previous qualitative research as many propositions and assumptions made in

this paper are based on field notes ascertained in the Spring of 2005. During that time, a six

week study was carried out which spanned the duration of a design project. That project was

to create a hypothetical multi-media installation for the new Wembley Stadium Museum as

part of a design competition. Field notes and video was taken during those six weeks and

much of the anecdotal evidence referenced in this paper is based on that research.

The questionnaire specifically was used to explore the ways in which design students gave

and received appraisal about their work, and in particular to whom. The questionnaire

purposely enquired about appraisal as a two-way phenomenon, where people give and

receive appraisal. Although seemingly similar, each direction has a subtle difference. Seeking

appraisal refers to whom you would want to review your work, whilst sought appraisal refers

to who would want you to review his or her work. In practice this means that the sharing of

appraisal is a two-way phenomenon with a network associated with each and this can enable

comparisons to be made. Theoretically the results from both questions should match. For

example, if person A seeks appraisal from person B, person B should answer that person A

has been sought for appraisal by person B. Any difference to this model would suggest that

the perceptions of students vary between each other, which can be an interesting insight in

itself. This student perception can therefore bias the resulting data. This can be rectified, to

some extent, by looking at each direction of appraisal, or comparing and taking a mean

average of the two. This can ultimately make the result more reliable than looking at one

direction alone

The introductory questions to the survey asked very general open-ended questions

concerning who they received appraisal from, and whose work they looked at outside the

course. Individuals were then asked about the people from the course of 20. They were asked

a series of questions about whom they sought for appraisal and who had in turn sought

appraisal from them. The students were asked further questions such as whom in the class

they generally spoke to and whom they shared information with. Each individual had to rate

their course mates from 0 to 5 for each question, with 5 being the closest relationship. This

enabled comparisons to be made between the appraisal network and other networks such as

general communication. The basis for the questionnaire was a sample SNA questionnaire

(Cross, 2004), with some slight modifications to include specific questions about appraisal.

Before visually and statistically analysing the data, certain steps were taken which would

enable certain techniques to be applied. Firstly the data was dichotomised, where it was split

to show high rated students (higher or equal to 4) and low rated students (less than or equal

to 2). Figures 1 and 2 are dichotomised data from the network seek and sought appraisal and

show high scoring relationships between people. In the discussion of seeking appraisal, it was

felt that using high scores was a more indicative network as all students in the class knew one

another and as a result would give at least a value of 1 when rating each person. The

dichotomised data also enables comparison to be made between the high and low rated

people and networks.

In order to ascertain the ability of each student’s design work, grade is used as an attribute

and in particular degree classification. The U.K degree award is separated into categories,

namely 1st (the highest degree award), 2:1, 2:2 and then 3rd class degree (being the lowest

degree awarded with honours). The design course discussed in this paper had no students

fail or passed without honours. A consequence, perhaps, of all weaker students being unable

to progress to the final year because of poor results in previous years. Within this paper, any

reference to high grades or low grades refers to 1st class and 2;1 being a high grade and 2:2

and 3rd class degree being a low grade.

Description of the network data as a wholeA lot can be ascertained from simply visualising the network data and patterns can be clearly

seen in Figures 1 and 2. Go-between individuals (e.g. Frank, Gayle and Jane – highlighted by

the square area) are visible and potentially isolated individuals (e.g. Maggie and James)

identified. Furthermore individuals who have un-reciprocated responses can be seen by the

arrows only going in one direction (this is the case with Hannah who is in the triangle in Figure

2). From the dichotomised network, it can be seen from Figures 1 and 2 that there is a group

of seven individuals who have rated each other as people who they seek for feedback most

often. This group is circled and comprises of Cara, Kim, Natalie, Laura, Peter, Catherine, and

Nick. It should be noted that all names have, of course, been changed for anonymity.

The sought feedback network (Figure 2) is subtly different to the seek feedback network

(Figure 1). Again, Jane (the squared area) is identified as a go-between individual. However

there is a new group of 7 circled (i.e. Frank, Cara, Kim, Natalie, Laura, Peter, and Catherine)

that exists. This group of 7 students are more distributed than those in Figure 2 and their ties

are not as strong. Interestingly, although there are 7 people in the “clique” for both seek and

sought feedback, there are slight differences in the people who constitute the group of seven.

In seek feedback, for example, Frank, is a go-between person, whereas in the sought

feedback network, he is within the clique of 7 (Figure 2). This may indicate that who students

think they seek for feedback has a central strong clustering of people. However, who people

think have sought feedback from them, is not as concentrated and there are less go-between

people. It is possible that people think that they ask those who are close to them to appraise

their work but are less confident about which students have approached them previously. The

distinction between seek and sought feedback is quite subtle and it can be seen from Figures

1 and 2 that the two question responses are still quite similar.

Figure 1: Seeking feedback network: HIGHLY RATED people

Figure 2: Sought feedback: HIGHLY RATED people

From the resulting appraisal network, the concept of density is used to compare the highly

rated network for seek and sought appraisal to the low rated seek and sought networks

(Table 1). For both appraisal directions, the lower rated networks are much denser than their

high rated counterparts. Indeed the density figure for seeking appraisal ≥4, is 0.1842 whilst

the density for seeking appraisal ≤2, is 0.6053. Student designers therefore had a greater

tendency to rate their peers with low marks and this could imply that the students generally

preferred not to ask their peers to reflect on their work. Figure 3 gives a comparison of

densities to other questions, and this pattern was seen for all networks that could be deemed

work related (such as information sharing). However, in the general communication network,

this was not the case. It could be inferred from this that the students in question were quite

happy to chat and socialise with a wide variety and number of people, but when it came down

to their design work, the students were much more selective about who they spoke to. This

was indeed an observed phenomenon as the group was quite negative about each other’s

work and were also quite selective about who they spoke to in order to reflect on their designs

(Joel et al., 2005). A phenomenon that has also been observed in other design courses

(Rodgers, 2005).

Seeking appraisal

network: LOW

RATED people

Seeking appraisal

network: HIGHLY

RATED people

Sought appraisal

network: LOW

RATED people

Sought appraisal:

HIGHLY RATED

people

Table 1: Densities of high rated and low rated appraisal networks

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

GeneralCommunication

Information Seek Feedback Sought Feedback

Network

Den

sity ≥4

≤2

Figure 1: Comparison of network densities

The appraisal network data as a whole can also be cross-referenced with the physical

position of students in the studio. In Figure 4, the nodes are re-positioned to reflect a plan

view of where individuals’ desks were actually positioned. Figure 4 reveals that the circled

grouping in Figure 2 is also apparent in the physical space. It is obvious from Figure 4 that

the group sits physically close together in the studio. This phenomenon was also displayed

with the sought appraisal network. It cannot be inferred from this however, that people simply

seek appraisal from those who sit close to them. This is because the individuals chose the

seating arrangement in the studio. The students, through being in their final undergraduate

year, knew all classmates and had built up friendships within the class. Therefore an

assumption can more likely be made that people would seek appraisal from their friends and

friends were positioned physically close together. Some individuals are exceptions to this

however. Hannah, for example, sits within the core group of students but she is not a core

member. Similarly Jane, sits at the extreme corner of the room and yet is sought for appraisal

by course peers on the other side of the room. These exceptions may imply that physical

position doesn’t automatically influence the choice of person who is sought for appraisal,

although in the majority this was the case.

Figure 4: The studio layout

Knowing that the highly rated appraisal networks are not particularly dense, geodesic

distance is used to indicate the levels of connectivity within the group. It is assumed that if

there are a great number of short paths between people, then there is high connectivity within

the network and a student is able to seek appraisal from another person very easily. In the

case of both appraisal questions, they produced average geodesic distance results that were

not particularly surprising: 2.138 for seeking appraisal and 1.873 for sought appraisal. It is

interesting to note however that the geodesic average for seeking appraisal is higher than for

sought appraisal. This may imply that some members of the course are not approachable or

that people perceive themselves as more approachable than they actually are. In comparison

to other questions, the appraisal questions are quite indicative of the general responses

given. An exception to this is the question “who do you gain information from on work related

topics”, which had the geodesic distance of 3.247. Therefore, to reach all members of the

class of 20, the smallest number of people you would need to inform would be 3.247, a

relatively high figure for a sample of 20 people.

Although geodesic distance revealed very little, the low density of the appraisal networks may

imply that there are many students who are vulnerable to not having their design work peer

reviewed. One appropriate technique to ascertain this is point connectivity. Point connectivity

“calculates the number of nodes that would have to be removed in order for one actor to no

longer be able to reach another” (Hanneman 2006). If there are multiple ways to reach

participants, then the participant is in a less vulnerable position. Point connectivity for the

seeking and sought appraisal networks was generally quite low. Figures 5 and 6 show the

accumulated point connectivity scores. It can be seen from both Figures 5 and 6 that many

individuals have an accumulated point score of around 20 (and it is assumed that vulnerability

≤ 20). They are therefore vulnerable of not getting appraisal from their course mates. It can

also be seen from Figures 5 and 6 that those people who are not vulnerable ( ≥40

accumulated score) have already been identified through the visual networks (those circled in

Figures 1 and 2), as being members of the core within the design studio. This may imply that

being a core member makes a student less vulnerable to not receiving appraisal.

0 25 50 75

Natalie

Sonia

Peter

Hannah

Cara

Colin

Maggie

Cameron

Nick

Louise

Frank

Catherine

Kim

Laura

Anthony

Gayle

Debbie

Jane

James

Lauren

Stu

den

t

Accumulated score

0 25 50 75

Natalie

Sonia

Peter

Hannah

Cara

Colin

Maggie

Cameron

Nick

Louise

Frank

Catherine

Kim

Laura

Anthony

Gayle

Debbie

Jane

James

Lauren

Stu

den

t

Accumulated score

Figure 5: Point connectivity (seek appraisal) Figure 6: Point connectivity (sought appraisal)

The point connectivity for appraisal can be compared to general communication (Figure 7).

This network has much higher figures and hence there are many paths people can take to

communicate with everyone. This is a much healthier scenario.

0 25 50 75 100 125 150 175

Natalie

Sonia

Peter

Hannah

Cara

Colin

Maggie

Cameron

Nick

Louise

Frank

Catherine

Kim

Laura

Anthony

Gayle

Debbie

Jane

James

Lauren

Stu

den

t

Accummulated score

Figure 7: Point connectivity (general communication)

Specific questions and resultsNetwork vulnerable students produce poorer design work (lower grades)

The isolate has no connections with any other actor. Within the network data discussed in this

paper, there were no clear cut isolate students. However there were many students who can

be deemed as vulnerable, in that their accumulated point connectivity scores were ≤ 20. It can

be assumed that the impact of being a vulnerable student is exemplified for isolated students.

Table 2: Ego-net of Maggie

A vulnerable student can also be seen in Table 2, with the ego-net of Maggie. Maggie is just

one of many examples of vulnerability within the design course. It was hypothesised that

vulnerability would equate to poor design work, however T-Test results (Figure 8) show there

was little difference between high and low grade students in comparison to accumulated point

connectivity scores. This result was not expected.

10

15

20

25

30

35

40

45

Seek feedback Soughtfeedback

Network

Acc

um

ula

ted

Po

int

Co

nn

ecti

vity

S

core

Lower gradestudents

Higher gradestudents

Figure 8: T-Test comparison of accumulated point connectivity with grade

Go-between students produce better design work (higher grades)

Table 3: Ego-net of Jane

In a network, a go-between person sits between two other actors. This is shown in Table 3

with Jane’s role. Jane, by being a go-between, is in a powerful position. If for example, Frank

wants to talk to Lauren, he will need to communicate through Jane. The centrality (go-

between) measures that were applied to the data ascertained whether certain people were

more influential or powerful in the network. In the network data discussed in this paper, the

betweenness centrality measure is used, as the data is dichotomised data (filtered high

values and low values), is directed and binary (a value of 1 is given to a person who has rated

another person 4 or 5, whereas if that person had not given 4 or 5 rating then the value is

zero) .

The centrality measures show that Jane is nearly always the highest ranked individual. Her

scores are consistently high (particularly for sought appraisal). Jane can be seen as a go

between because visually she is identified as that (squared area in Figure 2) in addition to

using the centrality measures (shown in Figure 9). By being a go-between she has power and

influence within the course. This can be seen with her inclusion within the core when the core-

periphery analysis was carried out. It could be implied from this, that Jane being a powerful

and central figure, has influence over the entire course itself. If, for example, she is continually

sought for appraisal by her peers, her opinions about their work (assuming they take her

advice) have a direct impact on the design work of the entire class. It could be argued that

she has a hugely powerful role in influencing the outcome and progression of her course

mate’s design work. It is therefore logical to suggest, that what Jane considers to be a “good”

design filters through to the rest of the class. Of course, this is only a proposition and further

research would need to be carried out in order to prove this further.

0

10

20

30

40

50

60

70

Natalie

Sonia

Peter

Hannah

Cara

Colin

Maggie

Cam

eron

Nick

Louise

Frank

Catherine

Kim

Laura

Anthony

Gayle

Debbie

Jane

James

Lauren

Student

Bet

wee

nes

s-va

lue

Seek

Sought

Figure 9: Go-between levels of students

It is proposed that higher grade students are more likely to seek appraisal and to be sought

for appraisal. Figure 10 shows T-test results for seeking and sought appraisal. The mean for

both seeking and sought appraisal is greater for higher grade students than lower grade

students. Thus there is a positive relationship between how much of a go-between a student

is and higher grades.

10

10.5

11

11.5

12

12.5

13

13.5

14

Seek feedback Soughtfeedback

Network

Bet

wee

nes

s ce

ntr

alit

y

Lower gradestudents

Higher gradestudents

Figure 10: T- test comparison of betweeness centrality with grade

Un-reciprocated appraisal has no bearing on design work

An un-reciprocated actor has more connections going in one direction than another. For

example, a student may seek a lot of appraisal but not be sought for appraisal themselves. An

un-reciprocated role can be seen as Hannah in Table 4. Hannah has five incoming network

connections but no outgoing connections for sought appraisal. Those five connections have

therefore answered that Hannah has sought appraisal from them, but Hannah herself has not

listed anyone as having sought appraisal from her. Indeed the reciprocity of the sought

appraisal network was 40% whilst the reciprocity of the seek appraisal was 35%. This could

imply that in this particular design course, appraisal is often one directional.

Table 4: Ego-net of Hannah

Un-reciprocated students can be seen from within the same network, as is the case for

Hannah in Table 4, however un-reciprocated students may also be revealed when comparing

related networks. The seek and sought networks showed how some individuals seek

appraisal but are not sought for appraisal. For example, people sought appraisal from Frank

whereas he did not seek appraisal from other people as much. This disparity in Frank’s ego-

net was also revealed when comparing Frank’s betweeness measures for seek and sought

appraisal, which is shown in Figure 9. It is possible that a consequence of which is that

students may not seem vulnerable because they score highly on one network measure when

in fact they are quite isolated. Hannah, for example, may seem quite well connected because

she is rating her course mates highly. However, because her course mates are not

reciprocating a connection she can be seen as being isolated by the rest of the group.

It should be noted that there were only two individuals with clear cut un-reciprocated survey

responses, and although both students had relatively high grades there is not enough

evidence to suggest that reciprocity has any bearing on a student’s design work.

Core network members produce average design work (average grades)

The core appraisal network members can be seen with the ego-net of Natalie in Table 5. This

grouping fits neatly into a core periphery classification rather than to other types such as

clique. The stringent nature of the network clique definition means that there are multiple

combinations of cliques that could exist within the 20 course students and for that reason is

not explored in further detail in this paper. When core periphery analysis was applied to the

data, the students fitted into either the core category or periphery category as was expected

after observing the group. Figures 11 and 12 show a breakdown of core-periphery

membership. The core members from the design studio had average ranging grades, whilst

peripheral membership contained those with the highest and lowest marks. It cannot be

assumed however, that membership in the core dictates averages marks. Further longtitudal

studies would need to be applied in order to compare the marks of a student before entering

the core and then again once a core member, or after leaving the core. However, knowing the

group from previous studies, course tutors felt that some of the core members were not

fulfilling their grade potential, as tutors referred to them as “hanging out with the wrong

crowd”. Similarly, one core member had previously been outside of the core and her grade

average at that point was higher. Although there is no survey data from this time, anecdotal

evidence from the previous qualitative study tends to suggest that being part of the core

contains students into average grade boundaries, if not even lowering their grades.

Table 5: Ego-net of Natalie

Figure 21: Core/Periphery for seek appraisal Figure 12: Core/Periphery for sought appraisal

There was one individual who can be considered an exception to the core-peripheral findings.

By observation alone, this person would be considered a more peripheral person in the class,

however in the network they are far more central and powerful and are also shown as being a

core member when core periphery analysis is carried out. That individual, Jane, consistently

scored highly for seeking appraisal and very highly for sought appraisal. Similarly Jane scored

highly throughout all of the networks. In fact on the information network and general

communication Jane was the highest rated individual for both prestige and influence. This

was a phenomenon that was not expected after observing the group. This phenomenon can

be seen as a caveat to the network analysis, as the network algorithms used considers Jane

a core group member because she has high betweeness measures (see Figure 9).

ConclusionsThis paper has assessed the impact of sharing appraisal. To do this, the concept of sharing

appraisal has been explored as a two-way phenomenon and compared to other networks

such as communication. The highly rated appraisal networks were shown as less dense than

other networks including the low rated appraisal networks. It is argued that students gave high

rated values to only those students they trust and were, in general, much more dismissive of

their peer’s work. This meant that the scope for refining design work was limited and that a

large number of people were vulnerable to not having their work peer reviewed.

This paper looked at certain roles within the appraisal network. Isolated and vulnerable

students were discussed, however vulnerability seemed to have little bearing on grade result.

However because reciprocity levels in the appraisal network were low, it is proposed that this

may disguise the fact that a student is potentially vulnerable. Core group members were

considered, and it was shown how being in a core makes the student less vulnerable, but

keeps them within a set creativity domain. In comparison students in the periphery were

gaining the very highest and lowest grade results. Further work would need to be carried out

in order to ascertain whether this phenomenon occurred over time. Finally go-between

students were analysed and it was shown how go-between students, on average, produced

better design work and achieved higher grades. It is proposed that this was because their

popularity in the appraisal network helped them refine and improve their work. It was also

argued that students who were go-betweens were in a powerful position as they could

influence the work of others.

Many of the results discussed in this paper and the arguments put forward require further

work in order to strengthen the propositions. This paper is based on one year group, which

may have its own micro culture which isn’t reflected in subsequent years from the design

course. To be much more thorough the same questionnaire should be given to other groups

in different years and in different design schools. This would ultimately give longitudal data

that can reinforce the arguments put forward in this paper.

It is envisaged that the concept of appraisal can be expanded and it is hoped that this paper

raises some interesting questions. Firstly about the impact that appraisal generally has on

design outcomes; but also about the impact that certain network roles have within design

courses in higher education.

ReferencesArgyris, C and Schön, D A (1974) Theory in Practice: Increasing Professional Effectiveness,

Jossey-Bass, San Francisco

Ashton, P and Durling, D: (2000), Doing the Right Thing-Social Processes in Design

Learning. The Design Journal. 3(2): 3-13.

Boyer, E L and Mitgang, L D (1996) Building Community: A New Future for Architecture

Education and Practice, The Carnegie Foundation for the Advancement of Teaching,

Princeton

Bruckman, A., (1998) Community Support for Constructionist Learning In Computer

Supported Cooperative Work: The Journal of Collaborative Computing, Vol 7,

Craig, D. L., (2000) Supporting collaborative design groups as design communities Design

Studies, Vol 21, No2, pp 187-204

Cross, Rob, and Parker, Andrew. (2004) The Hidden Power of Social Networks. Harvard

Business School Press.

Davies, A and Reid, A (2001) ‘Uncovering problematics in design education: learning and the

design entity’, in Swann, C and Young, E (eds) Re-Inventing Design Education in the

University, proceedings of the international conference, School of Design, Curtin University,

Perth, pp. 178-184

Dong, A. (2006). How am I doing? The language of appraisal in design. Design computing

and cognition, Eindhoven, Springer

Fredrickson, M P (1990) Design Juries: A Study in Lines of Communication, Journal of

Architectural Education, Vol 43 No 2, pp. 22-26

Goldschmidt, G (2002) ‘One on one: a pedagogic base for design instruction in the studio’, in

Durling, D and Shacklton, J (eds), Proceeding of Common Ground, DRS international

conference, Brunel U, Stoke-on-Trent, Staffordshire University Press, pp. 430-437

Hanneman, R. and Riddle, M. (2006)

http://www.faculty.ucr.edu/~hanneman/nettext/index.html Retrieved 01/11/2006

Joel, S., Smyth, M. and Rodgers, P. (2005) 'An ethnographical orientated study of designers

in a collaborative design project' in International Workshop in Understanding Designers 05,

University of Provence, Aix-en-Provence, France

Mirochnik, E (2000) Teaching in the First Person: Understanding Voice and Vocabulary in

Learning Relationships, P. Lang, New York

Nicol, D and Pilling, S (eds) (2000) Changing Architectural Education: Towards a New

Professionalism, E and FN Spon, London, New York

Oxman, R. (2004). Think-maps: teaching design thinking in design education. Design Studies

25(1): 63-91.

Porter, W L and Kilbridge, M (eds) (1981) Architecture Education Study, Andrew Mellon

Foundation, Boston

Rodgers, P.A., (2005), Reflecting on Reflection: Reflective Patterns of Design Students, CD

Proceedings of the 6th European Academy of Design Conference, University of the Arts,

Bremen, Germany, 29-31 March 2005.

Schön, D A (1984) The Architectural Studio as an Exemplar Education for Reflection-in-

Action Journal of Architectural Education, Vol 38 No 1, pp. 2-9

Shih, S.G., Hu, T.P. and Chen, C.N., (2006), A game theory-based approach to the analysis

of cooperative learning in design studios, Design Studies, Vol. 27, Issue 6, pp. 711 – 722.

Teasley, S. D., (1997) Talking about reasoning: how important is the peer in peer

collaboration, In Resnick, L. B., Saljo, R., Pontecorvo, C. and Burge, B., Discourse, tool and

reasoning: essays on situated cognition Springer, New York, pp. 361 – 384

Trigwell, K (2001) Judging university teaching, International Journal for Academic

Development, Vol 6 No 1, pp. 65-73

Uluoğlu, B., (2000) Design knowledge communicated in studio critiques, Design Studies, Vol.

21, Issue 1, pp. 33 - 58.