Abstract Computer-supported sketching-based design tools are becoming increasingly available to aid designers as it bridges the gap between traditional design tools/media, such as paper and pen and computer-aided design (CAD) software. However, there has been little empirical research on the effects of using this type of informal design tool, and almost none on the effects of beautification, using such tools, on the design process. Beautification is described as the process of tidying up a hand-drawn design; and formality is described as the outcome of beautification i.e. level of tidiness and professionalism conveyed in the appearance of a design. The main purposes of this study were: 1) explore the concept of beautification in the context of sketch-based design tools by examining the dimensionality of beautification; and also 2) to investigate levels of formality of designs, from hand-drawn (non-beautified) sketches to computer-rendered (beautified) diagrams, and their effects of on design performance (i.e. number of changes made to designs presented) during early stages of the design process. Results showed that: 1) as the level of ii
428
Embed
abstract - Computer Science · Web viewIn order to test whether the data satisfied the normality assumptions for a parametric repeated samples t-test (see Cohen, 1988), histograms,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Abstract
Computer-supported sketching-based design tools are becoming increasingly
available to aid designers as it bridges the gap between traditional design tools/media, such
as paper and pen and computer-aided design (CAD) software. However, there has been
little empirical research on the effects of using this type of informal design tool, and
almost none on the effects of beautification, using such tools, on the design process.
Beautification is described as the process of tidying up a hand-drawn design; and formality
is described as the outcome of beautification i.e. level of tidiness and professionalism
conveyed in the appearance of a design. The main purposes of this study were: 1) explore
the concept of beautification in the context of sketch-based design tools by examining the
dimensionality of beautification; and also 2) to investigate levels of formality of designs,
from hand-drawn (non-beautified) sketches to computer-rendered (beautified) diagrams,
and their effects of on design performance (i.e. number of changes made to designs
presented) during early stages of the design process. Results showed that: 1) as the level
of formality increases, the number of changes made (total, quality and expected changes)
decreases, and vice versa (i.e. a negative linear relationship between formality and design
performance); 2) experts performed at a higher level in comparison to novices’
performance across levels of formality; 3) subjects enjoyed working on designs that with
higher formality more than designs with a lower formality; 4) there was no difference
found in the preference between designing on paper compared to designing on the tablet
PC (InkKit) during the experiment; and 5) design tool preference(s) in real world design
situations was more diverse than the design medium/tool preferred in the experiment.
Important implications arose from this study include: 1) design education on the effects of
formality as a result of beautification; 2) improvements on the design process such as
easier preparation for client presentation and improved efficiency; and 3) sketch-based tool
development, in particular InkKit, to support more satisfying, natural designer-design tool
ii
interaction. Future directions in research such as replication and extension of the present
study, and methodological improvements are also recommended and discussed.
iii
Acknowledgement
First, I would like to thank my supervisors, Dr Brenda Lobb, Dr Beryl Plimmer and
Dr Doug Elliffe. Brenda, you helped open up my mind to see (and feel) answers from
different angles and be a critical reader, thinker and doer; most importantly, you pulled me
up and pushed me forward when I lost faith in my work and myself – thank you. Beryl,
thank you for making me believe in the computer science side of me again. Without your
encouragement and support at the beginning of this project, I would not have made it
through the programming days in the lab. Thank you Doug, for helping me refine the
experiment and data collection procedures; and for clearing up the grey clouds in my mind
on results analyzes.
I would also like to thank my family: Mum and Dad, for giving me life in the first
place and for your unconditional love and support, I love you; and Susan, my sister, my
best-friend, for being a good listener and advisor, and thank you for being true, I love you
too. Also my gym buddy and dining buddy, Iris, thanks for making me laugh when I don’t
smile, for keeping me positive, and for your encouragement throughout the year.
ii
Table of Contents
Abstract..........................................................................................................iiAcknowledgement.........................................................................................iiTable of Contents.........................................................................................iiiList of Tables...............................................................................................viiList of Figures...............................................................................................ixChapter 1. Introduction................................................................................1
1.1. Design Research: The design process...............................................................21.2. Design process as problem-solving...................................................................3
1.2.1. Factors affecting design performance.....................................................41.2.1.1. Expert vs. novice designers....................................................................41.2.1.1. Individual differences in problem-solving...................................71.1.2.2. Other factors affecting design performances.......................................11
1.1. Human-Computer-Interaction (HCI) and Design...........................................121.2. Prototypes, Prototyping and Prototyping tools...............................................14
1.4.1. Traditional Design tools for Prototyping..............................................181.4.1.1. Paper and Pen(cil)................................................................................181.4.1.2. Computer-Aided Design (CAD) Tools................................................191.4.1.3. Combination of Paper and Pen, and CAD...........................................211.4.1.4. Paper prototypes verses Digital prototypes..........................................23
1.3. Current Trend in Design Tools Research: Informal Sketch-based interface. .251.3.2. 2-Dimensional (2-D) sketch-based systems..........................................271.5.2. 3-Dimensional (3-D) Sketch-based systems.........................................281.5.3. On Improving Computer-Supported sketch tools.................................291.5.4. Bridging the Gap: A closer look at ‘Beautification’ (‘Formalization’)30
1.5.4.1. ‘Beautification’ versus ‘Formality’......................................................321.5.4.2. Practicality of Beautification in the design process.............................341.5.4.3. Beautification techniques and supporting systems..............................35
1.4. Related studies: Interaction with hand-drawn versus computer-rendered diagrams..........................................................................................................42
1.5. The Present study: Aims and hypotheses........................................................48
2.4.1. Room Setup...........................................................................................592.4.2. The Tablet PC........................................................................................592.4.3. Morae Recorder (2004).........................................................................602.4.4. Inkit and the programming of beautification functions.........................60
2.4.4.1. Horizontal Alignment..........................................................................612.4.4.2. Vertical Alignment..............................................................................632.4.4.3. Standardization....................................................................................632.4.4.4. Line Smoothing...................................................................................64
2.5. Stimuli and Materials......................................................................................68
iii
2.5.1. Instruction Sheets..................................................................................682.5.2. The five designs each representing a different level of formality.........682.5.3. Post-task Questionnaire.........................................................................74
3.1. Data-screening of performance data...............................................................753.2. Analysis of performance data: One-way repeated measures ANOVA...........76
3.2.1. Analysis of “Total Changes” made across levels of formality..............773.2.1.1. Between-Subject Factors........................................................................79
3.2.1.1a. Design experience..............................................................................803.2.1.1b. Study major/specialization................................................................823.2.1.1c. Study Level........................................................................................83
3.2.1.2. Multiple Regression analysis..................................................................853.2.2. Analysis of “Quality Changes” made across levels of formality..........86
3.2.2.1. Between-Subject Factors........................................................................893.2.2.1a. Design Experience.............................................................................893.2.2.1b. Study major/specialization................................................................913.2.2.1c. Study Level........................................................................................93
3.2.2.2. Multiple Regression Analysis.................................................................953.2.3. Analysis of “Expected Changes” made across levels of formality.......97
3.2.3.1. Between-Subjects Factors.......................................................................993.2.3.1a. Design Experience.............................................................................993.2.3.1b. Study major/specialization..............................................................1013.2.3.1c. Study Level......................................................................................103
3.2.3.2. Multiple Regression Analysis...............................................................1043.3. Additional Analysis of performance data.....................................................106
3.3.1. Paired comparisons: Total, Quality and Expected changes................1063.3.2. Extra changes: Quality – Expected; and Total – Quality....................1073.3.3. Order Effect.........................................................................................109
3.4. Analysis of the “Overall Enjoyment” rankings of the five designs..............1133.4.1. Ranking according to design appearance (aesthetics).........................1143.4.2. Ranking according to perceived “effort required”..............................1153.4.3. Ranking according to perceived “fun and/or stimulating level”.........116
3.5 Design Tool Preference..................................................................................1173.5.1. Design tool preference in the experiment...........................................1173.5.2. Design tool preference in the experiment...........................................119
4.1. Effects of formality on design task performance..........................................1224.2. Between-subject effects: Expertise...............................................................132
4.2.1. Design experience................................................................................1324.2.2. Study major/specialization..................................................................1334.2.3. Study Level..........................................................................................134
4.4.1. Relationship between total, quality and expected changes.................1364.5. “Overall Enjoyment” rankings of the five designs ranking of designs.........138
4.5.1. Rankings according to aesthetic aspects of designs............................1384.5.2. Rankings according to effort required.................................................1404.5.3. Rankings according to Stimulation/fun level......................................141
iv
4.6. Design tool preference..................................................................................1424.6.1. Design tool preference during the experiment....................................1424.6.2. Design tool preference in the real world.............................................145
4.7. Implications...................................................................................................1464.7.1. Implications on sketch-based tool development: Recommendations for
InkKit...................................................................................................1464.7.2. Improvements in the design process....................................................1484.7.3. Implications on design education........................................................149
4.8. Methodological issues and limitations..........................................................1514.9. Future research and directions......................................................................151
Chapter 5. Summary and Conclusion.....................................................155References..................................................................................................157Appendices.................................................................................................168
Appendix A. The five designs and the outline of design errors present in each design............................................................................................................169Appendix A1.1. Low formality design on paper – Online Magazine
subscription form.................................................................................170Appendix A.1.2. Online Magazine: Planned design errors...........................171Appendix A2.1. Low formality design on tablet PC – Samson’s Bank $1
million loan application form..............................................................172Appendix A2.2. Samson’s loan: Planned design errors................................173Appendix A3.1. Medium-low formality design on tablet PC – University of
Strawberries graduation application form...........................................174Appendix A3.2. University of Strawberries: Planned design “errors”.........175Appendix A4.1. Medium-high formality design on tablet PC – Dog
Registration Form................................................................................176Appendix A4.2. Dog Registration’s: Planned design “errors”.....................177Appendix A5.1. High formality design on tablet PC – America’s Next Top
Model application form.......................................................................178Appendix A5.2. America’s Next Top Model: Planned design errors...........179
Appendix B. Post-task Questionnaire..................................................................180Appendix C. Results of post-task questionnaire..................................................184Appendix D. Participant information sheets and consent forms..........................185Appendix E. Functional Aspects of Inkit............................................................192Appendix F. Instruction sheets containing the requirements and scenario
associated with each design..........................................................................193Appendix F1. Instructions including the requirements and the scenario for the
low formality (on paper) design..........................................................194Appendix F2. Instructions including the requirements and the scenario for the
low formality (on Tablet PC) design...................................................197Appendix F3. Instructions including the requirements and the scenario for the
medium-low formality design.............................................................200Appendix F4. Instructions including the requirements and the scenario for the
medium-high formality design............................................................203Appendix F5. Instructions including the requirements and the scenario for the
high formality design...........................................................................206Appendix G. Screen shots during font creation using My Font Tool for Tablet PC
(2004)............................................................................................................209Appendix H. Testing the normality assumption – Total number of changes made
across levels of formality..............................................................................211
v
Appendix I. Testing the normality assumption – Number of quality changes made across levels of formality..............................................................................218
Appendix J: Testing the normality assumption – Number of expected changes made across levels of formality....................................................................225
Appendix K. Mean total number of changes made across each level of formality – according to a combination of between-subjects factors (design experience, major/specialization and study level)............................................................232
Appendix L. Mean number of quality changes made across each level of formality – according to a combination of between-subjects factors (design experience, major/specialization and study level)............................................................233
Appendix M. Mean number of expected changes across each level of formality – according to between-subjects factors (design experience, major/specialization and study level)............................................................234
Appendix N. One-way ANOVA and post-hoc multiple comparisons between total, quality, and expected changes made across each level of formality....235
Appendix O: “Extra changes” made in designs...................................................236Appendix O1. “Extra changes” made in the Low Formality Design presented
on paper: International Online Magazine Subscription Form.............237Appendix O2. “Extra changes” made in the Low Formality Design on tablet
PC: Samson’s Bank $1 million Loan Application Form.....................240Appendix O3. “Extra changes” made in the Medium-Low Formality Design:
University of Strawberries Graduation Form......................................243Appendix O4. “Extra changes” made in the Medium-High Formality Design:
Dog Registration Online Form............................................................245Appendix O5. “Extra changes” made in the High Formality Design: 2007
America’s Next Top Model Online Application Form.......................247Appendix P. “Overall Enjoyment” rankings of the five designs across each level
of formality...................................................................................................249Appendix Q. Number of changes made in the five designs across levels of
formality by subjects whose “Overall Enjoyment” ranks was dependent on the appearance (aesthetics) of designs..........................................................251
Appendix R. Number of changes made in the five designs across levels of formality by subjects whose “Overall Enjoyment” ranks was dependent on perceived effort required...............................................................................252
Appendix S. Number of changes made in the five designs across levels of formality by subjects whose “Overall Enjoyment” ranks was dependent on the level of fun/stimulation when working on the designs...........................253
Appendix T. Subjects reasons for design tool preference during the experiment254Appendix U. Subjects reasons for design tool preference in real life design
Table 1. Level of formality associated with each condition, and the medium used for the presentation and review of designs...........................................................1
Table 2. Taxonomy of beautification showing different variables associated with beautification..................................................................................................2
Table 3. Ratio of elements aligned (and its description) in each design representing a different level of formality.............................................................................20
Table 4. Systematic smoothing applied (% smoothed) to the original hand-drawn lines of textboxes, dropdown menus and radio buttons; and fonts used for labels to represent different levels of formality in the designs presented to the participant...............................................................................................21
Table 5. Mean and standard deviation for total changes made at each level of formality........................................................................................................27
Table 6. Mean differences and their significance at the .05 level in terms of total number of changes made between each condition........................................29
Table 7. Mean and standard deviation for total changes made, and the mean difference between groups, at each levels of formality according to subjects’ design experience (total n =30): none to some (non-CS/SE) design experience (n = 15) and CS/SE design experience (n = 15)........................30
Table 8. Mean and standard deviation for total changes made, and the mean difference between groups, at each level of formality according to subjects’ major/specialization in university (Total n =30): non-CS/SE related major (n = 10) and CS/SE related major (n = 20)..................................................32
Table 9. Mean and standard deviation for total changes made, and the mean difference between groups, at each level of formality according to subjects’ study level (total n=30): undergraduate (n=22) and graduate/postgraduate (n=8).............................................................................................................33
Table 10.1. R, Adjusted R Square and R Square change for total changes made across levels of formality, with formality level, design experience, study level and major/specialization as predictors entered...................................................36
Table 10.2. The unstandardized and standardized regression coefficients, and the t-value and significance of each between-subject variables included in the mode for explaining total changes made......................................................36
Table 11. Mean and standard deviation for quality changes made at each level of formality .......................................................................................................36
Table 12. Mean differences and their significance at the .05 level in terms of the number of quality changes made between each condition...........................38
Table 13. Mean and standard deviation for quality changes made, and the mean difference between groups, at each level of formality according to design experience (total n=30): none to some (non-CS/SE) design experience (n=15) and CS/SE design experience (n=15) ..............................................39
Table 14. Mean and standard deviation for quality changes made, and the mean difference between groups, at each level of formality according to subjects’ major/specialization in Auckland University: Non-CS/SE related major (n = 10) and CS/SE related majors (n = 20)....................................................41
Table 15. Mean and standard deviation of quality changes made, and the mean difference between groups, at each level of formality according to subjects’ study level: undergraduate (n=22) and graduate/postgraduate (n=8).......43
Table 16.1. R, Adjusted R Square and R Square change for the number of quality changes made across levels of formality, with formality level, design
vii
experience, study level and major/specialization as predictors entered......45Table 16.2. The unstandardized and standardized regression coefficients, and the t-
value and significance of each between-subject variables included in the model for explaining quality changes made.................................................46
Table 17. Mean and standard deviation for expected changes made across levels of formality........................................................................................................46
Table 18. Mean differences and their significance at the .05 level in terms of the number of expected changes made between each condition.........................48
Table 19. Mean and standard deviation for expected changes made, and the mean difference between groups, at each level of formality according to design experience (Total n =30): CS/SE design experience (n = 15), none to some (non-CS/SE) design experience (n = 15)......................................................49
Table 20. Means and standard deviations for expected changes made, and the mean difference between groups, at each level of formality according to major/specialization (n=30): non-CS/SE related major (n=10); CS/SE related major (n=20.....................................................................................51
Table 21. Mean and standard deviation of expected changes made, and the mean difference between groups, at each level of formality according to study levels (n=30): undergraduate (n = 22); graduate/postgraduate (n = 8).....52
Table 22.1. R, Adjusted R Square and R Square change for the number of expected changes made across levels of formality, with formality level, design experience, study level and major/specialization as predictors entered......54
Table 22.2. The unstandardized and standardized regression coefficients, and the t-value and significance of each between-subject variables included in the model for explaining expected changes made..............................................55
Table 23. The number of extra changes (quality – expected) made in each design, grouped according to the type of change......................................................57
Table 24. The number of extra changes (total – quality) changes made in each design, grouped according to the type of change......................................................57
Table 25. Mean ranks and standard deviation, in terms of overall perceived enjoyment and other underlying factors for subjects’ rankings (including appearance, perceived effort required, and perceived fun/stimulating level), when working on each design in comparison to other designs presented.............63
viii
List of Figures
Figure 1. Partial counterbalancing: orders of presentation of conditions..................1Figure 3. The practice design – presented to the participant prior the first experiment
condition.........................................................................................................7Figure 4. A bird-eye view of the room set-up for the experimental sessions...............10Figure 5. Horizontal alignment of elements.................................................................12Figure 6. Vertical alignment of elements......................................................................13Figure 7. Standardization of the size of objects............................................................14Figure 8. Levels of smoothing of hand-drawn objects (lines) to represent levels of
formality.....................................................................................................15Figure 9. The four fonts used to represent different levels of formality.......................17Figure 10.1. Low Formality (on paper) – Online Magazine subscription form..........22Figure 10.2. Low Formality (on Tablet PC) – Samson’s Bank $1 million loan
application form.....................................................................................22Figure 10.3. Medium-Low Formality – University of Strawberries graduation
application form.....................................................................................23Figure 10.4. Medium-High Formality - Dog Registration Form................................23Figure 10.5. High Formality design - America’s Next Top Model application form..24Figure 11. Multi-line graph showing mean total changes made across levels of
formality which is represented by the black bold line; each participant’s performance (in terms of total changes made across levels of formality) is also illustrated – see individual lines.........................................................28
Figure 12. Multi-line graph of mean total changes made across levels of formality according to subjects’ design experience: none to some (non-CS/SE) design experience and CS/SE design experience.......................................31
Figure 13. Multi-line graph of mean total changes made across levels of formality according to subjects’ major/specialization in university: Non-CS/SE related major and CS/SE related majors...................................................33
Figure 14. Multi-line graph of mean total changes made across levels of formality according to subjects study level: undergraduate and graduate/postgraduate...............................................................................34
Figure 15. Multi-line graph showing mean quality changes made across levels of formality which is represented by the black bold line. Each participant’s performance (in terms of quality changes made across levels of formality) is also illustrated – see individual lines.....................................................37
Figure 16. Multi-line graph of mean quality changes made across levels of formality according to subjects’ design experience: none to some (non-CS/SE) design experience and CS/SE design experience.......................................40
Figure 17. Multi-line graph of mean quality changes made across levels of formality according to subjects’ major/specialization in university: Non-CS/SE related major and CS/SE related majors...................................................42
Figure 18. Multi-line graph of mean quality changes made across levels of formality according to subjects’ study level: undergraduate and graduate/postgraduate...............................................................................44
Figure 19. Multi-line graph showing mean expected changes across levels of formality which is represented by the black bold line. Each participant’s performance (in terms of expected changes made across levels of formality) is also illustrated – see individual lines....................................47
Figure 20. Multi-line graph of mean expected changes made across levels of formality according to subject’s design experience...................................................50
ix
Figure 21. Multi-line graph of mean expected changes made across levels of formality according to subject’s major/specialization: non-CS/SE related major and CS/SE related majors.................................................................................52
Figure 22. Multi-line graph of mean expected changes made across levels of formality according to subjects’ study levels: undergraduate and graduate/postgraduate...............................................................................53
Figure 23. Multi-line graph of mean total changes, mean quality changes and mean expected changes across levels of formality...............................................56
Figure 24a. Mean total changes made across levels of formality according to the order of conditions presented.....................................................................59
Figure 24b. Mean total changes made across levels of formality according to the presentation of conditions in the 54321 (n=4) and 12345 directions (n=4)59
Figure 25a. Mean quality changes made across levels of formality according to the order of conditions presented.....................................................................60
Figure 25b. Mean quality changes made across levels of formality according to the presentation of conditions in the 54321 (n=4) and 12345 (n=4) directions60
Figure 26a. Mean expected changes made across levels of formality according to the order of conditions presented.....................................................................61
Figure 26b. Mean expected changes made across levels of formality according to the presentation of conditions in the 54321 (n=4) and 12345 (n=4) directions61
Figure 27. A bar graph showing mean rank and standard deviation, in terms of preference, according to the overall enjoyment, in working on each design with a different level of formality...............................................................62
Figure 28. Bar graph showing subjects’ design tool preference during the experiment67Figure 29a. Bar graph showing the proportion of subjects – according to study major:
CS/SE (n=20) and non-CS/SE majors (n=10) – preferring different design tools during the experiment (paper and pen; Tablet (Inkit); or no preference)..................................................................................................68
Figure 29b. Bar graph showing the proportion of subjects – according to study major: CS/SE design experience (n=15) and none to some non-CS/SE design experience (n=10) – preferring different design tools during the experiment (paper and pen; Tablet (Inkit); or no preference)...................68
Figure 30. Bar graph showing subjects’ design tool preference in real life design situations....................................................................................................69
Figure 31a. Bar graph showing the proportion of subjects – according to study major: CS/SE (n=20) and non-CS/SE majors (n=10) – preferring different design tools in real life design situations...............................................................70
Figure 31b. Bar graph showing the proportion of subjects – according to study major: CS/SE design experience (n=15) and none to some non-CS/SE design experience (n=10) – preferring different design tools in real life design situations....................................................................................................71
x
Chapter 1. Introduction
Computer-supported sketching-based design tools – also referred to as informal
design tools as they support natural human-computer interaction (i.e. sketching) – are
becoming increasingly available to aid designers across various design disciplines. The
main advantage of using such tools is that it bridges the gap between traditional design
tools/media, such as paper and pen and computer-aided design (CAD) software, such that
sketching (a natural, important design behavior) is supported, while also providing
additional functions such as editing and version control, as well as recognition and
beautification of pen input. However, there has been little empirical research on the
effects of using this type of informal design tool, and almost none on the effects of
beautification (design formality) using such tools, during the design process. In other
words, there is a need to examine the effects of design formality (appearance) as a result of
beautification (beautifying sketched content), on designers’ cognition, and hence, design
performance and outcome.
With this in mind, designers’ interaction with design tools/mediums and design
formality (as a result of beautification) were examined in the present study. The main
purposes of this study were to use an experimental approach to: 1) further explore the
concept of beautification in the context of sketch-based design tools by examining the
dimensionality of beautification; and also 2) to investigate levels of formality of designs,
from rough, (hand-drawn, non-beautified) sketches to formal (beautified) diagrams, and
their effects of on design performance during early stages of the design process.
This chapter of the thesis is organized into a few sections. First, research on the
design process and the manner in which it has been studied in the past are discussed, and
design as an emerging topic within the area of human-computer interaction is also
highlighted. The next section is on the development and the use of design tools, within
1
which, two traditional design tools (pen and paper and computer-aided design tools) are
compared, followed by a discussion on the recent trend of design tools with a sketching-
based interface. Furthermore, the concept of beautification (and formality) is described,
and research on informal sketch-based design tools that supports beautification is
reviewed. Research relevant to this study is also presented. Finally, in the last section of
this chapter, the aims and hypothesis of the present study are outlined.
1.1. Design Research: The design process
Design has long been of interest to many groups, from academics (e.g. researchers,
philosophers and psychologists) to practitioners (e.g. educators, engineers, architects) ever
since design activities began. It was during the late 1960s, that scientific research on
design began, according to Bayazit (2004), with different design research associations
founded across the globe during the time. For example, the Design Research Society,
which started the Design Studies journal – a journal dedicated to all design related studies
and research across multiple disciplines including architectural design, engineering design,
industrial design and software design. Since then, design research grew steadily
throughout the years (see Bayazit, 2004; Roth, 1999; and Downton, 2003 for fuller
historical accounts of design research and its current state). Cross (1999) distinguished
three categories of research on design, based on people, process and products:
Design epistemology – study of ways of knowing of designers
Design praxiology – study of practices and processes of design
Design phenomenology – study of the form and configuration of artifacts.
One of the major topics in design research is on the design process, which has been
much studied across disciplines, based on different approaches and perspectives, including
psychological, social, philosophical and mathematical. Researchers have described the
design process using different stage-process models. For example, Crampton-Smith and
2
Tabor (1996) described a generic design process that consists of understanding,
abstracting, structuring, representing and detailing. On the other hand, from the interviews
with eleven professional Web designers in their workplace, Newman et al. (2003) found
four main phases: discovery, design exploration, design refinement, and production. Such
pattern of iterative refinement was also discovered by Rowe (1987) who reviewed of a
number of staged-process models of design that were proposed in the early 1960s. Along
with the models to describe the design process, a wide range of studies were conducted
with focus on different aspects of the design process including the design process as a
whole (e.g. Atman, et al, 1999; Atman, et al, 2005); factors affecting the design process
(e.g. Darke, 1979; Naga & Noguchi, 2003; Ward, 1989); and design tools and
methodology used during the design process (e.g. Grosjean, & Brassac, 2000; Bilda, Gero,
& Purcell, 2006; Shneiderman, et al, 2006).
1.2. Design process as problem-solving
Design has been described as a complex and fastidious mental activity (Romer,
Leinert, & Sachse, 2000), which can be viewed as a kind of problem-solving
(Goldschmidt, 1997; Hegarty 1991; Rowe 1987; Smith and Browne 1993; Thomas &
Caroll, 1979). From a broader cognitive psychology perspective, problem-solving
encompasses a wide range of activities in which one is required to identify the solution to
a current problem (Green & Gilhooly, 2005); hence, the process of designing can be
viewed as part of problem-solving. As noted by Green and Gilhooly (2005), problem
solving is an activity that draws together the various different components of cognition –
for example, visual perception for the understanding a graphically presented problem and
for drawing a solution; as well as memory to recover any prior knowledge one might have
that could be relevant to solving a new problem; and attention which plays an important
role in all problem solving. Thus, the design process can be understood as a problem-
3
solving process. Furthermore, the major focus on research on problem-solving has been
on task performance (see Ericsson, 1991); hence, research on design as problem solving
has also as been on design performance (in terms of qualitative and quantitative
measurements such as time used, design quality and design outcome).
1.2.1. Factors affecting design performance
1.2.1.1. Expert vs. novice designers
Expertise in design has been viewed by many (e.g. Cross, 1999) as an important
factor that affects designers’ performance in the design process. One of the major
approaches towards exploring the concept of expertise within the context of problem-
solving (design) is to compare experts and novices. According to Cross (2004), novice
behaviour is usually associated with a ‘depth-first’ approach to problem solving, i.e.
sequentially identifying and exploring sub-solutions in depth, whereas the strategies of
experts are usually regarded as being predominantly ‘top-down’ and ‘breadth-first’
approaches. Many of the classic studies of expertise have been based on examples of
game-playing (e.g. chess), or on comparisons of experts versus novices in solving routine
problems (e.g. mathematics and physics). For example, early chess studies carried out by
De Groot (1946/1965) showed that instead of having superior information processing
2005; Landay, & Myers, 2001). However, empirical research at the other end to examine
the effects on the designers’ cognition, perception, behaviour, work (design) performance
etc, of using such tools during the design process, is still at its infancy. Especially, the
lack of research on the extent of beautification that result in beautified designs which
appear more or less formal, and the effects of interacting with such designs on the design
process – will it affect design behavior, design performance and design outcome? Hence,
46
in addition to the need to further explore beautification techniques within the context of
sketch-based tools, there is a need to examine their effects on designers during the design
process.
The main purposes of this study were to use an experimental approach to: 1)
further explore the concept of beautification in the context of sketch-based design tools by
examining the dimensionality of beautification and its techniques; and 2) to also
investigate the effects of design formality (beautification outcome) – from rough (hand-
drawn, non-beautified) sketches to formal (beautified) diagrams – on design performance.
More specifically, this study explores the effects of different formality level of designs on
design performance, by measuring the number of changes made: total changes, quality
changes and expected changes made (described in Section 2: Method). Also taken into
account are factors that may play a role in affecting the relationships between formality
level of designs and design outcome, including expertise (e.g. design experience,
education, domain-specific knowledge), design perception and design medium. In
addition, the present study has a particular focus on the early stages of the design process
because of their determinant influence on design costs and design outcomes.
Hypothesis:
1) That the number of functional changes made (total, quality and
expected changes) will differ as levels of formality of a design increase (or
decrease).
2) That the number of functional changes made (total, quality and
expected changes) will differ between ‘experts’ and ‘novices’ as levels of formality
of a design increase (or decrease). More specifically:
47
a. That there is a difference in design performance between
subjects with task related design experience and subjects with no
design experience to some non-task related design experience.
b. That there is a difference in design performance between
subjects with task-related domain - specific knowledge (i.e. study
major/specialization) and subjects with no task-related domain-specific
knowledge to some task-related domain specific knowledge.
c. That there is a difference in design performance between
subjects with higher education (study) level (university
graduates/postgraduates) and subjects with lower education level in
comparison (university undergraduates).
3) That subjects will enjoy working on designs that appear more
formal (higher formality – i.e. more beautified) more than designs that appear less
formal, rougher with a sketchy look-and-feel (lower formality – i.e. less
beautified).
4) That there is no difference in preference between designing on
paper compared to designing on the tablet PC (InkKit).
5) That design medium/tool preference in real world design
situations would be more diverse than design medium/tool preference during the
experiment.
48
Chapter 2. Method
2.1. Experimental Design
A within-subject repeated measures design was used in this experiment to measure
the effects of formality on design performance. There were five conditions, all of which
were presented to each subject one after another. Latin square design, as shown in Figure
1, was used in to control order effects of the conditions presented (Heiman, 2001). In
addition, between-subject comparisons were also made to test Hypothesis 3 where experts
and novices may be affected by formality differently during the design process.
Subjects
Figure 1. Latin square design: orders of presentation of conditions i.e. rotation of conditions in two directions (from 1 to 5, and from 5 to 1), to control for practice effects.
2.1.1. Independent Variable: Level of Formality
There was one independent variable – levels of formality of a web interface design
prototype (HTML forms) with four levels: from low formality to high formality. There
were five conditions in the experiment, as shown in Table 1, with four conditions each
involving one HTML form design with a different level of formality presented on the
tablet PC, and for comparison, as interaction with digital and traditional media differs
(Bilda & Demirkan, 2003) which in turn, may affect design-decisions (Black, 1990), one
condition involved one HTML form design with low formality was presented on paper.Table 1.Level of formality associated with each condition, and the medium used for the presentation and review of designs.
As the number of deliberate errors was the same in each design presented to the
participants, the number of corrections (expected changes) made in each design was
measured to allow for controlled comparisons between conditions, hence, to explore the
effects of formality on design-decisions during early stages of the design process (i.e.
during early prototyping). In addition to expected changes, quality changes and total
changes made were also measured as it was anticipated that along with expected changes,
participants would made other changes to the design that were not deliberate errors (refer
to Appendix A for the outline of design errors that was present in each design). The three
measurements (total changes, quality changes, and expected changes) were important for
the assessment of validity and reliability of the experimental stimuli – the five designs
presented to the participants i.e. whether the number of quality changes and expected
55
changes were similar; and whether the total number of changes made was statistically
different from the number of quality changes. If the number of changes made were not
statistically different, then it could be statistically reasoned, that the five designs were
equivalent, and that the results was due to the experimental manipulation of the
independent variable.
Participants were directed to make functional changes by the instruction to
“improve the design[s] to better serve its [their] purposes”, therefore, participants did not
have to make any beautification changes such as alignment of elements (see other
examples in Table 2) to ‘tidy-up’ the design. Thus, beautification changes made by
participants were not counted as a functional change. Furthermore, beautification changes
were not measured as a dependent variable it was predicted that the process of tidying up a
design was time consuming (e.g. Newma, et al., 2003) especially in an experiment with
only 11 minutes for each condition.
In addition, a post-task questionnaire was used to record the following variables
(see Appendix B for response options details in the questionnaire):
o Post-task rankings of overall enjoyment of designs in the order from the most-liked
design (1) to the least liked design (5). Reasons for rankings were also recorded.
o Preference for design medium in the experiment. Response options were
preference for pen and paper; preference for the tablet PC; or no preference.
o Preference for design medium in the real world. Response options were open.
o Demographics including (open response) :
Design experience.
Study specialization/major.
Study Level.
2.2. Participants
Thirty adults – sixteen male (mean age of 22.81, SD = 5.87) and fourteen female
(mean age of 21.14, SD = 1.03) between 18 and 44 years of age (total mean age of 22.03,
SD = 4.36) volunteered to participate in the study.
Participants (n=20 who majored/specialized in Computer Science (CS) or Software
Engineering (SE) in their study, as well as participants with non-CS/SE study backgrounds
(n=10) were recruited from the University of Auckland. All participants were current
students/recent graduates from the University of Auckland (mean years of study at
university: 3.20, SD = 1.186). Out of thirty, twenty-five participants were students
56
(twenty-two undergraduates and three post-graduates), five participants, recruited through
researcher’s personal contacts, were recent graduates from the University of Auckland in
2006, all currently working in computer/software engineering-related industries. Overall,
there were twenty-two undergraduates, and eight graduates/postgraduates.
Papers taken most frequently during the course of study were the computer science
stage one papers including CS101. All participants (n=22) who majored/specialized in
computer science(CS)/software engineering(CS) had taken or taking CS101 and only two
out of ten participants who majored in non-CS/SE subjects (including other engineering,
business and information-system students) had taken CS101. Relevant papers taken by
participants who majored in CS/SE were: CS101 and CS105 (stage one); CS230 (stage
two); CS345 (stage three) and SE SE450 (equivalent to CS345). Others papers taken by
participants with non-CS/SE majors included: information system, engineering,
biomedical science, psychology and business papers. Fifteen participants had CS and/or
SE design experience such as HTML design, website and interface designs and software
design, whereas, the other fifteen participants’ design experience ranged from no design
experience to some non-CS/SE related design experience.
All participants were exposed to Inkit (the design tool used in the experiment) for
the first time. All participants had normal eyesight or corrected-to-normal by spectacles or
contact lenses. For the summary of demographics, see Appendix C.
Participants were each reminded not to discuss the experiment with their peers, and
were thanked with $2 worth of chocolate as a token of appreciation as well as entering the
draw to win $50 cash.
2.3. Procedure
Study approval was obtained from the University of Auckland Human Participants
Ethics Committee (UAHPEC). Each participant took part in a single session
approximately one hour long. An experimental protocol was produced to standardize
experimental procedures and thus, help minimize experimental errors and variability.
Participants were first instructed to make changes to improve each design presented to
them. Five early designs of online forms were then presented to the participants one after
another – four designs were presented on the tablet PC and one was presented on paper
(refer to Table 1). Upon completion of experimental tasks, participants filled in the post-
task questionnaire. The detailed procedure is described next.
57
Upon arrival, the participant was asked to read the information sheet and to sign
the consent form (see Appendix D). The participant was then asked to adjust the work
station, to suit him or her, including the chair height and its distance to the desk, the desk
height (by adjusting the lever), screen angle (of the tablet PC) and the positioning of the
mouse. The experimenter further checked if the participant was comfortable with the
lighting level and room temperature, and whether there was anything else the participant
needed to do before starting the experiment, to minimize disruption of the experimental
procedures.
InKit, an informal design tool (i.e. the experimental apparatus described below in
section 2.4.4.) was presented to the participant on the tablet PC (described in section 2.4.3)
during which a brief introduction to Inkit – its authors and purposes – was given. The
practice design, as shown in Figure 3, containing sketches of four common types of
elements found in HTML forms (i.e. text boxes, dropdown menus, labels and radio
buttons) was presented and a description of each element was given in terms of usage and
functionality.
Figure 3. The practice design – presented to the participant prior the first experiment condition. It contains the four main elements (label, textbox, dropdown menu, and radio button) used in the to-be-given designs.
Instructions were explained again, informing the participant that he or she could
make any changes to the design. In addition to the types of changes described earlier (p.2)
including adding, deleting, changing, resizing and relocation of any elements in a design,
participants were instructed that annotation was also acceptable (e.g. make notes,
explanation, draw arrows etc) to indicate changes that should be made.
While explaining each type of change, the experimenter also demonstrated how the
change could be made in Inkit with pen-input, on the tablet PC (see Table 1 in Appendix
58
05 for the demonstration details). Although passively shown already in the previous steps,
the participant was informed explicitly on the three ‘modes’ in Inkit (see Table 2 in
Appendix 05), and how to change modes (i.e. by tapping on the icon associated to the
mode, and an exclusively highlighted icon meant that desired mode has been successfully
selected).
The next five to ten minutes was the guided familiarization process in which
participants were able to get familiar with Inkit. At the end of the period, participants
showed that they could draw (add and annotate), erase (delete), select and move
(relocation), and select and resize (resize), and hence, changing modes (drawing, erasing
and selecting). The instruction sheet (described in section 2.5.1 below) containing the
requirements and the scenario for the first condition was then presented. Instructions were
read out-loud by the experimenter and subjects were to ask questions if they were unclear
(see Appendix F for each set of requirements and scenario associated with each design).
Before starting the experiment, participants were explicitly told to stay within the
application and to use only the three functionalities (draw, erase, select) at all times. The
designs were then presented successively to the participant depending on the randomly
selected order of conditions for the particular participant (refer to Figure 1). The five
designs presented (one design in each condition) are described below in section 2.5.2.
The first form design was presented either on paper or the tablet PC according to
the condition order. Each subject was given 11 minutes in each condition – 1 minute for
reading the instructions and 10 minutes for working on the design. At the end of eleven
minutes, the participant was asked to stop making changes or to finish off any changes
they were making. One extra minute was given if the participant had asked for more time.
When the changed design file was saved to the participant’s unique folder, the next design
was presented (again, either on paper or the tablet PC according to the condition order)
together with its associated instruction sheets.
With the design presented on paper, the participant was given a sheet of blank A4
paper, along with the design and the instruction sheets which also contained the
requirements and the scenario. The participant was instructed to make changes on the
original design using the blue ball-point pen given, and to use the blank paper if more
space was needed. With designs presented on the tablet PC in Inkit, the participant used
the specialized pen for the tablet PC to draw directly on the tablet screen (with immediate
input feedback – i.e. what you draw is what you see) as if drawing on a piece of paper.
While the participant was working on the design, the experimenter monitored the
process on the display panel output on the other side of the room, and recorded (on paper)
59
any observations such as design behaviors, usability issues, timing issues such as over-
time or under-time etc that were of interest. This information was also recorded on the
tablet PC using Morae Recorder (2000), as described in section 2.4.3.
When the last condition has ended, the participant was asked to fill in the
anonymous paper-and-pencil questionnaire (refer to Appendix B) and to notify the
experimenter when page 3 in the questionnaire was reached so that the experimenter could
show him/her the five designs that they had worked on (in their order of presentation) to
facilitate answering of subsequent questions.
2.4. Apparatus
2.4.1. Room Setup
Each experiment session involved only one subject which took place in a quiet
room in the Department of Computer Science building, at the University of Auckland.
The room was set up so that neither the subject nor the experimenter could see each other
during each experiment condition (see Figure 4) to minimize effects of observation.
2.4.2. The Tablet PC
The experiment was conducted using a Toshiba Tablet PC (Edition 2005, Intel®
Pentium® M, 1600MHz, 590MHz, 512RAM, Microsoft Windows XP Operating system).
The stimuli were presented on a 15” CRT colour (LCD) screen on the Tablet PC with
1280 X 1024 pixel resolution. Colour quality was set to the highest at 32 bits and
brightness level was set to the maximum to ensure clarity and recordability. Screensaver
was turned off. There was no glare or reflections on the screen since there were no bright
lights in subjects’ visual field. Although there was a window near the workstation, the
Venetian blind with half closed slits were pulled down throughout the experiment to filter
out direct sunlight.
60
Figure 4: A bird-eye view of the room set-up for the experiment sessions where the
experimenter (separated from the participant by the dividing wall) was able to view the exact screen display that the participant was seeing and working on.
2.4.3. Morae Recorder (2004)
Morae Recorder (2004) on the Tablet PC was used to record and save all actions
performed by the participant on the computer including input from the mouse, pen and/or
keyboard, visible on the screen.
2.4.4. Inkit and the programming of beautification functions
The computer program that was used in the study is called Inkit – an informal
design tool developed by the Human Computer Interaction Group in the Computer
Science Department at Auckland University (see Chung, Mirica, & Plimmer, 2005; Tang,
61
2005; Young, 2005; for more detailed descriptions of Inkit, its architecture and codes).
Since the study was exploratory in nature, despite that Inkit was, at the time of the study,
still at the development and testing stage for more complex features (other than stable
basic computer functions such as save, open, copy, paste, drag and drop, resize etc), Inkit
served as a useful tool for the main purpose of the study. And thus, beautification
functionalities required in informal sketch-based tools based on previous research on
Wang, Sun & Plimmer, 2005), were explored and implemented with codes written in C#,
using Inkit (March, 2006) as the fundamental building block. During beautification,
objects (drawn ‘shapes’) are first recognized by ‘recognition engine’ to identify what type
of ‘things’ they are – for example, in terms of user interface, shapes could be recognized
as textboxes, labels, dropdown menus, radio buttons, buttons etc. A blue rectangle that
wraps tightly around the object would then appear, and beautification occurs at this point
by manually selecting (and combining) beautification functions listed on the menu: 1)
Horizontal Alignment; 2) Vertical alignment; 3) Standardization; and 4) Line smoothing at
33%, 50%, 66%, and 100% (some codes available upon request).
2.4.4.1. Horizontal Alignment
The horizontal alignment function aligns objects by first, categorizing which
horizontal group each object belongs to, then calculating the average bottom point of each
horizontal group according to bottom point of the objects’ surrounding rectangle within
the same group, and finally moving the objects to its aligned positions. So at the end,
objects belonging to the same horizontal group would sit on the same point on the y-axis,
with its original points on the x-axis retained. See Figure 5 below for an illustration of
horizontal alignment.
62
Figure 5. Horizontal alignment of elements: (A) the original hand drawn elements; (B) the blue rectangle that wraps tightly around each object indicate that each object has been recognized as an user interface (UI) element; (C) the elements are aligned horizontally when the “horizontal alignment” button is clicked – the bottom point of the blue rectangle is aligned to the same point on the y-axis.
2.4.4.2. Vertical Alignment
(A)
(B)
(C)
63
The same goes to the vertical alignment function but with the reversed concept –
each object is categorized into its vertical group, then the average left point of each
vertical group according to the left point of the objects’ surrounding rectangle within the
same group, and objects were moved to its aligned positions. So at the end, objects that
belong to the same vertical group would be aligned to the left on the same point on x-axis,
with its original points on the y-axis retained. See Figure 6 below for an illustration of
vertical alignment.
Figure 6. The same steps in Figure 111 occur in vertical alignment where objects are first recognized by the recognition engine, and recognized objects are shown within the blue rectangular bounding box. The elements are aligned vertically when the “vertical alignment” button is clicked, shown in the diagram where the bottom point of the blue rectangle is aligned to the same point on the x-axis.
2.4.4.3. Standardization
The size (height and width) of each object that belongs to the same object group
(for example, textbox, dropdown menu, radio button, and label) would be the closely
approximated after standardization. An algorithm for standardization was designed where
the sizes of objects in each group are averaged and also calculated by a specific numerical
factor for normalization purposes. See Figure 7 for an illustration of standardization of the
size of objects.
64
Figure 7. Standardization of the size of elements: (A) the original hand drawn elements; (B) the blue rectangle that wraps tightly around each object indicate that each object has been recognized as an UI element; (C) the elements are standardized when the “standardize” button is clicked – the elements are standardized to a closely approximated size (height and width) within the blue rectangle; (D) additionally, vertical alignment and horizontal alignment are applied.
2.4.4.4. Line Smoothing
Smoothing of lines was achieved through designing algorithms that involved
identifying all the points on the line, flattening (smoothing) the line by mathematically
shifting the points (in terms of x and y coordinates) to a closer approximation to the
corresponding point on the line of the mathematically generated object based on the
corners. In other words, the smoothing function systematically straightens lines of hand-
drawn objects at different levels – 33.3%; 66.6% and 100%. One of the assumptions
made, as described below in 2.5.2, was that as the lines look smoother (straighter and more
computerized), the diagram will appear more formal. In this study, smoothing was
possible for textboxes, dropdown menus and radio buttons. See Figure 8 for an illustration
of different levels of smoothing hand-drawn lines objects to represent different levels of
formality.
(A) (B)
(C) (D)
65
Figure 8. Levels of smoothing of hand-drawn objects (lines) to represent levels of formality: (A) the original hand-drawn objects, representing low formality; in (B), (C) and (D), lines in grey denotes the original hand-drawn objects and lines in black are the original lines that have been 33.3%, 66.6% and 100% smoothed, respectively. (A) Represents low formality; (B) represents medium-low formality; (C) represents medium-high formality; and (D) represents high formality.
However, there has been very little methodological support (in terms of mathematical
and computer programming) for rendering hand written electronic ink to computer fonts
still in the form of electronic ink. Despite the lack of research, attempts to smooth
(morph) characters and/or words in a label were made through techniques such as
mathematically mapping and transforming points, and also manipulating a label’s entity –
but results of such techniques did not reach a satisfactory level for the experiment. Like
Pomm and Werlen’s (2004) exploration on hand writing morphing techniques, beautifying
hand-written texts could be done as an individual study. This is particularly true given the
tight time frame for this study. An alternative method was used to achieve different levels
of formality of text for the purpose of this study by programming specified fonts.
(A)(A) (C)(C)
(B)(B) (D)(D)
66
For the lowest level of formality, the experimenter’s ‘normal’ hand writing using
pen-input was used (i.e. characters not exactly aligned horizontally and vertically, spacing
between characters and words was not exact and each character looked slightly different
every time). As for achieving medium-low formality, a new font – the experimenter’s
computerized (standardized) handwriting – was created by using My Font Tool for Tablet
PC (Lanier, 2004). Handwriting of individual letters from ‘a’ to ‘z’ both in capital and
lower case letters, as well as numbers and some common symbols such as comas, full
stops, exclamation marks, question marks etc, were recorded through pen-input. Character
spacing, word spacing, and line spacing were adjusted, and the final step was to compile
the data to create the new standardized handwriting font to be installed on the tablet PC
(see Appendix G for some screen shots during font creation). This tool ensured that, when
the labels were created, characters were aligned exactly according to the base-line, mid-
line and roof-line horizontally and vertically; character spacing and word spacing were
exact horizontally and vertically, and each character was standardized so that the same
look-and-feel of the character was used. With such mathematical and physical
manipulation of font properties, visually, the computerized characters (words) looked
more formal than the handwritten characters (words).
Although there has been no direct and conclusive empirical evidence to determine
‘font formality’, much of the research on fonts has been on text/font legibility and
readability. For example, Arditi (2004) looked at the effects of customized fonts (varying
in size, serifs and san serifs) by forty visually impaired users and found that different
individuals produce different distinct fonts that resulted in enhanced legibility. However,
no comparisons in legibility were made between the customized fonts and the highly
legible standard fonts such as Times New Roman. In other studies, for example Arditi and
Cho (2005), font properties were manipulated by the experimenters. In Arditi and Cho’s
(2005) experiment, lower-case fonts varying only in serif size (0%, 5%, and 10% cap
height) were used, and legibility was accessed using size thresholds and reading speed. It
was found that serif fonts were slightly more legible than sans serif, but had no effect on
reading speeds (rapid serial visual presentation and continuous reading speed). No
difference in legibility was found when typefaces differed only in the presence or absence
of serifs. Other studies on the effects of font typography examines cognitive performance
such as information recall (Gasser, et al., 2005); visual search and information retrieval in
web pages (Ling & van Schaik, 2006); letter recognition (Sanocki, 1998) and classic
experiments on reading comprehension (Poulton, 1965). Furthermore, web usability
studies on fonts also examined legibility and performance. For example, Bernard, et al.
67
(2001) examined popular online fonts – ornate fonts (e.g. Bradley and Corsiva), sans serif
fonts (e.g. Arial, Verdana, Tahoma etc) and serif fonts (e.g. Courier New, Georgia, Times
New Roman, etc) – and found, also, no difference in legibility between the font types,
however, performance, in terms of reading time, was different. Bernard, et al. also
examined perception of various fonts and found that Courier, Commic, Verdana, Georgia,
and Times New Roman were perceived as the most legible font types. In addition,
aesthetic appeal related to specific font types were also explored, and found that Courier
and Time New Roman (serif fonts) were perceived as being the most business-like,
whereas Comic (sans serif font) was perceived as being the most fun and youthful. In the
field of advertisement, it was suggested that san serifs (non-serif) fonts looked more
playful, youthful and fun (also suggested by Bernard, et al. 2001), and have now become
more popular in terms of its use in logos for multi-national brands (e.g. fast-food
companies) to small businesses (e.g. retail shops).
In addition to the low formality and medium-low formality fonts represented by
non-computerized and computerized handwriting on the tablet PC, the fonts to represent
medium-high formality and high formality were also determined. With no conclusive
evidence on font formality, it was reasoned, therefore, an uncommon, san serif font (in this
case, Gulim) was to be used to represent beautified handwriting at medium-high level of
formality; and as there were some findings supporting the use of Times New Roman as a
common legible typeface compared to san serif fonts, Times New Roman was used for
representing fully beautified handwriting at high level formality. See Figure 9 below for
an illustration of the four fonts used to represent different levels of formality.
How are you? (Times Roman Numeral)
How are you? (Gulim) How are you? (Experimenter’s standardized Handwriting)
Figure 9. The four fonts used to represent different levels of formality: Times Roman Numeral – represents high formality; Gulim – representing medium-high formality; standardized handwriting – representing medium-low formality; and un-standardized handwriting on the tablet PC – representing low formality.
There are many variables that play a role in producing different typeface. While
every effort was made to control for common variables affecting typeface, as suggested by
68
Watzman (2005), including size, letter spacing, word spacing, line spacing and line
justification in all five designs, we can not be sure that the fonts used in this study to
represent the various levels of formality were an accurate representation of the levels.
This is acknowledged as a limitation of the study.
2.5. Stimuli and Materials
2.5.1. Instruction Sheets
In each condition, subjects were given a design of an online form to work on and to
make any changes to improve the design in terms of functionality and its purpose. In each
condition, an instruction sheet containing the requirements and the scenario (see Appendix
F for each set of instruction sheets associated with each design) was given before
presenting the associated design to the participant. The same format was used for all five
instruction sheets: (identical) instructions were printed at the beginning on each instruction
sheet, followed by the requirements and scenario associated with the design presented.
The purpose of including both the requirements and the scenario for each condition was to
help produce a more realistic design situation – requirements collected (Maybew, 2003)
and scenarios formed (Rosson & Carroll, 2003) in the early stages of the design process to
help shape the final product (although in a laboratory environment). In particular, with
respect to the conditions, the requirements helped the participant to identify whether the
correct information was being ‘collected’ from the end-user (who would be filling in the
HTML form), which in turn, guided the participant to add, delete and relocate
elements/items and/or item sets appropriately. The scenario, in addition, points out
whether an element was of the appropriate type (change element) and size (resize).
2.5.2. The five designs each representing a different level of formality
There were a total of five equivalent designs, and hence, a total of five conditions.
Variables had to be controlled between the five designs to increase validity of results so
we are measuring the effects of formality on design decisions. Hence, the designs were
made as equivalent as possible, in terms of: 1) the purpose of the forms – requiring users
to fill in personal information – hence, common online forms; 2) order of elements in the
design (i.e. textboxes, radio buttons, dropdown menus, and labels in the same order in
each design); 3) the balance of types of element (i.e. 12 textboxes, 10 radio buttons, 5
dropdown menus and 31 labels in each design), and thus, 4) the number of elements in
69
each design (i.e. total of 58). Within the design, the sizes of elements were also
controlled for by having closely approximated physical measurements i.e. textboxes all
had an approximate height of 1cm and a width of 8.5cm (and 5 textboxes with half the
width) at 100% screen radio; similarly for dropdown menus, with a triangle with a height
and width of 50mm on the right hand side within the dropdown menu; all radio buttons
had an approximate diameter of 1cm (sizes could not be perfectly exact due to different
levels of smoothness/roughness of beautified lines); labels had an approximate height of
1cm, however, the width of labels was uncontrollable as word length and the number of
words contained in a label varied. Mathematical standardization of element groups was
achieved by applying the standardization function in the beautification menu list.
Additionally, all labels were programmed to space 50 pixels apart vertically (on the y-
axis) and their associated controls (textbox, dropdown menus and radio buttons) were
aligned to the labels horizontally (on the x-axis) at 30 pixels apart.
The beautification variables shown in Table 2 were either controlled (size, spacing)
or systematically varied (alignment, smoothness). Alignment and smoothness of hand
drawn elements were combined and varied systematically, as described below, to produce
different levels of formality.
1) Alignment (vertical and horizontal)
According to perceptual theorist (e.g. Gestalt, Marr, Gibson, Rock), factors such as
orientation and grouping of stimuli, affect visual perception of a form. For example, axis-
alignment affects perceptual grouping (Boutsen & Humphreys, 1999); the importance of
balance of objects as an organizing design principle (Locher, Stappers, & Overbeeke,
1998). Therefore, by varying the extent of alignment, elements of the same type (e.g.
textboxes, dropdown menus, radio buttons, and labels) would appear differently grouped
and organized. In addition, according to web usability handbooks and guidelines based on
Gestalt principles (e.g. Brink, 2002), groups of elements and information should be
aligned for grouping purposes and easier comprehension – for example, alignment of
textboxes to textboxes; labels to labels; and “submit” buttons at the centre-bottom of the
page. If elements are unaligned, information becomes scattered, creating complexity and
additional visual features within the design. Brink (2002) such problems will make the
display look excessively cluttered and unprofessional - hence, it will affect the appearance
of a design (the more formal, the more professional a design appears).
Therefore, an assumption was made that different ratios of alignment would
produce different levels of formality of a design. In other words, the more elements
70
aligned the more formal the design will appear. To create different levels of formality in
the designs presented to the participants, elements in the design were aligned
systematically according to mathematic principles of ratios as shown below in Table 3.
Table 3. Ratio of elements aligned (and its description) in each design representing a different level of formality.
Formality Ratio Alignment descriptionLow formality (paper) 3:0 No elements are aligned exactlyLow formality (tablet PC) 3:0 No elements are aligned exactlyMedium-low formality 3:1 From top to bottom, and from left to right, every third
element is aligned to the element at the top that belong to the same group e.g. label to label, textbox to textbox, radio button to radio button, label to label)
Medium-high formality 3:2 From top to bottom, and from left to right, every second element out of three elements are aligned to the element at the top that belong to the same group e.g. label to label, textbox to textbox, radio button to radio button, label)
High formality 3:3 Every element aligned exactly according to its type
2) Smoothness of lines (lines that make up textboxes, dropdown menus, radio buttons
and labels)
Although there has been some studies on beautification of hand-drawn sketches in
no direct empirical evidence regarding smoothness of lines affecting formality was found.
However, from everyday examples, it can be noticed that smoothness of lines can be an
important factor that influence the appearance of objects. For example, it is natural to want
to see objects such as a black board, a table, walls, with smooth edges (aesthetically
pleasing), rather than rough, uneven edges, which may appear to be an unfinished product;
in addition, printed text may appear to be more formal than hand written text, which may
appear to be unfinished).
Therefore, a rational and valid assumption was made – that a computerized straight
line (without bumps) appear more formal than a hand drawn line (with bumps), and hence,
that the smoother the line (including lines that make up text), the more formal it will
appear. To create different levels of formality in the designs presented to the participants,
lines were smoothed systematically, as shown below in Figure 4.
Table 4. Systematic smoothing applied (% smoothed) to the original hand-drawn lines of textboxes, dropdown menus and radio buttons; and fonts used for labels to represent different levels of formality in the designs presented to the participants.
Low formality (paper) 0.0 Non-beautified original hand writing on paperLow formality (tablet PC) 0.0 Non-beautified original hand writing on the tablet PCMedium-low formality 33.3 Standardized original hand writing on the tablet PCMedium-high formality 66.6 GulimHigh formality 100.0 Times New Roman
When the two beautification variables described above – alignment and smoothing
of lines, were combined and varied systematically, and keeping the other two
beautification variables constant (size and spacing), equivalent designs (as described
above) that appeared more or less formal were created. In other words, different levels of
formality were produced by systematically varying alignment and smoothness of lines,
and hence, beautification taxonomy was successfully developed and two beautification
variables were tested and validated. Figure 10.1, 10.2, 10.3, 10..4 and 10.5 below shows
each design representing a different level of formality, with one low formality design
presented on paper, and four designs from low formality to high formality presented on the
tablet PC. Each design was presented one after another (according to the order of
presentation) to the participants to work on, with the instruction to improve each design by
making changes to them.
72
Figure 10.1. Low Formality (on paper) – Online Magazine subscription with 0% smoothed lines i.e. all original hand-drawn design on paper; non-computerized ‘normal’ ‘natural’ writing (of the researcher’s); some form of rough non-computerized vertical and horizontal ‘alignment’ with 0:3 ratio of exact (computerized) alignment of elements.
Figure 10.2. Low Formality (on tablet PC) – Samson’s Bank $1 Million Loan Application; 0% smoothed lines i.e. all original hand-drawn input, non-computerized ‘normal’ ‘natural’ writing (of the researcher’s); some form of rough non-computerized vertical and horizontal ‘alignment’ with 0:3 ratio of exact (computerized) alignment of elements
73
Figure 10.3. Medium-Low Formality – Graduation Application Form with 33% smoothed lines; characters in standardize hand writing (of the researcher) created using My Font Tool For Tablet PCs (2005); font size 18; vertical and horizontal alignment ratios of 1:3 in arithmetic order.
Figure 10.4. Medium-High Formality - Dog Registration Form with 66% smoothed lines; characters in Gulim font i.e. without serifs; font size at 18; vertical and horizontal alignment ratio of 2:3 in arithmetic order
74
Figure 10.5. High Formality design - America’s Next Top Model Application Form with: 100% smoothed lines i.e. perfect straight lines; characters in Times new Roman font i.e. With Serifs; font size at 18; vertical and horizontal alignment ratio of 3:3 (all elements aligned)
2.5.3. Post-task Questionnaire
A post-task questionnaire (see Appendix B) was used for recording participants’
demographic information such as age, gender, education level, programme studied, papers
taken, occupation, and design experience. Preference for design tools (pen and paper
verses the tablet PC) during the design tasks and in real-world design situation, as well as
participant’s “overall enjoyment” when working on each design in comparison to another
(by ranking from 1 to 5, from the most-liked design to the least-liked design; and the
reasons for the rankings) were also recorded in the questionnaire. Such information was
used to explore whether performance – in this case, design decisions to improve a design
at different levels of formality, was affected by factors such as design experience (e.g.
Cross, 2004,; Kavakli & Gero, 2002), study major/specialization and study level (e.g.
Atman, et al., 2005; Atman, et al., 1999) and design medium preference (e.g. Bailey &
Konstan, 2003; Black, 1990; Hann & Barber, 2001; Newman, et al., 2003), during the
design process.
75
Chapter 3. Results
For the purpose of analysis, design performance was measured in terms of number
of (functional) changes made. In other words, one dependent variable with three levels
(the total number of changes, quality changes and expected changes) was measured at
each level of the independent variable (formality). Subjective measures included overall
enjoyment rankings of designs, design tool preference. Data were analyzed using SPSS
for Windows version 14.0 (SPSS Inc.). Analysis of variance (ANOVA) with repeated
measures and unplanned pair-wise comparisons were conducted to analyze the effects of
formality on outcome measures (number of changes made). Between-subject effects were
also analyzed. Friedman’s rank test for several related samples were conducted to analyze
subjective measures including rankings of designs.
3.1. Data-screening of performance data
In order to test whether the data satisfied the normality assumptions for a
parametric repeated samples t-test (see Cohen, 1988), histograms, homogeneity of
variance, the skewness and kurtosis statistics, and normality tests and plots for the scores,
as well as assumptions of sphericity were examined. For all normality information on
total changes, quality changes and expected changes, see Appendixes H, I and J
respectively.
The histograms with normal curves were created for the mean scores of each of the
three dependent measures: 1) total changes (see Appendix H), 2) quality changes (see
Appendix I) and 3) expected changes (see Appendix J), across all five levels formality.
Initial visual inspection showed roughly normal distributions and a few slightly skewed
distributions. However, it was inadequate to conclude that the distribution was non-
normal from the skewed data from a small sample of n = 30 (Cohen, 1988). Thus, all data
were also plotted against the standardized version of the data and showed that the scores
were normally distributed at each level of formality i.e. a roughly linear relationships (see
normal Q-Q plots for level of formality in Appendix H, I and J).
The comparison of variance between levels of formality in each category of change
made, indicated that the scores were similar enough (Coakes & Steed, 2001), therefore the
homogeneity of variance assumption for each measure group (DV) was not violated. See
variance across level of formality in Appendix H, I and J.
The 95% confidence interval around the skewness and kurtosis scores of neither
cell included zero indicating that the scores were not normally distributed, with data
76
skewed both ways in positive and negative directions, with negative and positive values
for kurtosis indicated leptokurtic and platykurtic distribution (Heimanz, 2001). However,
looking at the values for skewness and kurtosis in each level of formality, the absolute
values of skewness and kurtosis statistics were almost always smaller than the standard
errors, indicating that the skewness and kurtosis were comparable with the zero value in a
normal distribution.
In order to examine the relationships between the skewness, kurtosis and variance,
the Kolomogorov-Smirnov (KS) and Shapiro-Wilk (SW) tests of normality were both
conducted (see the test of normality in Appendix H, I and J). KS tests (with Lilliefors
Significance Correction) assessed the kurtosis and skewness of each data group,
demonstrating that data were suitable for parametric testing (i.e. normally distributed) as
KS statistics at each level of formality showed no significance (p > .50). SW, calculated if
sample size is less than fifty (Coakes & Steed, 2001), also showed similar values, further
indicating that the data did not violate the normality assumption of parametric tests.
Moreover, there were no missing data, and there were no outliers except for one in
the low formality scores (participant 10) in the quality changes data group. However, this
outlier was included in the analysis as it fell within the upper quartile range in the other
levels of formality (see Figure 5 in the analysis section for quality changes for the box plot
of quality changes made at each level of formality).
Overall, therefore, the data were reasonably normally distributed, hence, normality
assumptions not violated and it was justifiable to use parametric analysis.
3.2. Analysis of performance data: One-way repeated measures ANOVA
One-way ANOVA with repeated measures were conducted to examine each
dependent variable (total change, quality changes and expected changes) under each
independent variable (level of formality). An alpha level of 0.5 was used for all statistical
tests.
Sphericity
Before analyzing one-way ANOVA with repeated measures for each type of
change made across levels of formality, sphericity had to be examined. According to
Brace, Kemp and Snelgar (2006), the assumption of sphericity is that the correlations
between all of variables are roughly the same – in other words, the null hypothesis in the
sphericity assumption is that the correlations among the number of changes made in each
77
level of formality are equal. Moreover, Tabachnick and Fidell (2001) suggested when
there are more than two levels of IV (in this case, formality) the test of sphericity must be
conducted to decide which test should be used to interpret significance. Therefore, the
Mauchly’s test of sphericity was conducted on each data group. The test was significant
for total change (Approx. Chi-square = 24.644, p < .05), indicating that the sphericity
assumption was violated. On the other hand, the test yielded no significance in both
quality changes (Approx. Chi-square = 10.004, p = .351) and expected changes (Approx.
Chi-square = 5.170, p = .820), meaning that the null hypothesis of sphericity was
accepted. Hence, the assumptions of sphericity were met and the normal within-subjects
ANOVA was not violated for both data groups.
3.2.1. Analysis of “Total Changes” made across levels of formality
Table 5 shows mean and standard deviation of total changes made at each level of
formality.
Table 5Mean and standard deviation for total changes made at each level of formality
Mean Std. Deviation1. Low formality (on paper) 18.73 6.572. Low formality (on tablet PC) 15.17 4.143. Medium-low formality 14.00 4.074. Medium-high formality 13.13 3.865. High formality 11.27 3.51
Since the Mauchly’s test of sphericity (for the data group of total change) was
significant, an alternate, multivariate approach was adopted (Brace, Kemp & Snelgar,
2006). The results for the ANOVA indicated a significant main effect of formality on the
total changes made to the designs, Wilk’s Lambda = .265, F (4.26) = 17.99, p < .001,
multivariate partial η2 = .74. A strong significant linear trend was also found, F (1, 29) =
59.59, p < .001, partial η2 = .67, over the mean value of total changes made at each level of
formality (illustrated in Figure 11). A weaker but significant cubic trend was also found, F
(1, 29) = 8.529, p < .01, partial η2 = .23, suggesting that overall, the number of total
changes made were the highest when participants were presented with the low-formality
design on the paper, and decreased as formality increased. Subjects’ performance was
slightly unstable, however, suggesting that order effects (discussed later) may have played
a role in contributing to the combination of linear and cubic trends.
78
0
2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
Low formality(paper)
Low formality Medium-lowformality
Medium-highformality
High formality
Levels of formality
Tota
l cha
nges
Mean total change
Figure 11. Multi-line graph showing mean total changes made across levels of formality which is represented by the black bold line; each participant’s performance (in terms of total changes made across levels of formality) is also illustrated – see individual lines.
As there was lack of empirical research with only one previous study (Plimmer,
2002) on the effects of formality on the design process, one could not precisely predict
what conditions would differ from each other and in what direction – therefore, as Brace et
al. (2006) suggested, unplanned pair-wise comparisons were conducted to examine the
differences between the mean total changes at each level of formality.
Pair-wise comparisons (with Bonferroni adjustment for multiple comparisons)
showed that the total number of changes made was significantly lower when participants
were presented with the high formality design, compared to other designs with lower
levels of formality: medium-high formality; medium-low formality; low formality on the
tablet PC and low formality on paper. Differences increased as the level of formality
decreased, as shown in Table 6. On the other hand, the total number of changes made in
the low formality design presented on paper was significantly higher than other levels of
formality presented on the Tablet PC: low formality on the Tablet PC; medium-low
formality; medium-high formality and high formality. Differences increased as the level
of formality increased, also shown in Table 6. Interestingly, even though there were two
79
low formality conditions, one presented on paper and one presented on the tablet, the total
number of changes made still differed significantly between these conditions – the mean
difference was 3.57 as can be seen in Table 6. This was also shown in Figure 11 where
the mean number of total changes made was much greater when made on paper than on
the Tablet PC. Furthermore, as shown in Table 6, no significant difference was found, in
terms of mean total changes, between medium-high formality and medium-low formality;
and between medium-low formality and low formality on the Tablet PC. However, the
total number of changes made at low formality was significantly higher than at medium-
high formality.
Table 6Mean differences and their significance at the .05 level in terms of total number of changes made between each condition.
(I) Factor 1 (J) Factor 1Mean Difference
(I-J)Low formality (on paper) Low formality (on Tablet PC) 3.57* Medium-low formality 4.73* Medium-high formality 5.60* High formality 7.47*Low formality (on Tablet PC) Low formality (on paper) -3.57* Medium-low formality 1.17 Medium-high formality 2.03* High formality 3.90*Medium-low formality Low formality (on paper) -4.73* Low formality (on Tablet PC) -1.17 Medium-high formality 0.87 High formality 2.73*Medium-high formality Low formality (on paper) -5.60* Low formality (on Tablet PC) -2.03* Medium-low formality -0.87 High formality 1.87*High formality Low formality (on paper) -7.47* Low formality (on Tablet PC) -3.90* Medium-low formality -2.73* Medium-high formality -1.87*
* The mean difference is significant at the .05 level.
3.2.1.1. Between-Subject Factors
In order to examine whether other factors affected the total changes made at each
level of formality, between subject effects including design experience,
major/specialization and study level were explored. Furthermore, each between-subject
factor had only two levels therefore no post-hoc tests were necessary.
3.2.1.1a. Design experience
Subjects’ design experience was examined first as it was hypothesized that there
would be a difference in the total number of changes made across levels of formality
80
between subjects who had more or less design experience. The subjects were categorized
into two groups: 1) subjects with no experience or some non-computer science/software
engineering design experience (n = 15); and 2) subjects with computer science (CS) /
software engineering (SE) design experience (n = 15). Table 7 shows mean and standard
deviation of total changes made at each level of formality according to subjects’ design
experience.
Table 7Mean and standard deviation for total changes made, and the mean difference between groups, at each levels of formality according to subjects’ design experience (total n =30): none to some (non-CS/SE) design experience (n = 15) and CS/SE design experience (n = 15)
Results from the ANOVA with design experience as the between-subject factor
showed that there was a significant formality-by-design experience interaction effect, F (4,
112) = 6.24, p < .001, partial η2 = .18, and a significant between-subject effect of design
experience, F (1, 28) = 8.49, p < .01, partial η2 = .23, on the total changes made across
levels of formality. This indicated that the total changes made across levels of formality
differed between subjects with no experience to some non-CS/SE design experience and
subjects with CS/SE design experience – and more specifically, subjects with CS/SE
design experience made consistently more changes across levels of formality compared to
subjects with no experience or some non-CS/SE experience (see Figure 12).
None to some (non-CS/SE)design experienceCS/SE design experience
Figure 12. Multi-line graph of mean total changes made across levels of formality according to subjects’ design experience: none to some (non-CS/SE) design experience and CS/SE design experience
A significant linear trend was also found when the effects of formality and design
experience were combined, F (1, 29) = 10.43, p < .005, partial η2 = .27. This suggested
that the linear trend was also significant in both groups, where subjects made less (more)
changes as the level of formality increased (decreased), regardless of magnitude
differences. There were between-group differences across levels of formality, as shown in
Table 7, and differences tend to decrease as formality increased (refer also to Figure 12).
There was also a significant but weak formality-by-design experience quadratic trend, F
(1, 29) = 4.41, p < .045, partial η2 = .14, and cubic trend, F (1, 29) = 4.50, p < .043, partial
η2 = .14. Such trends are also illustrated in Figure A, where the total changes increased
gradually from high to low formality on the Tablet but more markedly higher at low
formality on paper (more detectable in subjects with CS/SE design experience); points of
increase are detectable in the negative linear trend from low formality to high formality.
Two other between-subject factors – major/specialization and study level, were
explored mainly through visual inspection due to various reasons: the number of subjects
in each group could not be balanced; there was overlapping of subject factors, i.e. explicit,
isolative (i.e. nested) grouping of subjects was near impossible in the current study as
major/specialization, study level and design experience were all intimately-correlated, and
even if it were possible, a much larger sample would have been needed – therefore
82
subjects were grouped according to one factor only, and grouped data was examined
through multi-line graphs.
3.2.1.1b. Study major/specialization
Since the experimental task involved HTML (web) form design, it was of interest
to see whether total changes made across levels of formality differed between subjects
who had more or less HTML knowledge. Therefore to explore such between-subject
effect, subjects were grouped into two groups: 1) subjects with a non-CS/SE related major
(n = 10); and 2) subjects with a CS/SE major (n = 20). Table 8 below shows mean and
standard deviation of total changes made in each group across levels of formality.
Table 8Mean and standard deviation for total changes made, and the mean difference between groups, at each level of formality according to subjects’ major/specialization in university (Total n =30): non-CS/SE related major (n = 10) and CS/SE related major (n = 20)
Results from the one way ANOVA with study major as the between-subject factor showed
that there was a significant formality-by-major/specialization effect, F (4, 112) = 4.10, p
< .01, partial η2 = .13, as well as significant formality-by-major/specialization trends:
linear trend, F (1, 28) = 5.09, p = .032, partial η2 = .15, and quadratic trend, F (1, 28) =
5.37, p < .028, partial η2 = .16; however, no significant between-subject effect was found
(illustrated in Figure 13). Visual inspection of Figure 13 suggested that there was linear
trend across levels of formality, and the total changes made increased more rapidly at the
lower formalities in the CS/SE major group, while the other group showed a less
consistent linear trend. Interestingly, at medium-low formality subjects performed at the
same level – the total changes made was similar in the CS/SE major group and the non-
CS/SE major group. This could also explain the non-significant results from the between-
subject effects tests. Overall, the between-groups difference decreased as the formality
level increased (see mean differences in Table 8) – the gap between the two lines was
smaller at the higher levels of formality compared to lower levels of formality, as
explained by the significant formality-by-major/specialization interaction.
Major/Specialization(X) Non-CS/SE related major (Y) CS/SE related major
Figure 13. Multi-line graph of mean total changes made across levels of formality according to subjects’ major/specialization in university: Non-CS/SE related major and CS/SE related majors
3.2.1.1c. Study Level
As study level may have also played a role in producing particular trends among different
groups, subjects were classified into two groups: 1) undergraduates (n = 22); and 2)
graduates/post-graduates (n = 8). Table 9 below shows mean and standard deviation of
total changes made in each group across levels of formality.
Table 9 Mean and standard deviation for total changes made, and the mean difference between groups, at each level of formality according to subjects’ study level (total n=30): undergraduate (n=22) and graduate/postgraduate (n=8).
No significant statistics, such as formality-by-study level effect and trends, were
found from the results of ANOVA with study level as the between-subject factor, except
for between-subjects effects, F (1, 28) = 6.10, p = 0.02, partial η2 = .18. However,
examining statistics alone was not conclusion there was an unbalanced number of subjects
in each group. Visual inspection of Figure 14 suggested that there was a strong linear
trend in the undergraduate group, while the graduate/postgraduate group showed a weaker
linear trend with a slight increase in the mean total change at medium-high formality, after
the medium-low formality condition. It was also visible that the total changes made
increased rapidly at low formality (on paper) in both groups. Also, the between-subject
differences appeared to be greater nearer the two ends of the formality spectrum: low
formality on paper (mean group difference = 5.55) and low formality on the Tablet PC
(mean group difference = 3.49); and medium-high formality (mean group difference =
4.43) and high formality (mean group difference = 3.30); and the smallest between-group
difference at medium-low formality (mean group difference = 1.33) – see mean
differences in Table 9. This also suggested that there was some formality-by-study level
interaction. Although differing in magnitude, overall, a rough linear trend was visible for
both groups – as formality increased (decreased), total changes made decreased
(increased).
0
2
4
6
8
10
12
14
16
18
20
22
24
Lowformality(paper)
Lowformality
Medium-lowformality
Medium-highformality
Highformality
Levels of formality
Mea
n to
tal c
hang
es
Undergraduate
Graduate/postgraduate
Figure 14. Multi-line graph of mean total changes made across levels of formality according to subjects study level: undergraduate and graduate/postgraduate.
3.2.1.2. Multiple Regression analysis
The similar trends with the three factors further suggested that they were closely
related. The data set was re-grouped according to a combination of design experience,
study level and major/specialization (see Appendix K), and multiple regression analysis
85
was then conducted to examine and separate individual effects that contributed to the
overall effect of formality on the total changes made. In other words, these analyses
sought to discover how much each between-subject factor helped explain the effect of
formality on the total changes made.
Formality and the three between-subjects variables (design experience and study
level, and major/specialization) were entered one after the other respectively into SPSS.
Before looking at the actual results, in addition to the data screening earlier for normality
and outliers, multicollinearity was first examined. According to Brace et al. (2006), the
closer to zero the tolerance value is for a variable (vary between 0 to 1), the stronger the
relationship between this and the other predictor variables; and the higher the VIF value
(value from 1.0), the stronger relationship is between predictor variables; and such values
becomes a worry. However, results indicated high tolerance values (over .90), and low
VIF values (less than 1.08), therefore there was no multicollinearity issues.
Using the stepwise method, a significant model which included formality, design
experience and study level, emerged, F (3, 31) = 31.67, p < .0001. The model explained
73%% of the variance (Adjusted R2 = .730). Table 10.1 shows the adjusted R square and
change statistics of each predictor when added to the model. Formality level (model 1)
accounted for 36.1% of the variance (Adjusted R2 = .361, p <.0001), and the inclusion of
design experience in model 2 resulted in an additional 30.1% of the variance being
explained (R2 change = .301, F (1, 32) = 30.13, p < .0001). Study level helped explained a
further 7.4% of the variance when added upon formality and design experience (R2 change
= .074, F (1, 31) = 9.95, p = .005). However, study major/specialization was excluded
from the model as it did not have a significant impact when added (R2 change = .00, F (1,
30) = .003, p = .96) – hence, not a good predictor to explain total changes made across
levels of formality.
Table 10.1. R, Adjusted R Square and R Square change for total changes made across levels of formality, with formality level, design experience, study level and major/specialization as predictors entered
Model RAdjusted R
SquareStd. Error of the Estimate Change Statistics
a Predictors: (Constant), Formality Levelb Predictors: (Constant), Formality Level, Design experiencec Predictors: (Constant), Formality Level, Design experience, Study leveld Predictors: (Constant), Formality Level, Design experience, Study level, Major/specialization
Table 10.2 gives information for the predictor variables (formality and between-
subject variables) included in the significant model. The result suggests that formality
alone (the manipulated variable) has a strong significant impact on the total number of
changes made (β = -.62, t = -6.92, p < .0001). The negative statistics further suggests that
as formality level increases, the total changes decreases. The results for design experience
(β = .5, t = 5.57, p < .0001) and study level (β = .28, t = 3.05, p < .005) further indicates
that on top of the effects of formality on total changes made – people with more design
experience and/or at a high level of study (e.g. graduates) are more likely to make greater
number of changes than those with less design experience and/or at a lower level of study
(e.g. undergraduates).
Table 10.2 The unstandardized and standardized regression coefficients, and the t-value and significance of each between-subject variables included in the mode for explaining total changes made.
3.2.2. Analysis of “Quality Changes” made across levels of formality
Table 11 shows mean and standard deviation of quality changes made at each level
of formality.
Table 11Mean and standard deviation for quality changes made at each level of formality.
Formality level Mean Std. Deviation1. Low formality (paper) 15.73 5.502. Low formality (on tablet PC) 13.05 4.013. Medium-low formality 12.90 3.814. Medium-high formality 10.80 3.925. High formality 9.02 3.54
Since the Mauchy’s test of sphericity was not significant, the traditional test of
within-subjects effect was conducted and the results from ANOVA showed that there was
87
a significant main effect of formality on the number of quality changes made, F (4, 116) =
31.763, p < .001, partial η2 = .48. A significant linear trend was also found, F (1, 29) =
76.91, p < .001, partial η2 = .73, over the mean quality changes at each level of formality,
indicating that subjects made most quality changes in the low formality design on paper,
followed by low formality design on the Tablet PC, and the numbers dropped as formality
increased – see Figure 15; and also refer to the bold line for the means across levels of
formality. However, no significant quadratic, cubic nor order 4 trends were found.
0
2
4
6
8
10
12
14
16
18
20
22
24
26
Low formality(paper)
Low formality Medium-lowformality
Medium-highformality
High formality
Levels of formality
Qua
lity
chan
ges
Mean quality changes
Figure 15. Multi-line graph showing mean quality changes made across levels of formality which is represented by the black bold line. Each participant’s performance (in terms of quality changes made across levels of formality) is also illustrated – see individual lines.
As noted before, due to the lack of previous empirical research, unplanned pair-
wise comparisons were conducted to examine the difference in the mean quality changes
between levels of formality.
Pair-wise comparisons (with Bonferroni adjustment for multiple comparisons)
showed that the number of quality changes made was significantly lower when
participants were presented with the high formality design, compared to the designs with
lower levels of formality: medium-high formality; medium-low formality; low formality
on the tablet PC and low formality on paper. Difference increased as the level of formality
decreased, as shown in Table 12. On the other hand, the mean number of quality changes
88
made in the low formality design presented on paper was significantly higher than all
other levels of formality presented on the Tablet PC: low formality on the Tablet PC;
medium-low formality; medium-high formality and high formality. Difference increased
as the level of formality increased, also shown in Table 6. The increasing and decreasing
differences further emphasized the significant linear trend found. Furthermore, the mean
number of quality changes made was different between each level of formality, except
between medium-low formality and low formality on the Tablet PC (mean difference
= .33) –illustrated in Figure 15. It was also interesting to note that, similar to the data for
total changes, even though there were two low formality conditions, one presented on
paper and one presented on the tablet, the number of quality changes made still differed
significantly between these conditions – the mean difference was 2.68 as can be seen in
Table 12. This was also shown in Figure 15 where the mean number of quality changes
made was higher when made on paper than on the Tablet PC.
89
Table 12
Mean differences and their significance at the .05 level in terms of the number of quality changes made between each condition.
(I) Factor 1 (J) Factor 1Mean Difference
(I-J)Low formality (on paper) Low formality (on Tablet PC) 2.68* Medium-low formality 2.83* Medium-high formality 4.93* High formality 6.72*Low formality (on Tablet PC) Low formality (on paper) -2.68* Medium-low formality 0.15 Medium-high formality 2.25* High formality 4.03*Medium-low formality Low formality (on paper) -2.83* Low formality (on Tablet PC) -0.15 Medium-high formality 2.10* High formality 3.88*Medium-high formality Low formality (on paper) -4.93* Low formality (on Tablet PC) -2.25* Medium-low formality -2.10* High formality 1.78*High formality Low formality (on paper) -6.72* Low formality (on Tablet PC) -4.03* Medium-low formality -3.88* Medium-high formality -1.78*
* The mean difference is significant at the .05 level.
3.2.2.1. Between-Subject Factors
In order to examine whether other factors affected the quality changes made at
each level of formality, between subject effects including design experience, study level
and major/specialization were explored. Furthermore, each between-subject factor had
only two levels therefore no post-hoc tests had been conducted.
3.2.2.1a. Design Experience
Subjects’ design experience was examined first as it was hypothesized that there
will be a difference in the number of quality changes made across levels of formality
between subjects who had more or less design experience. Thus, subjects were
categorized into two groups: 1) subjects with no experience or some non-computer
science/software engineering design experience (n = 15); and 2) subjects with computer
mean and standard deviation of quality changes made at each level of formality according
to subjects’ design experience.
Table 13
90
Mean and standard deviation for quality changes made, and the mean difference between groups, at each level of formality according to design experience (total n=30): none to some (non-CS/SE) design experience (n=15) and CS/SE design experience (n=15)
Results from ANOVA with design experience as the between-subject factor
showed that there was a significant formality-by-design experience interaction effect, F (4,
112) = 4.07, p < .005, partial η2 = .13, along with the significant between-subject effects of
design experience, F (1, 28) = 7.31, p < .02, partial η2 = .21, on the number of quality
changes made across levels of formality. This indicated that the quality changes made
across levels of formality differed between subjects with no experience or some
non-CS/SE design experience and subjects with CS/SE design experience. More
specifically, subjects with CS/SE design experience made consistently more quality
changes across levels of formality compared to subjects with none-to-some (non-CS/SE)
experience (see Figure 16).
0
2
4
6
8
10
12
14
16
18
20
Lowformality(paper)
Lowformality
Medium-lowformality
Medium-highformality
Highformality
Levels of formality
Mea
n qu
ality
cha
nges
None to some (non-CS/SE)design experienceCS/SE design experience
Figure 16. Multi-line graph of mean quality changes made across levels of formality according to subjects’ design experience: none to some (non-CS/SE) design experience and CS/SE design experience
Significant trends were also found with the combined effects of formality and
design experience: linear trend, F (1, 28) = 7.04, p < .03, partial η2 = .20; as well as a weak
quadratic trend, F (1, 28) = 4.57, p < .05, partial η2 = .14 – illustrated in Figure 16 where
quality changes increased rapidly at low formality (on paper). This suggested that there
was a linear trend in both groups regardless of magnitude differences, where subjects
made less (more) changes as the level of formality increased (decreased). In addition to
the statistics, Figure 16 shows that there was a stronger linear trend across levels of
formality in the subjects CS/SE design experience however, the linear trend was less
consistent in subjects with none to some (non-CS/SE) design experience. At medium-low
formality, there was an increase in the mean quality changes made by subjects with non-
to-some (non-CS/SE) design experience and thus, between-group difference at such level
of formality was the smallest compared to other levels. The between-group differences at
low formality on paper was the largest (mean difference = 5.20), followed by low
formality on the Tablet PC (mean difference = 3.50), and the mean differences between
groups tended to decrease as formality increased (refer to Table 13 and Figure 16) – this
further highlighted the formality-by-design experience interaction.
Two other between-subjects factors – major/specialization and study level, were
explored primarily through visual inspection of multi-line graphs due to various reasons:
the number of subjects in each group could not be balanced; there were overlapping of
subject factors, i.e. explicit, isolative (i.e. nested) grouping of subjects was near
impossible in the current study as major/specialization, study level and design experience
were all intimately-correlated, and even if it was possible, a much larger sample was
needed – therefore subjects were grouped according to one factor only.
3.2.2.1b. Study major/specialization
Since the experimental task involved HTML (web) form design, it was of interest
to see whether quality changes made at each level of formality differed between subjects
who had more or less HTML knowledge. Therefore subjects were grouped into two
groups: 1) subjects with a non-CS/SE related major (n = 10); and 2) subjects with a CS/SE
major (n = 20). Table 14 shows the mean and standard deviation of quality changes made
in each group across levels of formality.
Table 14
92
Mean and standard deviation for quality changes made, and the mean difference between groups, at each level of formality according to subjects’ major/specialization in Auckland University: Non-CS/SE related major (n = 10) and CS/SE related majors (n = 20)
Results from ANOVA with study major/specialization as the between-subject factor
indicated that there was a significant formality-by-major interaction, F (4, 112) = 3.00, p <
.025, partial η2 = .10, and weak significant trends including a linear trend, F (1, 28) = 4.23,
p < .05, partial η2 = .13, a quadratic trend, F (1, 28) = 4.76, p < .05, partial η2 = .15, on the
number of quality changes made across levels of formality. However, no between-
subjects effects were found. Figure 17 highlighted the significant statistics and showed
that there was a strong linear trend of quality changes made across levels of formality and
a rapid increase at the low formality (on paper) in the CS/SE major group; where as, in the
non-CS/SE major group, there was a weaker linear trend with one non-linear point with
respect to other points. Interestingly, at medium-low formality subjects performed at the
same level – the mean quality changes made was similar in the CS/SE major group and the
non-CS/SE major group. Overall, subjects who majored in CS/SE made more quality
changes than subjects who majored in non-CS/SE areas of study. Between-group
difference (see mean differences in Table 14) was the largest at low formality presented on
paper (mean difference = 3.78) and decreased at low formality present on the Tablet PC
(mean difference = 2.05). Next, as formality level increased, the between-group
differences decreased – the gap between the two lines was smaller at the higher levels of
formality compared to lower levels of formality which further suggested that there was
some interaction (illustrated in Figure 17).
Major/Specialization(X) Non-CS/SE related major (Y) CS/SE related major
Mean Std. Deviation Mean Std. Deviation Mean Difference
Figure 17. Multi-line graph of mean quality changes made across levels of formality according to subjects’ major/specialization in university: Non-CS/SE related major and CS/SE related majors
3.2.2.1c. Study Level
As study level may have also played a role in producing particular trends among
groups, subjects were classified into two groups: 1) undergraduates (n = 22); and 2)
graduates/post-graduates (n = 8). Table 15 shows mean and standard deviation of quality
changes made in each group across levels of formality.
Table 15Mean and standard deviation of quality changes made, and the mean difference between groups, at each level of formality according to subjects’ study level: undergraduate (n=22) and graduate/postgraduate (n=8).
Although no significant formality-by-study level interaction was found from the
ANOVA results with study level as the between-subject factor, the between-subjects tests
was significant, F (1, 28) = 6.87, p < .015, partial η2 = .20, suggesting that number of
Study Level(X) Undergraduate (X) Undergraduate
Mean Std. Deviation Mean Std. Deviation Mean Difference
quality changes made at each level of formality between undergraduates and
graduates/post-graduates differed significantly. Figure 18 shows that the number of
quality changes made by graduates/postgraduates at each level of formality was
significantly higher compared to undergraduates. Visual inspection of Figure 18 also
suggested that there was a roughly linear trend in both the undergraduate and
graduate/postgraduate. However, trend tests showed that there was a no linear but a
significant cubic trend, F (1, 28) = 6.34, p < .02, partial η2 = .19, demonstrated in Figure 18
where points of inflection are noticeable e.g. quality changes made by: undergraduate
subjects at medium-low formality; and graduate/postgraduate subjects at medium-high
formality.
Furthermore, according to study level, the overall trend of quality changes made
across levels of formality was similar to previous between-subjects trends (design
experience and major/specialization) – refer to Figure 16, 17 and 18 – all with an increase
in the number of quality changes made at medium-low formality occurring after a drop at
the low formality condition presented on the Tablet PC. The smallest between-group
difference was at medium-low formality (mean difference = 2.06), where as the largest
between-group difference was at medium-high formality (mean difference = 4.39) – see
Table 15 for mean differences across levels of formality. Moreover, the non-parallel lines
further suggested that there was some (small) formality-by-study level interaction.
Although differing in magnitude, the overall linear trend was visible for both groups of
individuals – as formality increased (decreased), the number of expected changes made
decreased (increased).
95
0
2
4
6
8
10
12
14
16
18
20
Lowformality(paper)
Lowformality
Medium-lowformality
Medium-highformality
Highformality
Levels of formality
Mea
n qu
ality
cha
nges
Undergraduate
Graduate/postgraduate
Figure 18. Multi-line graph of mean quality changes made across levels of formality according to subjects’ study level: undergraduate and graduate/postgraduate
3.2.2.2. Multiple Regression Analysis
The similar trends with the three factors further suggested that they were closely
related. The data set was re-grouped according to a combination of design experience,
study level and major/specialization (see Appendix L), and multiple regression analysis
was then conducted to examine and separate individual effects that contributed to the
overall effect of formality on the quality changes made. In other words, the analyses
sought to discover how much each between-subject factor helped explain the effect of
formality on the number of quality changes made.
Formality and the three between-subjects variables (design experience and study
level, and major/specialization) were entered one after the other respectively into SPSS.
Before looking at the actual results, in addition to the data screening earlier for normality
and outliers, multicollinearity was first examined. According to Brace et al. (2006), the
closer to zero the tolerance value is for a variable (vary between 0 to 1), the stronger the
relationship between this and the other predictor variables; and the higher the VIF value
(value from 1.0), the stronger relationship is between predictor variables; and such values
becomes a worry. However, results indicated high tolerance values (over .90), and low
VIF values (less than 1.08), therefore there was no multicollinearity issues.
96
Using the stepwise method, a significant model which included formality, design
experience and study level, emerged, F (3, 31) = 27.15, p < .0001. The model explained
69.8% of the variance (Adjusted R2 = .698). Table 16.1 shows the adjusted R square and
change statistics of each predictor when added to the model. Formality level (model 1)
accounted for 37% of the variance (Adjusted R2 = .370, p <.0001), and the inclusion of
design experience in model 2 resulted in an additional 24.7% of the variance being
explained (R2 change = .247, F (1, 32) = 21.75, p < .0001). Study level helped explained a
further 7.4% of the variance when added upon formality and design experience (R2 change
= .074, F (1, 31) = 9.95, p = .005). However, study major/specialization was excluded
from the model as it did not have a significant impact when added (R2 change = .005, F (1,
30) = .54, p = .47) – hence, not a good predictor to help explain the number of quality
changes made across levels of formality.
Table 16.1Adjusted R Square and R Square change
Model RAdjusted R
SquareStd. Error of the Estimate Change Statistics
a Predictors: (Constant), Formality Levelb Predictors: (Constant), Formality Level, Design experiencec Predictors: (Constant), Formality Level, Design experience, Study leveld Predictors: (Constant), Formality Level, Design experience, Study level, Major/specialization
Table 16.2 gives information for the predictor variables (formality and between-
subject variables) included in the significant model. The result suggests that formality
alone (the manipulated variable) has a strong significant impact on the total number of
changes made (β = -.62, t = -6.61, p < .0001). The negative statistics further suggests that
as formality level increases, the number of quality changes made decreases. The results
for design experience (β = .45, t = 4.68, p < .0001) and study level (β = .30, t = 3.15, p
< .005) further indicates that on top of the effects of formality on quality changes made –
people with more design experience and/or at a high level of study (e.g. graduates) are
more likely to make greater number of quality changes than those with less design
experience and/or at a lower level of study (e.g. undergraduates).
Table 16.2The unstandardized and standardized regression coefficients, and the t-value and significance of each between-subject variables included in the model.
97
B Std Error B β tFormality -1.50 .227 -.623 -6.61Design experience 3.08 .659 .447 4.68Study Level 2.08 .659 .302 3.15
** p < .0001, * p < .005
3.2.3. Analysis of “Expected Changes” made across levels of formality
Table 17 below shows mean and standard deviation of expected changes made at each
level of formality.
Table 17Mean and standard deviation for expected changes made across levels of formality
Formality level Mean Std. Deviation1. Low formality (paper) 13.55 4.242. Low formality (on tablet PC) 11.18 3.253. Medium-low formality 10.22 3.344. Medium-high formality 9.02 3.455. High formality 8.00 3.30
In order to test whether formality had an effect on the number of expected changes
made, one way ANOVA with repeated measures was conducted. Results showed that
there was a significant main effect of formality on the number of expected changes made
to the designs, F (4, 116) = 29.28, p > .001, partial η2 = .50. As one could not tell how
levels of formality affected the number of expected changes made, trends test were also
examined. A significant linear trend was found, F (1, 29) = 92.70, p < .001, partial η2
= .76, over the mean value (expected change) for each level of formality.
Figure 19 shows that, overall, participants made the most expected changes in the
low-formality design on the paper, and the numbers dropped as formality increased.
However, no significant quadratic, cubic nor order 4 trends were found.
98
0
2
4
6
8
10
12
14
16
18
20
22
Low formality(paper)
Low formality Medium-lowformality
Medium-highformality
High formality
Levels of formality
Exp
ecte
d ch
ange
s
Mean expected changes
Figure 19. Multi-line graph showing mean expected changes across levels of formality which is represented by the black bold line. Each participant’s performance (in terms of expected changes made across levels of formality) is also illustrated – see individual lines.
Again, due to the lack of previous empirical research, unplanned pair-wise
comparisons were conducted to examine the differences in the mean quality changes
between levels of formality.
Pair-wise comparisons (with Bonferroni adjustment for multiple comparisons)
revealed that participants made significantly more expected changes when they were
presented with the low formality paper design, compared to designs with other levels of
formality presented on the Tablet PC: low formality on the Tablet PC; medium-low
formality; medium-high formality and high formality. Difference increased as the level of
formality increased, as shown in Table 18. On the other end, participants made
significantly fewer expected changes when they were presented with the high formality
design, compared to designs with medium-low formality; low formality on the tablet PC
and low formality on paper. Differences increased as the level of formality decreased;
however, no significant difference was found between high formality and medium-high
formality. Furthermore, as shown in Table 18, no significant difference was found
between medium-high formality and medium-low formality; and between medium-low
formality and low formality on Tablet PC. This suggested that subjects’ performance (in
terms of making expected changes) was comparable when they were presented with
99
designs with higher formalities (high formality and medium-high formality), and similarly
in designs with lower formalities on the Tablet PC (low formality and medium-low
formality; plus medium-low formality and medium-high formality).
Similar to total changes and quality changes, interestingly, although there were two
low formality conditions, one presented on paper and one presented on the tablet, the
number of expected changes made still differed significantly – the mean difference was
2.37 as can be seen in Table 18. This was also shown in Figure 19 where fewer expected
changes was made on the Tablet PC than on paper.
Table 18Mean differences and their significance at the .05 level in terms of the number of expected changes made between each condition.
(I) Factor 1 (J) Factor 1Mean Difference
(I-J)Low formality (on paper) Low formality (on Tablet PC) 2.37* Medium-low formality 3.33* Medium-high formality 4.53* High formality 5.55*Low formality (on Tablet PC) Low formality (on paper) -2.37* Medium-low formality 0.97 Medium-high formality 2.17* High formality 3.18*Medium-low formality Low formality (on paper) -3.33* Low formality (on Tablet PC) -0.97 Medium-high formality 1.20 High formality 2.22*Medium-high formality Low formality (on paper) -4.53* Low formality (on Tablet PC) -2.17* Medium-low formality -1.20 High formality 1.02High formality Low formality (on paper) -5.55* Low formality (on Tablet PC) -3.18* Medium-low formality -2.22* Medium-high formality -1.02
3.2.3.1. Between-Subjects Factors
In order to examine whether other factors affected the number of expected changes
a participant made, between subject effects including design experience, study level and
specialization were explored.
3.2.3.1a. Design Experience
Subjects’ design experience was examined first as it was hypothesized that there
will be a difference between the numbers of (expected) changes made by subjects who
have more or less design experience. The subjects were grouped into two groups: 1) no
experience or some non-CS/SE design experience (n = 15); and 2) CS/SE design
100
experience (n = 15). Table 19 below shows mean and standard deviation of expected
changes made at each level of formality according to subjects’ design experience.
Table 19Mean and standard deviation for expected changes made, and the mean difference between groups, at each level of formality according to design experience (Total n =30): CS/SE design experience (n = 15), none to some (non-CS/SE) design experience (n = 15).
ANOVA with design experience as the between-subject factor was conducted to
examine whether there was a difference in subjects’ performance according to design
experience. Results showed that in addition to the significant main effect of formality, F
(4, 116) = 29.28, p > .001, there was also a significant between-subject effect, F (1, 28) =
7.64, p < .01, partial η2 = .21, suggesting that subjects with CS/SE design experience made
more changes across levels of formality compared to subjects with no experience or some
non-CS/SE experience as can shown in Figure 20. Furthermore, a weak formality-by-
design experience linear trend was found (although not strictly ‘statistically significant’ at
the alpha level of .05), F (1, 28) = 4.15, p = .051, partial η2 = .13. Visual inspection of
Figure 20 further suggested that there was a stronger linear trend across levels of formality
in subjects with CS/SE design experience than subjects with no experience or some non-
CS/SE design experience – the mean number of expected changes made by subjects with
no experience or some CS/SE design experience at low formality was the same in
medium-low formality. However, on the whole, there was still a linear trend where
subjects made fewer (more) changes as the level of formality increased (decreased).
Additionally, as illustrated in Figure 20, there were magnitude differences between the
two groups across levels of formality. Although, no statistically significant formality-by-
design experience interaction was found, Figure 20 shows that the two lines, representing
subjects with different design experience, appeared to be non-parallel, and thus, there was
some interaction.
Design Experience(X) None to some (non-
CS/SE) design experience(Y) CS/SE design
experience
Mean Std. Deviation Mean Std. Deviation Mean Difference
None to some (non-CS/SE)design experienceCS/SE design experience
Figure 20. Multi-line graph of mean expected changes made across levels of formality according to subject’s design experience
Two other between-subjects, major/specialization and study level, were explored
through visual inspection due to various reasons: the number of subjects in each group
could not be balanced; there were overlapping of subjects i.e. explicit, isolative (i.e.
nested) grouping of subjects was near impossible in the current study as
major/specialization, study level and design experience are all intimately-correlated, and
even if it was possible, a much larger sample would be needed – therefore subjects were
grouped according to one factor only.
3.2.3.1b. Study major/specialization
Since the experimental tasks involved HTML (web) form design, it was of interest
to see whether the number of changes made across the levels of formality differed between
subjects who had more or less HTML form knowledge. Therefore to explore such
between-subject effects, subjects were grouped into two groups: 1) those with non-CS/SE
related major (n = 10); and 2) those with CS/SE major (n = 20). Table 20 shows mean and
standard deviation of expected changes made at each level of formality according to
design experience.
Table 20Means and standard deviations for expected changes made, and the mean difference between groups, at each level of formality according to major/specialization (n=30): non-CS/SE related major (n=10); CS/SE related major (n=20)
102
No significant effects were found after conducting one way ANOVA with study
major/specialization as the between-subject factor. However, since the number of subjects
in each group was not balanced (n = 20, n = 10), statistics produced was not conclusive.
Visual inspection of a multi-line graph (Figure 21) suggested that the number of expected
changes made by subjects who majored/specialized in CS/SE was higher than subjects
who majored in non-CS/SE subjects across the levels of formality, except at medium-low
level of formality, where there was no significant mean difference (mean difference =
0.13) between the two groups – refer also to Table 20. Furthermore, there was a strong
linear trend in the CS/SE major group, while the other group showed a less consistent
linear trend with a sudden increase in expected changes at medium-low formality. In
addition, this indicated that there was also some formality-by-major/specialization
interaction as the lines were non-parallel – gaps bigger at lower formalities compared to
higher formalities – illustrated in Figure 21.
0
2
4
6
8
10
12
14
16
Lowformality(paper)
Lowformality
Medium-lowformality
Medium-highformality
Highformality
Levels of formality
Mea
n ex
pect
ed c
hang
es
Non-CS/SE related major
CS/SE related majors
Major/Specialization(X) Non-CS/SE related major (Y) CS/SE related major
Mean Std. Deviation Mean Std. Deviation Mean Difference
Figure 21. Multi-line graph of mean expected changes made across levels of formality according to subject’s major/specialization: non-CS/SE related major and CS/SE related majors
3.2.3.1c. Study Level
As study level may have played a role in producing particular trends among different
groups, subjects were also categorized into two groups: 1) undergraduates (n = 22); and 2)
graduates/post-graduates (n = 8). Table 21 below shows mean and standard deviation of
expected changes made in each group at each level of formality.
Table 21Mean and standard deviation of expected changes made, and the mean difference between groups, at each level of formality according to study levels (n=30): undergraduate (n = 22); graduate/postgraduate (n = 8).
Results from the ANOVA with study level as the between-subject factor showed
no statistically significant formality-by-study level interaction, but a significant between-
subjects effect was found, F (1, 28) = 5.18, p < .03, partial η2 = .16, suggested that the
number of expected changes made across levels of formality between the undergraduates
and graduates/post-graduates differed in magnitude – graduates/postgraduates made more
expected changes across the levels of formality compared to undergraduates (illustrated in
figure U). Although no statistically significant linear trend found, a weak significant cubic
trend was detected, F (1, 28) = 4.12, p = .052 (slightly above the alpha level of .05).
Visual inspection of Figure 22 further indicated that the weak cubic trend was contributed
by the strong linear trend in the undergraduate group, and a less consistent linear trend in
the graduate/postgraduate group with a steep dip at the low formality condition presented
on the tablet PC, followed by a slight increase in the mean expected changes at medium-
high formality. The two lines representing undergraduates and graduates/postgraduates
also appeared to be non-parallel, suggesting that there may be some formality-by-study
level interaction. On the whole, the main linear trend was still visible for both groups but
differing in magnitude – as formality increased (decreased), the number of expected
Figure 22. Multi-line graph of mean expected changes made across levels of formality according to subjects’ study levels: undergraduate and graduate/postgraduate
3.2.3.2. Multiple Regression Analysis
The similar trends with the three factors further suggested that they were closely
related. The data set was re-grouped according to a combination of design experience,
study level and major/specialization (see Appendix M), and multiple regression analysis
was then conducted to examine and separate individual effects that contributed to the
overall effect of formality on the expected changes made. In other words, these analyses
sought to discover how much each between-subject factor helped explain the effect of
formality on the number of expected changes made.
Formality and the three between-subjects variables (design experience and study
level, and major/specialization) were entered one after the other respectively into SPSS.
Before looking at the actual results, in addition to the data screening earlier for normality
and outliers, multicollinearity was first examined. According to Brace et al. (2006), the
closer to zero the tolerance value is for a variable (vary between 0 to 1), the stronger the
relationship between this and the other predictor variables; and the higher the VIF value
(value from 1.0), the stronger relationship is between predictor variables; and such values
becomes a worry. However, results indicated high tolerance values (over .90), and low
VIF values (less than 1.08), therefore there was no multicollinearity issues.
Using the stepwise method, a significant model which included formality, design
experience and study level, emerged, F (3, 31) = 25.15, p < .0001. The model explained
105
68.1% of the variance (Adjusted R2 = .681). Table 22.1 shows the adjusted R square and
change statistics of each predictor when added to the model. Formality level (model 1)
accounted for 41% of the variance (Adjusted R2 = .41, F (1, 33) = 24.60, p <.0001), and
the inclusion of design experience in model 2 resulted in an additional 21.9% of the
variance being explained (R2 change = .219, F (1, 32) = 19.87, p < .0001). Study level
helped explained a further 6.2% of the variance when added upon formality and design
experience (R2 change = .062, F (1, 31) = 6.62, p = .015). However, study
major/specialization was excluded from the model as it did not have a significant impact
when added (R2 change = .019, F (1, 30) = 2.043, p = .16) – hence, not a good predictor to
help explain expected changes (but a better predictor than in total and quality changes)
made across levels of formality.
Table 22.1Adjusted R Square and R Square change
Model RAdjusted R
SquareStd. Error of the Estimate Change Statistics
a Predictors: (Constant), Formality Levelb Predictors: (Constant), Formality Level, Design experiencec Predictors: (Constant), Formality Level, Design experience, Study leveld Predictors: (Constant), Formality Level, Design experience, Study level, Major/specializationTable 22.2 gives information for the predictor variables (formality and between-
subject variables) included in the significant model. The result suggests that formality
alone (the manipulated variable) has a strong significant impact on the total number of
changes made (β = -.65, t = -6.74, p < .0001). The negative statistics further suggests that
as formality level increases, the number of quality changes made decreases. The results
for design experience (β = .43, t = 4.34, p < .0001) and study level (β = .25, t = 2.57, p
< .015) further indicates that on top of the effects of formality on quality changes made –
people with more design experience and/or at a high level of study (e.g. graduates) are
more likely to make greater number of expected changes than those with less design
experience and/or at a lower level of study (e.g. undergraduates).
Table 22.2The unstandardized and standardized regression coefficients, and the t-value and significance of each between-subject variables included in the model.
B Std Error B β tFormality -1.705 .253 -.654 -6.742**Design experience 3.178 .733 .426 4.336**Study Level 1.886 .733 .253 2.574*
106
**p < .0001, *p < .005
3.3. Additional Analysis of performance data
3.3.1. Paired comparisons: Total, Quality and Expected changes
One-way ANOVA was conducted to compare the three variables (the number of
total, quality and expected changes) across the five conditions, and also to check for
internal validity – whether total number of changes and the number of quality changes are
comparable with the number of expected changes across the five conditions and that the
designs presented to the participants were valid across all five conditions.
Post-hoc paired comparisons were conducted after finding significant differences,
from the one-way ANOVA results (p < .003), between total changes, quality changes and
expected changes across all levels of formality (see Appendix N). Post-hoc paired
comparisons revealed that there was no significant difference between total changes and
formality and medium-high formality; however, there was a significant difference at high
formality (mean differences = 2.26, p = .04). Between quality changes and expected
changes, there was no significant differences at: low formality (paper), low formality (on
Tablet PC), medium-high formality and high formality; however, there was a significant
difference at medium-low formality (mean difference = 2.68). Furthermore, the difference
between total changes and expected changes was significant at every level of formality.
Figure 23 below shows the total, quality and expected changes made across levels of
formality. Overall, the numbers in total, quality and expected changes were different
across levels of formality, as can be seen in Figure 22, but relatively comparable in terms
of the general negative linear trend and the of number of changes made.
107
0
2
4
6
8
10
12
14
16
18
20
Low formality(paper)
Low formality Medium-lowformality
Medium-highformality
High formality
Levels of formality
Mea
n nu
mbe
r of c
hang
es
Expected changes
Quality Changes
Total changes
Figure 23. Multi-line graph of mean total changes, mean quality changes and mean expected changes across levels of formality.
3.3.2. Extra changes: Quality – Expected; and Total – Quality
After statistical analysis of the three types of changes (total, quality and expected), it
was of interest to re-visit the data to count all other changes made to explore the different
types of changes made in addition to the expected changes. Therefore, the number of
extra changes made, in addition to expected changes (correction of deliberate “errors”),
that were of quality (i.e. changes that improved the design) was recorded. Changes that
were not expected, nor were considered quality changes were also recorded i.e. “other”
changes (total – quality). Each extra change found was grouped according to its type –
similar to the main types of functional changes (add, delete, change element type, resize,
relocation (refer to Figure 2 in the method section); but for the purpose of such
investigation, the “change of element” category was separated into two categories: change
of text in labels; and change of element type. Table 23 below shows the number of
individual quality changes made (quality – expected changes) in each category of change,
in each design.
Table 23. The number of extra changes (quality – expected) made in each design, grouped according to the type of change.
Category of change Low formality (on paper)
Low formality
Medium-low formality
Medium-high formality
High formality
108
Change of text in labels 9 16 11 4 2.5change of element type 5 3.5 6.5 4 2.5adding elements 15.5 15.5 9 11.5 10.5deleting elements 1 1 0.5 1 0relocation of elements 4 4 5 3 1resizing of elements 1 3.5 2.5 1 2.5miscellaneous changes 0 1 1 1 0
Similarly, Table 24 below shows the number of “other” individual changes made
(those did not count as quality changes or expected changes i.e. total – quality changes) in
each category of change, in each design. For more details on the extra individual changes
made in each design (quality – expected; total – quality) and the number of participants
making such changes, see Appendix O1, O2, O3, O4 and O5 (from low formality to high
formality designs).
Table 24. The number of extra changes (total – quality) changes made in each design, grouped according to the type of change.
Low formality (on paper)
Low formality
Medium-low formality
Medium-high formality
High formality
Change of text in labels 23 26 16 6 9change of element type 13 7 9 12 7adding elements 29 23 11 20 22deleting elements 8 6 4 5 4relocation of elements 32 18 16 13 10resizing of elements 2 3 5 1 7miscellaneous changes 0 1 1 1 1
Overall, visual inspection of Table 23 and Table 24 suggested that type of quality
changes made were mostly the addition of elements, and rarely deleted items; and the
changes (total minus quality changes) made were mostly relocation and addition of
elements, followed by change of text in labels and change of element type. The deletion
of elements was mostly considered as extra changes and only a few were of quality (as
reflected in Table 23 and Table 24). However, interestingly, the proportion of addition of
elements was relatively higher in the high formality design, which also reflects the low
number of other types of changes made. In other words, it was likely that subjects would
add elements to a design with high formality than making other types of changes.
3.3.3. Order Effect
By counterbalancing the order of presentation of the five conditions (presentation
of designs with different levels of formality – refer to Figure 1 in the methods section),
order effects was controlled for, and it was expected that overall, order effects would not
influence results.
109
Nonetheless, although order effects was controlled for, according to Heiman
(2000), order effects may reduce internal validity; therefore this was also examined as it
may have played a role in producing a (weaker/stronger) trend of changes made across
levels of formality. However, as there was only a small sample (N = 30), and that it was
not the purpose of the study to test for order effect, no statistical tests were performed to
search for significant order effects. Instead, graphs were used for visual inspection. Data
were grouped and examined according to the order of conditions presented (e.g. 54321,
12345, 43215, 23451…etc), one multi-line graph for total changes (Figure 24a); quality
changes (Figure 25a); and expected changes (Figure 26a). Visual inspection of the Figure
24a, 25a and 26a suggested that generally, although with different orders of conditions
presented, there was still a linear trend found in the number of total, quality and expected
changes made across levels of formality – also identified in the trends test in one-way
repeated measures ANOVA. There was little to some order effect – this was more
noticeable in orders 54321 and 12345. In order 54321, added with the practice effect over
the five conditions, such order levered the main effect of formality – less changes made in
the first condition (high formality), but as formality decreased, the number of changes
increased – also seemed to be affected by practice effect. However, order 12345 worked
against the main trend – in other words, with practice effect present, and at its strongest
during the fifth condition (high formality), the main effect of formality on the number of
changes made was weakened by the opposing force – practice effect. Figure 24b, 25b and
26b illustrates the contrast between the curves of orders 54321 and 12345.
The weaker or stronger trends shown in Figures 24a, 24b, 25a, 25b, 26a and 26b,
must be interpreted with extreme caution as there were only four subjects (n=4) in each
order group. Overall, as expected, order effect did not influence the results significantly
as linear trends were present regardless of the order of presentation of conditions, which
further highlights the general significant linear trend.
110
0
5
10
15
20
25
30
Low formality(paper)
Low formality Medium-lowformality
Medium-highformality
High formality
Levels of formality
Mea
n To
tal c
hang
es54321
43215
32154
21543
15432
12345
23451
34512
45123
45123
Figure 24a. Mean total changes made across levels of formality according to the order of conditions presented.
0
5
10
15
20
25
Low formality(paper)
Low formality Medium-lowformality
Medium-highformality
High formality
Levels of formailty
Mea
n to
tal c
hang
es
54321
12345
Figure 24b. Mean total changes made across levels of formality according to the presentation of conditions in the 54321 (n=4) and 12345 directions (n=4).
Figure 25a. Mean quality changes made across levels of formality according to the order of conditions presented.
0
2
4
6
8
10
12
14
16
18
20
22
24
Low form ality(paper)
Low formality Medium-lowformality
Medium-highform ality
High form ality
Levels of formality
Mea
n qu
ality
cha
nges
5432112345
Figure 25b. Mean quality changes made across levels of formality according to the presentation of conditions in the 54321 (n=4) and 12345 (n=4) directions.
Figure 26a. Mean expected changes made across levels of formality according to the order of conditions presented.
0
2
4
6
8
10
12
14
16
18
20
Low form ality(paper)
Low form ality Medium -lowform ality
Medium -highform ality
High form ality
Levels of formality
Mea
n E
xpec
ted
Cha
nges
54321
12345
Figure 26b. Mean expected changes made across levels of formality according to the presentation of conditions in the 54321 (n=4) and 12345 (n=4) directions.
3.4. Analysis of the “Overall Enjoyment” rankings of the five designs
113
Subjects’ rankings of the five designs with different levels of formality were
examined (see Appendix P). “Overall enjoyment” was defined as the subjects’ ranking of
the five designs in the order from the “most-liked” design with a rank of 1 (i.e. the design
they enjoyed working on the most in comparison to other the other designs presented) to
the “least-liked” design with a rank of 5 (i.e. the design they enjoyed working on the least
in comparison to the other designs presented). Figure 27 shows the overall ranking, from
the most liked (lower scores) to the least liked (higher scores), when working on each
design in comparison to other designs in terms of enjoyment.
3.37(sd = 1.35)
4.2(sd= 1.35)
3.17(sd = 1.15)
2.53(sd = 0.90)
1.7(sd = 1.02)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
High formality Medium-highformality
Medium-lowformality
Low formality Low formality(paper)
Levels of formality
Mea
n ra
nk
Figure 27. A bar graph showing mean rank and standard deviation, in terms of preference, according to the overall enjoyment, in working on each design with a different level of formality.
Overall, the high formality design was ranked most frequently as the “most liked”
design (by seventeen out of thirty, or 56.7% of participant). Most participants (twenty out
of thirty, or 66.7%) ranked the low formality design presented on the Tablet PC as the
least liked design. Interestingly, however, fourteen out of thirty (46.7%) participants
ranked the low formality design presented on paper as the second least liked. In other
words, when given low formality designs, participants liked (enjoyed) working on the
design presented on paper more than working on the design presented on the Tablet PC.
This was reflected in participants’ preference for design mediums in the experiment:
thirteen out of thirty (43.3%) participants preferred using paper and pen, while fifteen out
of thirty (50%) participants preferred using the Tablet PC, and two participants (6.6%)
indicating no preference of one medium over another medium.
114
In addition to the overall enjoyment, the underlying reasons for the rankings
expressed by the subjects were also investigated. Subjects’ responses were generally
categorized into three groups – subjects who ranked according to (a combination of): 1)
aesthetics of the design; 2) effort required to improve the design; and 3) level of
fun/stimulation when working on the design. One must note that some subjects’ response
overlapped into the groups (i.e. subjects ranked according to a combination of the two or
three factors); however, for the purpose of the analysis, the response were grouped and
analyzed in separate tests. Table 25 shows the mean and standard deviation of rankings of
the five designs with respect to overall enjoyment, aesthetics, perceived fun/stimulating
level, and perceived effort required when working with a design.
Table 25. Mean ranks and standard deviation, in terms of overall perceived enjoyment and other underlying factors for subjects’ rankings (including appearance, perceived effort required, and perceived fun/stimulating level), when working on each design in comparison to other designs presented.
3.4.1. Ranking according to design appearance (aesthetics)
Most participants (twenty-one out of thirty, or 70%,) ranked according to the
aesthetic properties of the designs (e.g. tidiness of lines and fonts, alignment of elements,
and whether it was easy to follow). See Table 25 which shows the mean rank of designs
with high formality to low formality according to aesthetics factors. To determine the
effect of formality level (aesthetics) on participants’ enjoyment in working on the design
(indicated by rankings), Friedman’s rank test for several related samples was used as
proposed by Winer, Brown, and Michels (1991) and Howell (2002). Analysis of the
ranked data showed that the subjects’ rankings of the five designs according to aesthetics
were significantly different, χ2 (4, N = 21) = 47.43, p < .0001, and the Kendall coefficient
of concordance of .57 indicated strong differences among the rankings. Post hoc analysis
by Wilcoxin signed-rank test, indicated that participants enjoyed working on the high
formality design (lower rankings) significantly more than the other four designs, all with p
< .0001, as participants indicated that the designs appeared “tidy”, “pretty” and “[format
is] easy to follow”. Results also showed that participants enjoyed working on the low
115
formality design on Tablet PC the least as it had the highest mean rank, and was
significantly higher when compared to the other four designs: low formality on paper (p
= .006); medium-low formality (p = .001); medium-high formality (p < .0001) and high
formality (p < .0001), as many participants had noted that the design appeared “untidy”,
aesthetically “unattractive", and that the layout was “hard to follow”. Interestingly
participants enjoyed working on the low formality design presented on paper significantly
more than the low formality design presented on the tablet PC (p = .006). No significant
differences were found between rankings of other pairs of designs (medium-high
formality, medium-low formality and low formality on paper) in terms of enjoyment when
a design appeared to be more or less formal (beautified).
Additionally, the overall rankings of designs according to appearance (aesthetics)
of designs were negatively associated with the trend of the number of changes made
across levels of formality (refer to Appendix Q for subjects’ performance – those who
ranked according to the aesthetics factor – across levels of formality). The number of
changes made increased linearly as formality decreased, and similarly, rankings increased
as formality decreased – i.e. subjects liked designs that appeared more formal than designs
that appeared less formal, but on the other hand, subjects made fewer changes to the
designs that they liked (higher formality designs) and made more changes to designs that
that liked less (lower formality designs). However, interesting, unlike subjects’
performance, the low formality designs presented on paper was ranked lower than the low
formality design presented on the tablet PC i.e. subjects enjoyed working on the low
formality design presented on paper than on the tablet PC.
3.4.2. Ranking according to perceived “effort required”
Moreover, eleven out of thirty (23%) participants ranked according to the
perceived effort (input) required to improve designs. The mean ranks from high formality
to low formality were 1.55, 2.55, 3.82, 3.91, 3.09 (see Table 25). Subjects’ reasons
associated with rankings indicated that a higher rank (e.g. 5) meant that the design
“required lots of changes” and therefore more effort is needed to improve the design; and a
lower rank (e.g. 1) meant that the design was generally “good” and “required fewer
changes” (i.e. least effort required).
As suggested by Howell (2002) and Winer, Brown, and Michels (1991),
Friedman’s rank test for several related samples was employed to determine the effect of
formality on participants’ perceived effort required when working on the design (which in
116
turn affected the overall enjoyment rankings). Analysis of the ranked data showed that the
subjects rankings of the five designs, according to perceived effort required, were a
significantly different, χ2 (4, N = 11) = 16.16, p < .003, with the Kendall coefficient of
concordance of .37 indicating fairly strong differences among the rankings. Post hoc
analysis by Wilcoxin signed-rank test, (and controlling for the Type I errors across these
comparisons at the .05 level using the LSD procedure) further suggested that the
participants enjoyed working on the high formality design (lowest mean ranking)
significant more than the other four designs: medium-high formality (p = .039); medium-
low formality (p = .004); low formality on Tablet PC (p = .009) and low formality on
paper (p = .015) – some participants reasoned that the high formality design had “fewer
mistakes” and “not many changes needed” compared to other designs which participants
indicated that they “wanted to change more” in the design. Results also suggested that
subjects enjoyed working on the medium-high formality design significantly more than the
medium-low formality design (p = .046) as well as the low formality design on the Tablet
PC (p = .034). No significant differences were found between rankings of other pairs of
designs (medium-low formality; low formality on Tablet PC and low formality on paper)
in terms perceived effort required when working on a design (i.e. the overall enjoyment
rankings)
Additionally, the overall rankings of designs according to the perceived effort
required was negatively related to the number of changes made across levels of formality
(refer to Appendix R for subjects’ performance – those who ranked according to effort
required – across levels of formality). The number of changes made increased (decreased)
linearly as formality decreased (increased) (see Appendix R for subjects performance),
and similarly, rankings increased as formality decreased. However, the rankings of low
formality designs – one presented on paper and the other presented on the tablet PC – was
not in linear order expected – refer to Table 25.
3.4.3. Ranking according to perceived “fun and/or stimulating level”
Few participants (seven out of thirty, or 23%) ranked according whether designs
were fun or stimulating to work on during the task. The mean rank of the designs from
high formality to low formality was in the order of 2.86, 2.71, 1.86, 3.29 and 4.29 (see
Table 25). Participants’ reasons associated with rankings indicated that a rank of 5 meant
that the design was the least interesting and stimulating to work on; whereas, a rank of 1
meant that the design that was the most fun and/or stimulating to work on out of the five
designs given. To explore the whether there were any differences in the overall enjoyment
117
rankings of the five designs in terms of level of fun/stimulation, Friedman’s rank test for
several related samples was conducted. Results showed that the ranked data did not differ
significantly, χ2 (4, N = 7) = 8.80, p = .66, which suggested that participants found the five
designs similar in terms of the level of fun/stimulation when working on the designs.
Visual inspections suggested that subjects enjoyed working on the medium-low formality
design as it was more fun/stimulating than other designs (mean rank of 1.86 – lowest
compared to other designs); but found the low formality design least fun/stimulating to
work on (mean rank of 4.29 – highest in relation to other designs). The fun/stimulating
level of other designs did not differ very much for the subjects, which was shown in the
similar rankings (2.86, 2.71, and 3.29 – refer also to Table 25 for variance). However,
results were only preliminary and inconclusive as the number of subjects was very small
(n= 7).
Additionally, although with varying perception of whether a design was fun and/or
stimulating to work on (see Table 25), it made no difference to the effect of formality – the
number of changes made as formality increased (decreased) still decreased (increased) –
see Appendix S for subjects’ performance across levels of formality – those who ranked
according to the fun/stimulation level of working on the designs. This may suggest that
the perception of fun/stimulation level when working on the designs did not correlate
strongly to the number of changes made to the designs; however, since fun/stimulation
level of designs varies in different individuals, and that there were only seven subjects
examined, interpretations must be made with caution.
3.5 Design Tool Preference
3.5.1. Design tool preference in the experiment
Subjects’ design tool preference in the experiment was examined to determine
whether subjects preferred using pen and paper, and/or the Tablet PC or whether there was
no preference between the two tools during the design tasks.
Figure 28 shows that overall, 50% of the participants preferred using the tablet PC
(Inkit), while thirteen (43.3%) participants preferred using paper and pen during the design
tasks; and two participants responded with no particular preference between using pen and
paper, and the Tablet PC during the design tasks. See Appendix T for the detailed
responses from subjects on design tool preference during the experiment.
118
0
10
20
30
40
50
60
No preference Preferred paper &pen
Preferred Tablet(Inkit)
Design tool preference during the experiment
Per
cent
age
(%)
Figure 28. Bar graph showing subjects’ design tool preference during the experiment.
Additionally, factors that were likely to affect design tool preference during the
experiment were briefly examined, including study major/specialization (see Figure 29a)
and design experience (see Figure 29b) – also refer to Appendix T for indication of
subjects’ study and design backgrounds along with detailed responses.
Visual inspection of the Figure 29a revealed that in the CS/SE major group, more
subjects preferred using the Tablet (55% or eleven out of twenty) than using paper and pen
(35% or seven out of twenty subjects), while two subjects had no preference; and in
contrast, in the non-CS/SE majors group, more subjects preferred using paper and pen
(60% or six out of ten) than using the Tablet (40% or four out of ten); and hence, with a
smaller number of subjects in the group, interpretation of group differences must be made
with extreme caution. Similarly, such contrasting pattern was also found in terms of
design tool preference between subjects with different design experience – in the CS/SE
design experience group, more subjects preferred using pen and paper (46.7% or seven out
of fifteen) than using the Tablet (40%, six out of fifteen); and in contrast, more subjects
with none-to-some non-CS/SE design experience preferred using the Tablet (60%, nine
out of fifteen) than using paper and pen ( 40% or six out of fifteen). With respect to no
particular preference for either of the two design tools used in the experiment, not much
could be concluded as there were only two subjects responding this way, and both with
CS/SE major and design experience.
119
0
10
20
30
40
50
60
70
No preference Preferred paper &pen
Preferred Tablet(Inkit)
Design tool preference during the experiment
Per
cent
age
(%)
CS/SE majorNon-CS/SE majors
0
10
20
30
40
50
60
70
No preference Preferred paper &pen
Preferred Tablet(Inkit)
Design tool preference during the experiment
Per
cent
age
(%)
CS/SE design expereince None to some non-CS/SE design experience
Figure 29a. Bar graph showing the proportion of subjects – according to study major: CS/SE (n=20) and non-CS/SE majors (n=10) – preferring different design tools during the experiment (paper and pen; Tablet (Inkit); or no preference).
Figure 29b. Bar graph showing the proportion of subjects – according to study major: CS/SE design experience (n=15) and none to some non-CS/SE design experience (n=10) – preferring different design tools during the experiment (paper and pen; Tablet (Inkit); or no preference).
3.5.2. Design tool preference in the experiment
Subjects’ design tool preference in the “real world” was also examined for
comparisons with design tool preference during the experiment – the similarities and
differences. Figure 30 shows that overall, most participants expressed that he/she would
prefer using pen and paper first then move on to using computer if they were in real life
design situations. On the other hand, many participants preferred using computers (with
other popular applications such as Photoshop, Visual Basic.Net). The number of subjects
expressing other preferences was similarly low. See Appendix U for the detailed
responses from each subject.
120
Figure 30. Bar graph showing subjects’ design tool preference in real life design situations
Factors that were likely to affect design tool preference in real life design situations
were briefly examined, including study major/specialization (see Figure 31a) and design
experience (see Figure 31b) – also refer to Appendix U for indication of subjects’
background along with the detailed responses.
Visual inspection of the Figure 31a revealed that subjects who majored in CS/SE had
a wider variety of preferences for a single and/or a combination of design tools, compared
to subjects with non-CS/SE majors. There was also a greater proportion of subjects with
non-CS/SE major preferring paper and pen than subjects with a CS/SE major.
Interestingly, no subjects with non-CS/SE major preferred using tablet (Inkit) if they were
doing design in the real world, compared to a small number of subjects with CS/SE major
who would prefer (choose) tablet as their design tool. When subjects were examined
according to their design experience, a different pattern of preference was found – as
shown in Figure 31b – there was a distinct preference for particular design tool(s) between
subjects with CS/SE design experience and subjects with none to some non-CS/SE design
experience. The majority of subjects with CS/SE design experience preferred using
computer (that has other design tools such as VB.net and Photoshop) as opposed to the
majority of subjects with none to some non-CS/SE design experience preferring the use
pen and paper first, then computer (software). Furthermore, the use of pen and paper, then
computer was the second most popular preference in the CS/SE design experience group;
however, as the number of subjects in the group with none to some non-CS/SE design
experience, no such statement could be made.
121
0
10
20
30
40
50
60
Nopreference
Paper andpen
Tablet PC(Inkit)
Computer(Othertools)
Pen andpaper, thenTablet PC
Pen andpaper, thenComputer
Design tool preference in real life design situations
Per
cent
age
(%)
CS/SE majorNon-CS/SE Major
Figure 31a. Bar graph showing the proportion of subjects – according to study major: CS/SE (n=20) and non-CS/SE majors (n=10) – preferring different design tools in real life design situations.
0
10
20
30
40
50
60
70
80
Nopreference
Paper andpen
Tablet PC(Inkit)
Computer(Othertools)
Pen andpaper, thenTablet PC
Pen andpaper, thenComputer
Design tool preference in real life design situations
Per
cent
age
(%)
CS/SE design experience
None to some non-CS/SE design expereince
Figure 31b. Bar graph showing the proportion of subjects – according to study major: CS/SE design experience (n=15) and none to some non-CS/SE design experience (n=10) – preferring different design tools in real life design situations.
Chapter 4. Discussion
There was significant effect of formality on design task performance, in terms of
total, quality and expected changes made; significant linear trends were also found,
indicating that as formality increases, the number of changes made decreases. Significant
effect of expertise, particularly design experience, study major/specialization and study
level on the number of changes made across levels of formality. The overall enjoyment
rankings of the five designs were ranked differently by different subjects, in three
categories, including those who ranked according to aesthetics and fun/stimulating level of
the design and perceived effort required to improve the design. No statistical analyses of
subjective measurement of design tool preference during the experiment, and design tool
preference in real design situations. The simple visual analyses served as an indication of
different tool preference in the laboratory experiment as opposed to the real world, and
whether it affected design performance in any ways. No difference in preference was
found between designing on paper compared to designing on the tablet PC (InkKit) in the
experiment. As expected, preference in real world design situations was more diverse
compared to preference in the experiment – this gave a brief indication of whether InkKit
will be an effective tool, in terms of usability – whether people will actually use it. The
following sections will discuss the findings in a more detailed manner.
4.1. Effects of formality on design task performance
In the following section, the mean total number of changes is referred to as “total
changes”; the mean number of quality changes made is referred to as “quality changes”;
and the mean number of expected change made is referred to as “expected changes”.
Overall, there was an effect of formality on the number of changes made (in
terms of total, quality and expected changes), and more specifically, the number of
changes made decreased as the designs’ level of formality increased; hence, a negative
relationship between formality and design performance. This suggests that formality plays
an important role in affecting the subjects’ design performance; especially on decisions on
making changes to improve the designs presented (in terms of functionality and usability).
Field’s (2004) findings showed that aesthetics play an important role in problem
solving, which further support the findings of this study that formality (as a kind of
aesthetics, a strong emotional response) play an important role design performance (as
problem solving performance). Furthermore, as design can be seen as a kind of problem
solving by many (e.g. Goldschmidt, 1997; Smith and Browne 1993; Thomas & Caroll,
1979), the underlying visual mechanisms, for example, maybe visual attention in diagram-
based problem-solving (e.g. Grant & Spivey, 2003) may help explain why designers make
more changes to a design that is hand-drawn and appear rough and sketchy (lower levels
of formality), compared to tidier designs, that appear more precise, polished and formal
(higher levels of formality) – maybe more visual attention is required to first understand
and then work on a design with lower levels of formality than designs with higher levels
of formality. Although no research directly supported this claim (question), Grant and
Spivey (2003) measured eye movements in problem solving, suggesting that visual
attention was an important factor in problem solving and highly correlated to the
frequency of correct solutions; in this case, visual attention may play a role in design
(problem-solving) performance, in terms of improving a design by making changes. A
factor that affects visual attention is perceptual grouping of elements (an aspect of Gestalt
psychology – Koffa, 1935) – in the context of the present study, in the low formality
designs, no elements are aligned exactly, therefore more elements are scattered, thus, more
attention and eye fixations (and time) are needed to process the initial design presented;
however, in high formality designs, more elements are aligned (i.e. perceptually grouped),
which allows easier and fast scanning of the design, hence, less attention on details. In
addition, in higher formality designs, lines are straightened and text is presented in
standardized computer fonts, thus allowing even fast scanning and better readability
compared to low formality designs with squiggly lines and handwriting, which maybe
harder to follow, thus requires even more time and attention is to process the initial design
presented; thus requires more attention on details.
Such difference in attention required to process a design presented may be useful
to explain the effects of formality on design performance – as the level of formality of
increases, the design may become easier to process (e.g. reading, scanning) as less
attention is required on details, thus, maybe fewer errors are noticed; and as for low
formality designs, as more detailed attention is required, more errors maybe noticed during
the course of processing of the design. Hence, design performance – number of changes
made (total, quality and expected changes) to improve the design presented – may be
affected in such way.
There are other alternative explanations for design performance across levels of
formality. Skeptically, it can be argued that the effect of formality on the design process,
and more specifically on the number of changes made, was simply due to the
experimenter’s subconscious bias during the course of designing of the five online forms
to be presented to the participants. It was possible that the higher formality designs were
subconsciously designed as being more “correct” than designs of lower formalities – such
effect could be seen as one of the limitations of the study. However, optimistically, it is
unlikely that results from the study was purely due to experimental bias as there were
statistically significant results and noticeable trends shown in the graphs including
significant negative linear trend as formality increases in total changes, quality changes
and expected changes decreases; plus differences in terms of number of changes made
between designs with different levels of formality. Furthermore, as the independent
variable (formality) was carefully manipulated and controlled for, and that the number of
planned deliberate “errors” to be corrected was the same in each design, the analysis of
expected changes enabled a controlled and systematic way of examining the differences in
subjects’ performance across levels of formality; and indeed, significant formality effect
and a linear trend were found.
On top of this, it is also debatable whether the subjects noticed more errors in the
low formality designs than higher formality designs (effects of formality); or whether the
errors in the low formality designs were just easier to detect or easier to improve than
errors in the high formality designs – in other words, varying difficulty in improving the
five designs (an experimental confound). The latter of the two arguments, can be viewed
as one of the uncontrollable limitations in the study even though all objective measures
have been taken to make each design as similar as possible. It is inevitable that perception
of task difficulty is subjective and personal for each individual, as such variable is
dependent on personal experience, skills and ability – according to Spector (2006), in the
context of Industrial and organizational psychology, work/task performance is dependent
on such factors. It becomes the question of interaction rather than causation when it
comes to debating about the effects of formality on errors being noticed in the designs
presented – results could be interpreted as: 1) formality affecting subjects’ visual attention
on a design, for example, more eye fixations in stimulus that appeared more complex
(Grant & Spivey, 2003) which could be the case with the lower formality designs as it was
scattered and less clustered and aligned as the high formality design). From aesthetics
approach in art (see Levinson, 2003 for a fuller account and discussion), it may be that the
more formal (or aesthetically pleasing) a design appears, the fewer errors one may notice,
as beauty may have ‘blinded’ the eyes (and the mind).
Linear Trends
The significant linear trend found in terms of the total changes, quality changes
and expected changes made in designs across levels of formality suggests that as the
formality of a design increases (i.e. as a design appears more formal, pretty and tidy): 1)
the number of changes made in attempt to improve the design (i.e. total changes)
decreases; 2) quality changes decreases; and 3) expected changes (i.e. the number of
‘planned errors’ corrected) decreases. Vice versa, as the formality of a design decreases
(as a design appears rougher, less tidied-up and sketchy): 1) the number of changes made
in attempt to improve the design (i.e. total changes) increases; 2) quality changes
decreases; and 3) expected changes (i.e. the number of ‘planned errors’ corrected)
increases. Moreover, the significant linear trend in the expected changes made across
levels of formality showed a more robust effect of formality on design performance. Even
when each design presented contained the same number of “planned errors” for
corrections, the number of corrections made to those errors (i.e. expected changes) still
differed significantly in a linear manner as the design looked more or less formal. Such
findings support the underlying concepts in design education (e.g. in design classes) and
design process prescription in design handbooks (e.g. Fowler & Stanwick, 2004; Brinck,
2004; Shesh & Chen, 2004), eventually, it is likely that such tools would be accessible for
design students (as well as design educators and professionals) from different disciplines –
only if and when usability testing shows good results which further facilitates
commercialization of such products for a wider user population. Hence, should sketch-
based design tools becomes one of the main stream design tools along with paper and pen,
and computers (using applications such as Computer-Aided Design (CAD), Microsoft
Photoshop, Fireworks etc), the effects of formality (beautification) should be emphasize
through design education, to help minimize the likelihood of error-filled designs lasting
through to the later stages in the design process, for example, in domains such as web
interface design; flow diagrams in mechanical design; and floor plans in architectural
design. It may not be too much of a problem in design projects and assignments for
students, but in the design industry, such mistake could be exceedingly costly for the
design company responsible. The impact could be even more profound in the design of
complex systems where errors/mistakes that are unnoticed until the later stages in the
design cycle (depending on its severity) could ultimately put the design at risk of being
redeveloped – i.e. back to the early stages of design; which could mean extra stress for the
designers and his/her company, as well as critical strains on the already-limited resources,
i.e. mainly time (and money – such as in physical product design when prototypes are
required). Hence, design education and training must address the effects of formality on
design performance, and thus, increase caution when beautifying designs.
Although, the findings from this study suggests that an active approach should be
taken towards addressing the issues with levels of beautification and formality in sketch-
based design tools in the design process – i.e. minimizing possible undesired effects of
formality in early design – there are other factors that may affect the use of beautification
and its effectiveness. For example: the nature and scope of the design task – whether it is a
complex design (e.g. a simple website with three to four pages or a whole information
system) or a simple design (e.g. a simple online form on a design that only requires a user
to fill in their name and address, or a complex online form that requires more extensive
user input); and the level(s) of formality one wants to achieve; here, recommendations for
different situations regarding the use of beautification could be useful.
4.8. Methodological issues and limitations
In addition to the limitations discussed earlier including: small and heterogeneous
sample; content in the five design; possible experimenter’s bias in the five forms designed;
and levels of formality created and their intervals; the other limitation of this study was
that InkKit was (and is) still in its development stage. Although it closely matches the
Tablet Diary (Tablet Edition, 2005) program in which users are able to write/draw
anything (using the pen), several participants commented that InkKit was “a bit hard to
use” in the experiment as they had to switch between different modes i.e. Inking mode,
Eraser mode, Selection mode by pointing the pen tip to the icons at the top of the screen.
According to Fitts Law (with respect to VDU display design), the larger the distance from
one object to another on the screen, the longer it takes for the eyes to fixate from one
object to another object on the screen. This suggests that, as the hand movement is guided
by the eyes, the time it takes for the hand to guide the cursor from one object to another
also increases. Hence, the design task in the experiment required participants to make
changes to an existing design, by deleting, adding, resizing, relocating etc, therefore,
participants had less freedom to stay in one mode. This mode switching may have
affected participants’ willingness to make changes; hence, results on the effects of
formality on the number of changes made (design performance) should be interpreted with
caution. An implication of this on interface design is to better accommodate switching of
modes to support natural human-computer interaction.
4.9. Future research and directions
This study is one of the first to examine the different levels of formality and their
effects on design performance, and thus, the findings from the present study raised many
specific as well as wider questions on the effects of formality on design performance;
which in turn, forms the basis for future direction in similar research. The limitations and
methodological issues found in the present study also provide useful resources for
improvements in methodological aspects in similar research in the future. In addition to
the research recommendations discussed earlier along with the limitations of the study, for
example, exploring more dimension and levels of formality, future research could also
explore the following.
More laboratory studies are needed to further explore the effects of sketch-based
design tools, particularly the effects of levels of formality of designs (as a result of
different degrees of beautification), on the design performance, behaviour, cognition and
perception during the design process. For example, more levels of beautification could be
explored by including aspects such as the HTML ‘look and feel’ (e.g. of dropdown menus,
textboxes, buttons, radio buttons); simple colours to represent differences between
components (Brinck, et al., 2002); hence, whether the negative linear relationship (levels
of formality and design performance) will still exist becomes an important question to be
answered. Field studies should also be done to check external validity to support the
(non)practicality and usefulness of beautification in real world situations such as
presentation to clients in early stages in the design process (Newman, et al, 2003).
As beautification has been regarded as an important aspect of informal sketching-
based tools (Plimmer and Apperley, 2004), different levels of beautification was explored
in the present study. It would be useful, in terms of design education, for future research
to further explore the effects of different degrees of beautification at other stages in the
design process. In addition, evaluation studies on informal sketching-based tools that
provide beautification functionalities might help with design tool development. Such
studies will evaluate whether beautification functions can be used effectively, and if
included in an informal sketching-based tool, at what level and to what extent should it be;
and whether it is plausible or practical? Furthermore, it would also be valuable to examine
the effects and usefulness of beautification in other (different) types of applications e.g.
programs that support pen-input of music scores, mathematic formulas, simple drawings,
and even kids drawing program (which has been an interest to many researchers since the
sketch-based applications became popular). In addition to different applications, the
effects and the effectiveness of beautification at different levels for different groups of
users could also be investigated. Moreover, one question yet to be answered is whether
there is an optimal level of beautification to create the appropriate level of formality of
designs; hence, future extensions to the present study to provide a more comprehensive
picture of relationships between different levels of beautification, formality and design
performance.
The present study examined individual performance in a laboratory setting only;
however, in real design situations, designers often work in teams (Stempfle & Badke-
Schaub, 2002). Thus, the present study could be expanded to examine the effects of
formality on team performance. However, results from such study may be difficult to
interpret and examine as variability within a team may differ between teams – a
combination of qualitative and quantitative data maybe useful for a fuller account and
analyzes.
One underlying (and fundamental) question raised from the present study, besides
the effects of formality (as a result of beautification) on design performance, is the effects
of using different design mediums. Therefore, future research should explore the effects
of formality on the design process using different design medium, in a more systematic
manner, for example, by comparing subjects between two groups (with similar design and
study backgrounds) where subjects in one group works on designs with different levels of
formality presented on the tablet PC; while subjects in the other group works on designs
with different levels of formality presented on paper (printed versions of the designs on
the tablet PC). Thus, differences and/or similarities of using different design mediums
combined with the effects formality could be explored.
Results from the present study suggest that individual differences played a role in
affecting the overall number of changes made across designs with different levels of
formality. It was observed that some subjects just make more changes overall in
comparison to others – maybe they are more motivated to make changes. Some subjects
tries hard and some subjects did not seem to exert too much effort – shown in the
consistent high number of changes compared the consistently low number of changes as
seen in individuals regardless to his/her design experience, study level or study
major/specialization. It was observed also, that some participants were able to concentrate
the whole way throughout the five conditions, utilizing every minute during the tasks as
opposed to other participants who made few changes and stopped before the 10minutes
time. This was also noticeable in the recorded on-screen actions throughout the five
conditions – some participants had little on-screen actions compared to others who’s
screen constantly changes; hence, the more changes on a screen (e.g. movements) the
larger the video file. Such observations supports that motivation and personality play a
significant role in influencing (work) performance, especially in the field of work,
industrial and organizational psychology as shown in many psychological studies reported
by Spector (2006). Future research on motivation and personality in the context of design
performance will contribute to a greater understanding of designers’ interaction with
different elements and artifacts within the design process – a long term research direction.
Furthermore, findings from such studies could be useful to the design industry, for
purposes such as personality screening during the recruitment process and motivation
increase to help improve performance (e.g. design quality, design decisions, innovation
and efficiency).
Chapter 5. Summary and Conclusion
There has been little of research on designers’ interaction with informal sketching-
based tools, particularly, within the context of beautification – an important aspect in such
tool. Moreover, the effects of beautification on the designers (e.g. behavior, perception,
cognition, performance), have been largely neglected as many researchers have been
focusing on improving recognition and beautification techniques in sketching-based tools.
Overall, the present study provided a strong starting point in a new path of sketch-based
tools research by exploring different levels of formality (created by applying different
degrees of beautification) and their effects on designers’ performance in the early stages of
the design process.
Beautification was explored in the present study, by developing a taxonomy of
degrees of beautification, which was validated by producing designs that appeared more,
or less formal (i.e. with different levels of formality). Furthermore, the designs produced
by systematically varying the degree of beautification, were also used for exploring the
effects of formality on design performance – measured in terms of number of changes
made to a design presented – during the early stages of the design process.
The findings in the present study confirmed previous findings by Black (1990) and
Plimmer and Apperley (2004) that reviewers/designers interact differently with designs
that appears untidy, sketchy, rough and informal (i.e. low formality designs) and designs
that appears tidy, computer-rendered, and formal (i.e. high formality designs). In addition,
the current study examined not only the two levels of formality that Black and Plimmer
and Apperley looked at, but also the other levels of formality in between – where a design
appeared more or less formal; hence, significant negative linear trend found.
Results showed that formality of a design affects design performance, such that as
the level of formality increases, the number of changes made (total, quality and expected
changes) decreases, and vice versa; demonstrating a negative linear relationship between
formality and design performance. Moreover, results showed that formality of designs
affected both the experts and novices, and that experts performed at a higher level in
comparison to novices’ performance (demonstrated by the significant between-subjects
effects). Subjective measurements included overall enjoyment and design tool
preferences. Subjects enjoyed working on designs that appeared more formal (higher
formality – i.e. more beautified) more than designs that appeared rougher and less formal
(lower formality – i.e. less beautified). There was no difference found in the preference
between designing on paper compared to designing on the tablet PC (InkKit) during the
experiment. On the other hand, results showed that design tool preference(s) in real world
design situations was more diverse than design medium/tool preference during the
experiment.
Important implications arose from this study include: 1) design education on the
effects of formality as a result of beautification, and the caution one should take when
using beautification functions in informal sketching-based tools such as InkKit; 2)
improvements on the design process such as easier preparation for client presentation and
improved efficiency which could leave more time for actual ‘designing’; and 3) informal
sketching-based tool development, in particular InkKit, to support more satisfying, natural
designer-design tool interaction.
As beautification has been regarded as an important aspect of informal sketching-
based tools (Plimmer and Apperley, 2004), and that designs can be beautified to different
extents, thus appearing more or less formal; one fundamental question that arise, is
whether there is an optimal level of beautification to create the appropriate level of
formality of designs – this is particularly important to understand for designers in practical
settings as client presentation is often frustrating; as one of the participants in Newman et
al.’s (2003) study as a design process itself. Hence, future replications and extensions to
the present study will provide a more comprehensive model of relationships between
different levels of beautification, formality and design performance. This, in turn, will
help nurture our understanding on designers’ interaction with informal sketch-based tools
in the context of beautification; hence, a new path and angle towards design process
research.
References
Adams, R. S., Turns, J., & Atman, C. (2003). What could design learning look like? In N. Cross and E. Edmonds (Eds.), Expertise in Design. University of Technology, Sydney, Australia: Creativity and Cognition Press.
Agar, P., & Novins, A. (2003). Polygon recognition in sketch-based interfaces with immediate and continuous feedback. Paper presented at the Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia, Melbourne, Australia.
Akin, O. (1986) Psychology of Architectural Design, Pion, London.
Akin, O., & Lin, C. (1995). Design protocol data and novel design decisions. Design Studies: Analysing Design Activity, 16(2), 211-236.
Alavi, M. (1984). An assessment of prototyping approach to information systems development. Communications of the ACM, 27(6), 556-563.
Aliakseyeu, D., Martens, J.-B., & Rauterberg, M. (2006). A computer support tool for the early stages of architectural design. Interacting with Computers, 18(4), 528-555.
Alvarado, C., & Davis, R. (2006). Resolving ambiguities to create a natural computer-based sketching environment. Paper presented at the International Conference on Computer Graphics and Interactive Techniques, Boston, Massachusetts.
Arditi, A. (2004). Adjustable typography: An approach to enhancing low vision text accessibility. Ergonomics, 47(5), 469-482.
Arditi, A., & Cho, J. (2005). Serifs and font legibility. Vision Research, 45, 2926–2933.
Arvo, J., & Novins, K. (2000). Fluid Sketches: Continuous recognition and morphing of simple hand-drawn shapes. Paper presented at the Proceedings of UIST ’00, San Diego, 73-80.
Arvo, J., & Novins, K. (2005). Appearance-preserving manipulation of hand-drawn graphs. Paper presented at the Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East Asia, Dunedin, New Zealand.
Atman, C. J., Cardella, M. E., Turns, J., & Adams, R. (2005). Comparing freshman and senior engineering design processes: an in-depth follow-up study. Design Studies, 26(4), 325-357.
Atman, C. J., Chimka, J. R., Bursic, K. M., & Nachtmann, H. L. (1999). A comparison of freshman and senior engineering design processes. Design Studies, 20(2), 131-152.
Bakera, E. L., & O’Neil Jr, H. F. (2002). Measuring problem solving in computer environments: current and future states. Computers in Human Behavior, 18(6), 609–622.
Bailey, B. P., & Konstan, J. A. (2003). Are Informal Tools Better? Comparing DEMAIS, pencil and paper, and authorware for early multimedia design. CHI '03 Proceedings of the SIGCHI conference on Human factors in computing systems 5(1), 313-320.
Beaudouin-Lafon, M., & Mackay, W. (2003). Prototyping tools and techniques. In J. A. Jacko & A. Sears (Eds.), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications (pp. 1-1277). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Benyon, D., Davies, G., Keller, L., Preece, J., & Rogers, Y. (1993). A Guide to Usability: Human Factors in computing. Workingham, England: Addison-Wesley.
Bernard, M., Mills, M., Peterson, M., & Storrer, K. (2001, July). A comparison of popular online fonts: Which is best and when? Usability News, 3(2). Retrieved June 13, 2006, from, http://psychology.wichita.edu/surl/usabilitynews/3S/font. htm
Bilda, Z., & Demirkan, H. (2003). An insight on designers' sketching activities in traditional versus digital media. Design Studies, 24(1), 27-50.
Bilda, Z., Gero, J. S., & Purcell, T. (2006). To sketch or not to sketch? That is the question. Design Studies, In Press, Corrected Proof.
Black, A. (1990). Visible planning on paper and on screen: The impact of working medium on decision-making by novice graphic designers. Behaviour & Information Technology, 9(4), 283-296.
Boehm, B. W., Gray, T. E., & Seewaldt, T. (1984). Prototyping verses specifying: A multiproject experiment. IEEE Transactions on Software Engineering, SE-10(3), 290-303.
Bolz, D. (1993, February). Some aspects of the user interface of a knowledge based beautifier for drawings. Paper presented at the Proceedings of the 1st international conference on Intelligent user interfaces Orlando, Florida, United States, 45-52.
Bo-Yu, S. C. (2003, February). Sketch: A domain-independent system for sketch recognition. Paper presented at the Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia, Melbourne, Australia
Boutsen, L., & Humphreys, G. W. (1999). Axis-alignment affects perceptual grouping: Evidence from simltanagnosia. Cognitive Neuropsychology, 17(7), 655-672.
Braisby, N., & Gellatly, A. (Eds.). (2005). Cognitive Psychology. Oxford: Oxford University Press.
Brink, T., Gergle, D., & Wood, S. D. (2002). Design web site that works: Usability for the web. San Francisco: Morgan Kaufmann Publishers.
Burns, C. M., & Vicente, K. J. (2000). A participant-observer study of ergonomics in engineering design: How constraints drive design process. Applied Ergonomics, 31(1), 73-82.
Carroll, J. M. (2006). Introduction to this special issue on foundation of design in HCI. Special Issue of Human-Computer Interaction, 21(1), 1-3.
Carroll, J. M., & Rosson, M. B. (1991). Deliberated evolution: Stalking the view matcher in design space. Human-Computer Interaction, 6(3/4), 197-200.
Carroll, J M., Thomas, J C & Mallaotra, A. (1980). Presentation and representation in design problem-solving. British Journal of Psychology, 71, 143-153.
Cheutet, V., Catalano, C. E., Pernot, J. P., Falcidieno, B., Giannini, F., & Leon, J. C. (2005). 3D sketching for aesthetic design using fully free-form deformation features. Computers & Graphics, 29(6), 916-930.
Christiaans, H & Dorst, C. (1992). Cognitive models in industrial design engineering: A protocol study. In D. L. Taylor & D. A. Stauer (Eds.), Design Theory and Methodology - DTM92. American Society of Mechanical Engineers, New York, USA.
Chung, R., Mirica, P., & Plimmer, B. (2005, July 6-8). InkKit: a generic design tool for the tablet PC Paper presented at the Proceedings of the 6th ACM SIGCHI New Zealand chapter's international conference on Computer-human interaction: making CHI natural CHINZ '05, Auckland, NZ.
Crampton Smith, G., & Tabor, P. (1996). The role of the artist-designer. In T. Winogard (Eds.), Bringing design to software (pp. 37-57). New York: ACM Press.
Cross, N. (1985). Styles of learning, designing and computing. Design Studies, 6, 157-162.
Cross, N. (2004). Expertise in design: an overview. Design Studies: Expertise in Design, 25(5), 427-441.
Cross, N., Christiaans, H., & and Dorst, K. (1994). Design expertise amongst student designers. Journal of Art and Design Education, 13(1), 39–56.
Cross, N., Christiaans, H., & Dorst, K. (Eds.) (1996). Analysing design activity. John Wiley & Sons Ltd., Chichester, UK.
Czaja, S. J., Sharit, J., Ownby, R., Roth, D. L., & Nairs, S. (2001). Examining age differences in performance of a complex information search and retrieval task. Psychology and Aging, 16, 564-579.
Czerwinski, M. (2003). Humans in human-computer interaction. In J. A. Jacko & A. Sears (Eds.), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications (pp. 1-1277). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Damm, C. H., & Hansen, H. R. (2002). Ideogramic. http://www.ideogramic.com/ Retrieved 27 April, 2006.
Darke, J. (1979). The primary generator and the design process. Design Studies, 1(1), 36-44.
Dickinson, J. K., Yu, Z., Zeng, Y., & Antunes, H. (2005). Pen-tablet as a CAD interface alternative. Robotics and Computer-Integrated Manufacturing, 21(4-5), 465-474.
Dillon, A. (2003). Designing usable electronic text (2nd ed.). New York: Taylor & Francis.
Downton, P. (2003). Design research. Melbourne: RMIT University Press.
Dumas, J., & Redish, G. (Revised Edition) (1999). A practical guide to usability testing. London: Intellect Books.
Eisentraut, R. (1999). Styles of problem solving and their influence on the design process. Design Studies, 20(5), 431-437.
Eisentraut, R., & Gunther, J. (1997). Individual styles of problem solving and their relation to representations in the design process. Design Studies: Descriptive models of design, 18(4), 369-383.
Elias, J. S. (2005). The engineering design mind: A cognitive model and a case study. Dissertation Abstracts International: Section B: The Sciences and Engineering, 66, 11B (UMI No. 3194788).
Ericsson, K. A., & Simon, H. A. (1984). Protocol Analysis. Cambridge: MIT Press.
Fields, C. (2004). The role of aesthetics in problem solving: Some observations and a manifesto. Journal of Experimental and Theoretical Artificial Intelligence, 16(1), 41-55.
Fitzmaurice, G. W., Balakrishnan, R., Kurtenbach, G., & Buxton, B. (1999, May). An Exploration into Supporting Artwork Orientation in the user interface. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit Pittsburgh, Pennsylvania, United States
Forbus, K. D., Ferguson, R. W., & Usher, J. M. Towards a computational model of sketching, Intelligent User Interfaces, 77-83.
Fowler, S., & Stanwick, V. (2004). Web application design handbook: Best practices for web-based software. Boston: Morgan Kaufmann Publishers.
Gasser, M., Boeke, J., Haffernan, M., & Tan, R. (2005). The influence of Font type on information recall. North American Journal of Psychology, 7(2), 181-188.
Goel, V. (1996). Sketches of thoughts. Cambridge, Massachusetts: The MIT Press.
Goldschmidt, G. (1991). The dialectics of sketching. . Creativity Research Journal, 4(2), 123-143.
Goldschmidt, G. (1994). On visual design thinking: The vis kids of architecture, Design Studies, 15, 158-174.
Goldschmidt, G. (1995). The designer as a team o f one. Design Studies: Analysing Design Activity, 16(2), 189-209.
Goldschmidt, G. (1997). Capturing indeterminism: representation in the design problem space. Design Studies, 18(4), 441–445.
Goldschmidt, G. (2003). The backtalk of self-generated sketches. Design Issues, 19(1), 72-88.
Grandjean, E., Hunting, W., & Pidermann, M. (1983). VDT workstation design: Preferred settings and their effects. Human Factors, 25(2). 161-175.
Grant, E. R., & Spivey, M. J. (2003). Eye movements and problem solving: Guiding attention guides thought. Psychological Science, 14(5), 462-466.
Green, A. J. K., & Gilhooly, K. (2005). Problem solving. In N. Braisby & A. Gellatly (Eds.), Cognitive Psychology (pp. 344-381). Oxford: Oxford University Press.
Grekas, D. N., & Frangopoulos, C. A. (2000). A heuristic algorithm for drawing of a flow diagram. Advances in Engineering Software, 32(3), 239-253.
Grosjean, S. F., P; Brassac, C. (2000). Those "psychological tools" inside the design process. Knowledge-Based Systems, 13(1), 3-9.
Gross, M. D. (1996). The electronic cocktail napkin: A computational environment for working with design diagrams. Design Studies, 17, 53-69.
Gross, M. D., & Do, E. Y.-L. (1996, April 13-18). Demonstrating the electronic cocktail napkin: a paper-like interface for early design. Paper presented at the Conference on Human Factors in Computing Systems - CHI'96 Proceedings.
Gross, M. D., Do, E.Y.-L., McCall, R.J., Citrin, W.V., Hamill, P., Warmack, A., Kuczun, K.S. (1998). Collaboration and coordination in architectural design: approaches to computer mediated team work. Automation in Construction 7(6), 465-473.
Hamade, R. F., Artail, H. A., & Jaber, M.Y. (2005). Learning Theory as Applied to Mechanical CAD: Training of Novices. International Journal of Human-Computer Interaction, 19(3), 305-322.
Hanna, R., & Barber, T. (2001). An inquiry into computers in design: attitudes before-attitudes after. Design Studies, 22(3), 255-281.
Hannah, G. G. (2002). Elements of design: Rowena Reed Kostellow and the structure of visual relationships . New York: Princeton Architectural Press.
Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interaction, 19(4), 319-349.
Hegarty M (1991) Knowledge and processes in mechanical problem solving. In: Sternberg, R. J., Frensch, P. A. (Eds.), Complex problem solving: Principles and mechanisms. Hillsdale: Erlbaum.
Helander, M. (Ed.). (1988). Handbook of Human-Computer Interaction. New York: Elsevier Science Publishers.
Henzen, A., Ailenei, N., Fiore, F. D., & Patterson, F. V. R. J. (2005). Sketching with a low-latency electronic ink drawing tablet. Paper presented at the Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East Asia Dunedun, New Zealand.
Ho, C.-H. (2001). Some phenomena of problem decomposition strategy for design thinking: Differences between novices and experts. Design Studies, 22, 27-45.
Holt, K. (1991). The impact of technology strategy on the engineering design process. Design Studies, 12(2), 90-95.
Hong, J. I., & Landay, J. A. (2006). SATIN: a toolkit for informal ink-based applications. Paper presented at the International Conference on Computer Graphics and Interactive Techniques, Boston, Massachusetts.
Houde, S & Hill, C. (1997). What do prototypes prototype? In M Helander, T Landauer and P Prabhu (2nd
ed. rev.), Handbook of human-computer interaction (pp367-381). Elsevier Science: Amsterdam.
Hse, H. H., & Newton, A. R. (2005). Recognition and beautification of multi-stroke symbols in digital ink. Computers & Graphics, 29(4), 533–546.
Jacko, J., & Stephanidis, C. (Eds.). (2003). Human-Computer Interaction: Theory and practice (Part 1) (Vol. 1). Mahwah, New Jersey: Lawrence Erlbaum Associates.
Jacko, J. A., & Sears, A. (Eds.). (2003). The Human-Computer Interaction handbook: Fundamentals, evolving technologies and emerging applications. Mahwah, New Jersey: Lawrence Erlbaum Associates.
Joiner, R., Messer, D., Light, P., & Littleton, K. (1998). It is best to point for young children: A comparison of children’s pointing and dragging. Computers in Human Bahaviour, 14, 513-529.
Jonson, B. (2005). Design ideation: the conceptual sketch in the digital age. Design Studies, 26(6), 613-624.
Kara, L. B., & Stahovich, T. F. (2004). Sim-U-Sketch: a sketch-based interface for SimuLink. Paper presented at the Proceedings of the working conference on Advanced visual interfaces, Gallipoli, Italy.
Kavakli, M., Scrivener, S. A. R., & Ball, L. J. (1998). Structure in idea sketching behaviour. Design Studies, 19(4), 485-517.
Khalid, H. M. (2001). Modelling and analysis of CAD expert behaviour in using manual input devices. International Journal of Industrial Ergonomics, 27(2), 79-92.
Kim, D. H., & Kim, M. J. (2006). A new modeling interface for the pen-input displays. Computer-Aided Design, 38(3), 210-233.
Koffka, K. (1935). Principles of Gestalt psychology. London: Routledge.
Kramer, A. (1995, May 7-11). Supporting design activities in the written medium. Paper presented at the CHI' 95 Mosaic of Creativity, Denva, Colorado, USA.
Kruger, C., & Cross, N. (2006). Solution driven versus problem driven design: strategies and outcomes. Design Studies, In Press, Corrected Proof.
Kvan, T., & Yunyan, J. (2005). Students’ learning styles and their correlation with performance in architectural design studio. Design Studies, 26, 19-34.
Lahti, H., Seitamaa-Hakkarainen, P., & Hakkarainen, K. (2004). Collaboration patterns in computer supported collaborative designing. Design Studies, 25(4), 351-371.
Landauer, T. K. (1996). The trouble with computers: Usefulness, usability and productivity. Cambridge MA: MIT Press.
Landay, K. A. (1996, April). SILK: Sketching interfaces like kraz. Paper presented at the Conference on Human factors in computing systems, Vancouver, British Columbia, Canada
Landay, J. A., & Myers, B. A. (2001, March). Sketching interfaces: Toward more human interface design. IEEE Computer, 34(3), 56-64.
Lanier, P. (2004). My Font Tool for Tablet PC (Version 1.0) [Computer software]. Microsoft Corporation.
Laurel, B., & Mountford, S. J. (1990). Art of HCI Interface design. New York: Addison-Wesley Publishing Co.
Launis, M., Vuori, M., & Lehtela, J. (1996). Who is the workplace designer? -- Towards a collaborative mode of action. International Journal of Industrial Ergonomics Computer-aided Ergonomics and Safety, 17(4), 331-341.
LaViola, J. J., & Zeleznik, J. R. C. (2006). MathPad2: a system for the creation and exploration of mathematical sketches. Paper presented at the International Conference on Computer Graphics and
Interactive Techniques, Boston, Massachusetts.
Levinson, J. (Ed.). (2003). The oxford dictionary of aesthetics. New York: Oxford University Press.
Lawson, B., & Loke, S. M. (1997). Computers, words and pictures. Design studies, 18(2), 171-183.
Lewis, J. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7, 57-78.
Lin, J., Newman, M. W., Hong, J. I., & Landay, J. A. (2000, 1-6 April). DENIM: Finding a tighter fit between tools and practice for Web site design. Paper presented at the Proceedings of the SIGCHI conference on Humana factors in computer systems, The Hague, The Netherlands.
Ling, J., & Van Schaik, P. (2006). The influence of font type and line length on visual search and information retrieval in web pages. International Journal of Human-Computer Studies, 64(5), 395-404.
Liu, Y.-J., Tang, K., & Joneja, A. (2005). Sketch-based free-form shape modelling with a fast and stable numerical engine. Computers & Graphics, 29(5), 771-786.
Lloyd, P., Lawson, B., & Scott, P. (1995). Can concurrent verbalization reveal design cognition? Design Studies, 16(2), 237-259.
Locher, P. J., Stappers, P. J., & Overbeeke, K. (1998). The role of balance as an organizing design principle underlying adults' compositional strategies for creating visual displays. Acta Psychologica, 99(2), 141-161.
Luczak, H., & Springer, J. (1997). Ergonomics of CAD systems. In M. G. Helander, Landauer, T.K., Prabhu, P.V. (Ed.), Handbook of Human Computer Interaction (pp. 1349-1394). Amsterdam: North-Holland.
Van der Lugt, R. (2005). How sketching can affect the idea generation process in design group meetings. Design Studies, 26(2), 101-122
Maarse, K. F., Schomaker, L. R. B., & Teulings, H.-L. (1988). Automatic identification of writers. In G. C. c. d. Veer & G. Mulder (Eds.), Human-Computer Interaction: Psychonomic Aspects (pp. 335-352). Berlin, Heidelberg: Springer-Gerlag.
MacLean, A., Young, R. M., Bellotti, V. M. E., & Morgan, T. P. (1991). Questions, options, and criteria: Elements of design space analysis. Human-Computer Interaction, 6, 319-355.
May, J. (2001). Human-Computer Interaction. In International Encyclopedia of the Social & Behavioral Sciences (pp. 7031-7035). Oxford: Pergamon.
Maybew, D. J. (2003). Requirements specification within the usability engineering life cycle. In J. A. Jacko & A. Sears (Eds.), The Human-Computer Interaction handbook: Fundamentals, evolving technologies and emerging applications (pp. 264-285). Mahwah, New Jersey: Lawrence Erlbaum Asssociates.
McGown, A., Green, G., & Rodgers, P. A. (1998). Visible ideas: information patterns of conceptual sketch activity. Design Studies, 19 (4), 431–453.
Molich, R., & Nielsen, J. (1990). Improving a human-computer dialogue. Communications of the ACM, 33, 338-348.
Mulet, E., & Vidal, R. (2006). Functional requirements for computer-based design support systems, derived from experimental studies. Knowledge-Based Systems, 19(1), 32–42.
Mullen, M. (1991). PICTIVE: An exploration in participatory design. Proceedings of ACM Conference on Human Factors in Computing System (CHI’ 91) (pp. 225-231). New York: ACM Press.
Naga, Y., & Noguchi, H. (2003). An experimental study on the design thinking process started from difficult keywords: modeling the thinking process of creative design. Journal of Engineering Design, 14(4), 429–437.
Nakakoji, K., Tanaka, A., & Fallman, D. (2006, April). "Sketching" nurturing creativity: commonalities in art, design, engineering and research. Paper presented at the CHI '06 extended abstracts on Human factors in computing systems CHI '06 Montréal, Québec, Canada.
Nakarada-Kordic, I. (2004). Web aesthetics and usability : The effect of perceived attractiveness of web interface design on visual search of web sites. Unpublished Master of Science Thesis, University of Auckland, Auckland, NZ.
Nealen, A., Sorkine, O., Alexa, M., & Cohen-Or, D. (2005). A sketch-based interface for detail-preserving mesh editing. Paper presented at the International Conference on Computer Graphics and Interactive Techniques, LA, California.
Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. San Francisco: Freeman.
Newman, M. W., Lin, J., Hong, J. I., & Landay, J. A. (2003). DENIM: An informal web site design tool inspired by observations of practice. Human-Computer Interaction, 18(3), 259-325.
Newell. A. (1968). On the analysis of human problem solving protocols. In J. C. Gardin & B. Jaulin (Eds.). Calcul et formalization dans les sciences de l’homme (pp. 146-185). Paris: Centre National dela Recherche Scientific.
Nielsen, J., & Mack, R. L. (Eds) (1994). Usability inspection methods. New York: John Wiley.
Overbeeke, K., & Wensveen, S. (2004). Beauty in use. Human-Computer Interaction, 19(4), 367-369.
Oh, J.-Y. (2004). Design Evaluater: Critiquing freehand sketches. Masters of Science thesis, Design Machine Group, Department of Architecture. University of Washington, Seattle.
Oh, J.-Y., Stuerzlinger, W., & Danahy, J. (2006). SESAME: Towards better 3D conceptual design systems. Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS, 2006, 80-89.
Okabe, M., Owada, S., & Igarashi, T. (2005). Interactive design of botanical trees using freehand sketches and example-based editing. Eurographics, 24(3), 487-496.
Pancer, S. M., George, M., & Gebotys, R. J. (1992). Understanding and predicting attitudes toward computers. Computers in Human Behavior, 8, 211-222.
Plimmer, B., & Apperly, M. (2001). Computer-Aided sketching to capture preliminary design. Paper presented at the Third Australasian User Interfaces Conference (AUIC2002), Melbourne, Australia. In J. Grundy & P. Calder (Ed.), Conferences in Research and Practice in Information Technology (pp 9-12). Australian Computer Society.
Plimmer, B. E., & Apperley, M. (2003). Software for Students to Sketch Interface Designs. In Proceedings of Interact, Zurich, 73-80.
Plimmer, B., & Apperley, M. (2004). Interacting with sketched interface designs: An evaluation study. Paper presented at the CHI '04 Conference on Human Factors in Computing Systems archive, Vienna, Austria.
Plimmer, B. E, & Grundy, J. (2005, January). Beautifying sketching-based design tool content: issues and experiences. Paper presented at the Proceedings of the Sixth Australasian conference on User interface (AUIC2005), Newcastle.
Pomm, C., & Werlen, S. (2004, May). Smooth morphing of handwritten text. Paper presented at the Proceedings of the working conference on Advanced visual interfaces, Gallipoli, Italy.
Poulton, E. C. (1965). Letter differentiation and rate of comprehension in reading. Journal of Applied Psychology, 49(5), 358-362.
Purcell, A. T., & Gero, J. S. (1998). Drawings and the design process: A review of protocol studies in design and other disciplines and related research in cognitive psychology. Design Studies, 19(4), 389-430.
Qin, S. F., Wright, D. K., & Jordanov, I. N. (2000). From on-line sketching to 2D and 3D geometry: a system based on fuzzy knowledge. Computer-Aided Design, 32(14), 851-866.
Qin, S. F., Wright, D. K., & Jordanov, I. N. (2001). A conceptual design tool: a sketch and fuzzy logic based system. Proceedings of the Institution of Mechanical Engineers - Part B: Engineering Manufacture, 215(1), 111-116.
Reason, J. (1990). Human error. Cambridge: Cambridge University Press.
Romer, A., Leinert, S., & Sachse, P. (2000). External support of problem analysis in design problem solving. Research in Engineering Design: Theory, Applications, and Concurrent Engineering, 12(3), 144–151.
Rosson, M. B., & Carroll, J. M. (2003). Scenario-based design. In J. A. Jacko & A. Sears (Eds.), The Human-Computer Interaction handbook: Fundamentals, evolving technologies and emerging applications (pp. 264-285). Mahwah, New Jersey: Lawrence Erlbaum Asssociates.
Rowe, P. G. (1987). Design thinking. MIT Press, Cambridge, MA.
Rubine, D. (1991). Specifying gestures by example. Computer Graphics (SIGGRAPH '91 Proceedings), 24(5), 329-337.
Rupp, M., Burg, A., & Beck, E. (2003). Rapid prototyping for wireless designs: the five-ones approach. Signal Processing, 83(7), 1427-1444.
Sachse, P., Leinert, S., & Hacker, W. (2001). Designing with computer and sketches. Swiss Journal of Psychology, 60 (2), 65–72.
Sano, D. (1996). Designing large-scale Web sites: A visual design methodology. New York, Wiley.
Sanocki, T. (1998). Font regularity constraints on the process of letter recognition. Journal of Experimental Psychology: Human Perception and Performance, 14(3), 472-480.
Saund, E., & Moran, T. P. (1994). A perceptually-supported sketch editor. Paper presented at the ACM Symposium on User Interface Software and Technology, Marina del Ray. CA.
Savage, J. C. D., Miles, C., Moore, C. J., & Miles, J. C. (1998). The interaction of time and cost constraints on the design process. Design Studies, 19(2), 217-233.
Scaife, M. and Rogers, Y. (1996). External cognition: How do graphical representations work? International Journal of Human-Computer Studies, 45, 185-213.
Schenk, P. (1991). The role of drawing in the graphic design process. Design Studies, 12(3), 168-181.
Schweikardt, E., & Gross, M. D. (2000). Digital clay: deriving digital models from freehand sketches. Automation in Construction, 9(1), 107-115.
Sear, A, Karat, C.-M., Oseitutu, K., Karimullah, A., & Feng, J. (2001). Productivity, satisfaction, and interaction strategies of individual with spinal cord injuries and traditional users interacting with speech recognition software. Universal Access in the Information Society, 1, 1-12.
Sezgin, T. M., Stahovich, T., & Davis, R. (2001, November). Sketch based interfaces: early processing for sketch understanding. Paper presented at the Proceedings of the 2001 workshop on Perceptive user interfaces, Orlando, Florida.
Shelly, G. B., Cashman, T. J., & Woods, D. M. (2005). HTML: Complete concepts and techniques (Third Ed.). Shelly Cashman Series, Australia.
Shesh, A., & Chen, B. (2004). SMARTPAPER: An interactive and user friendly sketching system. Eurographics, 23(3), 301-310.
Shneiderman, B., Hewett, T., Fischer, G., Jennings, P., Czerwinski, M., Kules, B., et al. (2006). Creativity support tools: Report from a U.S. National Science Foundation sponsored workshop. International Journal of Human-Computer Interaction, 20(2), 61-77.
Smith, G.F., & Browne, G.J. (1993). Conceptual foundations of design problem solving. IEEE Trans Systems, Man and Cybernetics, 23(5), 1209–1219.
Spector, P. E. (Ed.). (2006). Industrial and organizational psychology: research and practice: Hoboken, New Jersey: Wiley & Sons.
Stammerjohn, L. W., Smith, M. J., & Cohen, B. G. F. (1981). Evaluation of work station design factors in VDT operations. Human Factors, 23(4), 401-412.
Stempfle, J., & Badke-Schaub, P. (2002). Thinking in design teams - an analysis of team communication. Design Studies, 23(5), 473-496.
Taggart, J. (1975). Sketching, an informal dialogue between designer and computer. In N. Negroponte (Ed.). Reflections on computer aids to design and architecture (pp 147-162). New York: Petrocelli Charter.
Tang, G. L. (2005). Ink Kit: The user interface of the generic design transformational tool. Honours project report, University of Auckland, New Zealand.
Thomassen, A. J. W. M., Teulings, H.-L., & Schomaker, L. R. B. (1988). Real-time processing of cursive writing and sketched graphics. . In G. C. c. d. Veer & G. Mulder (Eds.), Human-Computer Interaction: Psychonomic Aspects (pp. 335-352). Berlin, Heidelberg: Springer-Gerlag.
Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13(2), 127-145.
Tovey, M. (1997). Styling and design: intuition and analysis in industrial design. Design Studies, 18(1), 5-31.
Turner, A., Chapman, D., & Penn, A. (2000). Sketching space. Computers & Graphics, 24(6), 869-879.
Tversky, B. (1999). What does drawing reveal about thinking? Visual and Spatial Reasoning in Design, MIT.
Ullman, D. G., Wood, S., & Craig, D. (1990). The importance of drawing in the mechanical design process. Computers & Graphics, 14(2), 263-274.
Van Dijk, C. G. C., & Mayer, A. A. C. (1997). Sketch input for conceptual surface design. Computers in Industry, 34(1), 125-137.
Van Someren, M W, Barnard, Y., & Sandberg, J. (1994). The think aloud method: A practical guide to modeling cognitive processes. Academic Press, London, UK.
Van der Veer, G. C. c. d., & Mulder, G. (Eds.). (1988). Human-Computer Interaction: Psychonomic Aspects. Berlin, Heidelberg: Springer-Gerlag.
Verstijnen, I. M., Van Leeuwen, C., Goldschmidt, G., Hamel, R., & Hennessey, J. M. (1998). Sketching and creative discovery. Design Studies, 19(4), 519-546.
Wagner, A. (1990). Prototyping: A day in the life of an interface designer. In B. Laurel (Eds.) The art of human-computer interface design (pp. 79-84). Addison-Wesley: Reading MA.
Wang, B., Sun, J., & Plimmer, B. (2005, July 6-8). Exploring sketch beautification techniques. Paper
presented at the Proceedings of the 6th ACM SIGCHI New Zealand chapter's international conference on Computer-human interaction: making CHI natural CHINZ '05, Auckland, New Zealand.
Ward, A. (1989). Phenomenological analysis in the design process. Design Studies, 10(1), 53-66.
Welch, M. (1999). Analyzing the tacit strategies of novice designers. Research in Science & Technological Education, 17(1), 19-34.
Wilpert, B. (2006). Psychology and design processes. Safety Science, In Press, Corrected Proof.Wong, Y.Y. (1992). Rough and ready prototypes: Lessons from graphic design. Paper presented at the
Proceedings of Human Factors in Computing Systems CHI '92, Monterey, 83-84.
Yang, M. C. (2005). A study of prototypes, design activity and design outcome. Design studies, 26, 649-669.
Young, M. (2005). Ink Kit: The back end of the generic design transformation tool. Honours project report, University of Auckland, New Zealand.
Zeleznik, R. C., Herndon, K. P., & Hughes, J. F. (2006). SKETCH: an interface for sketching 3D scenes. Paper presented at the International Conference on Computer Graphics and Interactive Techniques, Boston, Massachusetts.
Zenka, R., & Slavík, P. (2003, April). Rendering: New dimension for sketches Paper presented at the Proceedings of the 19th spring conference on Computer graphics.
Appendices
Appendix A. The five designs and the outline of design errors present in each design
Appendix A1.1. Low formality design on paper – Online Magazine subscription form
(item set)Label on left & radio button on the right
Change sides: radio button on left, label on the right
12 Contact No. Dropdown menu Textbox13 Contact No. 1 Textbox Split into 2 Textboxes (1 smaller
one for the area code)14 Ethnic group Ethnicity item No Item15 Last day of
courseTextboxes in item set Dropdown menus in item set
16 Give speech “Yes, I want…” & “No, I don’t want…”
No Item
17 Payment No Item - Dropdown menu + labels OR- Radio Buttons + labels
18 Mailing Address: House/Street
Dropdown Textbox
19 Want to borrow: Radio buttons in item set
Check boxes in item set
20 Hood color 1 Dropdown 2 dropdown 21 And the hood
color isRadio button on left hand side
None (just label and dropdown)
22 Continue to next page
Question type “Proceed to next page” button
Appendix A4.1. Medium-high formality design on tablet PC – Dog Registration
Form
Appendix A4.2. Dog Registration’s: Planned design “errors”
Item Change from: Change to:1 Title Label: “Registation” Registration2 Title Label: “Applicaiton” Application3 First owner’s name One textbox 2 textboxes for First name and
Last name4 2nd owner’s name One textbox 2 textboxes for First name and
Last name5 Address 2 One line of input 2 lines of input: Item
“Town/City” with a textbox6 Contact No. No item Add Contact Number item (with
2 textboxes)7 Sterilized “Yes”, “No” Add “Don’t know” item8 Sterilized Label on left & radio
button on the rightChange sides: radio button on left, label on the right
9 Gender Dropdown menu Radio buttons & labels: “ Female” and “Male”
10 Age Long dropdown Shorter dropdown11 Appearance Textboxes in item set Dropdown menus in item set12 Registration with vet Textbox - Radio Buttons & labels: “Yes”
and “No” OR- Checkbox
13 Registered with Dog lovers society
Existing item None
14 Dog’s special Conditions
Dropdown - Text Area OR- checkboxes
15 Purpose of the dog Textbox - Dropdown OR- Radio buttons
19 The shop is: Dropdown Textbox20 The shop is: Radio button on left
hand sideNone (just label and dropdown)
21 Proceed to next page No item “Proceed to next page” Button(s) 22 Dog’s information Scattered around Group dog’s information with
label(s) to separate from owner’s info
Appendix A5.1. High formality design on tablet PC – America’s Next Top Model application form
Appendix A5.2. America’s Next Top Model: Planned design errorsItem Change from: Change to:
1 Title “American’s” “America’s”2 Title “Applicattion” “Application”3 Full Name 1 Textbox 2 Textboxes & Labels: “First Name”,
“Last Name”4 Town/City Textbox Dropdown menu5 State Textbox Dropdown menu6 Contact No. 1 Textbox 2 Textboxes (1 smaller one for the
area code)7 Status “Mr”, “Mrs” “Married”, “Single” or “In a
relationship”8 Status Label on the left &
radio button on the right
Radio button on left, label on the right
9 Status Only 2 choices Add one more item to choose “Married”/ “Single”/ “In a relationship”
10 Gender Gender Item None11 Age Long dropdown menu Shorter menu for 2 digit number12 Date of birth Textboxes in item set Dropdown menus in item set13 Height Textbox Dropdown menu14 Weight Textbox Dropdown menu15 Why Enter Dropdown menu Text Area 16 Occupation Textbox Dropdown17 Experience Dropdown menu Text Area18 Experience “Experience” “Modeling experience”19 Heard from Radio buttons in item
21 Other reasons Dropdown menu Textbox22 Submit? Submit question Submit button
Appendix B. Post-task Questionnaire
Exploring the effects of different degrees of beautification on the design process
Post-task Questionnaire
Please answer the questions below by ticking the appropriate box or writing in your answer. Do not write your name anywhere on this form as your recorded information should remain anonymous. Take as much time as you need to answer those questions.
Part 1
1a. What is your age (years)? _________________ 1b. Gender: Female Male
2. What is your highest / current education level (please circle appropriately)?
Secondary
Other technical or professional training (please specify): _____________________
Tertiary (please specify the following if you have graduated):
Graduated at: _________________________ In year: ____________
Other Institution (please specify):__________________________
Study Programme: _________________________________________
What is your major/specialization? ____________________________
How many years have you been studying at university/other
institutions for? _______________
(Please continue to page 2)
Please indicate what paper(s) you have taken and/or are taking currently (please tick appropriately). Or write paper name(s) at the space provided below the table.
Tick Paper(s) (or Equivalent)SOFTENG 450 – Software Development Methodologies
COMPSCI 101 - Principles of Programming
COMPSCI 105 - Principles of Computer Science
COMPSCI 230 - Software Design and Construction
COMPSCI 280 - Applications Programming
COMPSCI 345 - Human Computer Interaction
OTHER papers: __________________________________________________
Please notify the experimenter that you have finished filling in this page.
(Please continue to page 3)
5. Please rank, in order, which design you enjoyed working on the most to the design you enjoyed working on the least. i.e. put “1” as the most-liked design, “2” as the second most-liked design…and so on, to “5” as the least-liked design. Please also state the reason if you can. The Experimenter will now show you the 5 designs that you have worked on.
Forms Rank Reasons
1st Form
2nd Form
3rd Form
4th Form
5th Form
6. In this experiment, did you prefer working on the Tablet PC or on paper? Why?
That’s all for now! Thank you very much for participating! Please give this to the researcher when you have finished.
APPROVED BY THE UNIVERSITY OF AUCKLAND HUMAN PARTICIPANTS ETHICS COMMITTEE on 13/04/2006 for a period of 3 years from 13/04/2006. Reference 2006 / 045.
Appendix C. Results of post-task questionnaireDescriptive statistics of participant’s characteristics (N=30)
Measure Items Frequency PercentGender: Male (Mean age = 22.81) 16 53.3
Preference of design medium: In the experiment No preference 2 6.6
Pen and paper 13 43.3Tablet PC 15 50.0
In the real world No preference 1 3.3Paper and paper 3 10Tablet PC (Inkit) 3 10Computer (Other tools) 8 26.7Pen and paper, then Tablet PC 1 3.3Pen and paper, then Computer 14 46.7
Corrected vision Yes 30 100No 0 0
Appendix D. Participant information sheets and consent forms
Department of PsychologyThe University of AucklandTamaki CampusPrivate Bag 92019Auckland
Tel: 09 373 7599 ext 86870
PARTICIPANT INFORMATION SHEET: STUDENT
Title: Investigation of the form design process.
To participants:
My name is Louise Yeung and I am doing my thesis in partial fulfilment of the requirements for the Master of Science in the Department of Psychology at The University of Auckland. Together with my supervisors, Drs Brenda Lobb, Beryl Plimmer and Douglas Elliffe, I am investigating how people design the forms we have to fill in so often during our lives, for example, to order a book, apply for a driver’s licence or register a dog.
You are invited to participate in our research and we would appreciate any assistance you can offer us, although you are under no obligation to do so: your participation is voluntary and neither your grades nor academic relationships with the Departments of Psychology or Computer Science or members of staff will be affected whether or not you participate.
Participation involves one visit to our laboratory at The University of Auckland, for approximately one hour, and requires that your eyesight is normal or corrected-to-normal by spectacles or contact lenses. If you agree to participate, I will ask you to be seated at a desk and using a tablet (a specialized pen-driven computer), present you with five different form designs, one after the other. I will ask you to check each design using the scenario provided (e.g. register a dog, order a book, apply for credit) and make any changes you think necessary to improve the form. The changes you make and the time you spend working on each form’s design will be recorded automatically by the tablet. At the conclusion, you will be asked to note your age, education level and design experience on the tablet.
All the information you provide remains anonymous. I will give your information a code number; your name will not be recorded and no-one will be able to identify you from any of the recorded data. Your consent form will be held in a secure file for 6 years, at the end of this time it will be shredded. Your name will not be used in any reports arising from this study. Each participant will have a separate set of data, you will not see other participants’ data and they will not see yours. The computer is password protected to protect this data. The anonymous information collected during this study may be used in future analyses and publications and will be kept indefinitely. When it is no longer required all copies of the data will be destroyed. At the conclusion of the study, a summary of the findings will be available from the researchers upon request.
If you don’t want to participate, you don’t have to give any reason for your decision. If you do participate, you may withdraw at any time during the session. You can also ask for the information you have provided to be withdrawn at any time until 1/8/2006, without explanation and with penalty, by contacting me (details overleaf). If you choose not to participate, or to withdraw yourself or your information, your grades or academic relationships with the Departments of Psychology or Computer Science or members of staff will not be affected.
If you agree to participate in this study, please first complete the consent form attached to this information sheet. Your consent form will be kept separately from your data so that no-one will be able to identify you from the information you provide.
Thank you very much for your time and help in making this study possible. If you have any questions at any time you can phone me [cell phone no. to be inserted here], my supervisor Dr Brenda Lobb (09-373-7599 ext. 86870) or the Head of Department, Associate Professor Fred Seymour (3737599 ext 88414), or you can write to us at:
Department of Psychology, Tamaki CampusThe University of AucklandPrivate Bag 92019Auckland.
For any queries regarding ethical concerns, please contact The Chair, The University of Auckland Human Participants Ethics Committee, The University of Auckland, Research Office - Office of the Vice Chancellor, Private Bag 92019, Auckland. Tel. 3737599 ext 87830.
APPROVED BY THE UNIVERSITY OF AUCKLAND HUMAN PARTICIPANTS ETHICS COMMITTEE on for a period of 3 years from 2006. Reference 2006/045.
Department of PsychologyThe University of AucklandTamaki CampusPrivate Bag 92019Auckland
Tel: 09 373 7599 ext 86870
CONSENT FORM: STUDENT
THIS CONSENT FORM WILL BE HELD FOR A PERIOD OF AT LEAST SIX YEARS
Title: Investigation of the form design process
Researchers: Louise Yeung, Dr Brenda Lobb, Dr Beryl Plimmer, Dr Douglas Elliffe
I have been given and understood an explanation of this research project. I have had an opportunity to ask questions and have them answered. I understand that at the conclusion of the study, a summary of the findings will be available from the researchers upon request.
I understand that the anonymous data collected from the study will be held indefinitely and may be used in future analyses.
I understand that I may withdraw myself or any information traceable to me at any time up to 1stAugust, 2006 without giving a reason, without any penalty.
I understand that my grades and relationships within the Departments of Psychology and/or Computer Science will be unaffected whether or not I participate in this study or withdraw my participation during it.
I agree to take part in this research by completing the laboratory session.
I confirm that my eyesight is normal or corrected-to-normal.
Signed:
Name: (please print clearly)
Date:
APPROVED BY THE UNIVERSITY OF AUCKLAND HUMAN PARTICIPANTS ETHICS COMMITTEE on 13/04/06 for a period of 3 years from 2006. Reference 2006/045.
Department of PsychologyThe University of AucklandTamaki CampusPrivate Bag 92019Auckland
Tel: 09 373 7599 ext 86870
PARTICIPANT INFORMATION SHEET
Title: Investigation of the form design process
To participants:
My name is Louise Yeung and I am doing my thesis in partial fulfilment of the requirements for the Master of Science in the Department of Psychology at The University of Auckland. Together with my supervisors, Drs Brenda Lobb, Beryl Plimmer and Douglas Elliffe, I am investigating how people design the forms we have to fill in so often during our lives, for example, to order a book, apply for a driver’s licence or register a dog.
You are invited to participate in our research and we would appreciate any assistance you can offer us, although you are under no obligation to do so.
Participation involves one visit to our laboratory at The University of Auckland, for approximately one hour, and requires that your eyesight is normal or corrected-to-normal by spectacles or contact lenses. If you agree to participate, I will ask you to be seated at a desk and using a tablet (a specialized pen-driven computer), present you with five different forms, one after the other. I will ask you to check each design using the scenario provided (e.g. register a dog, order a book, apply for credit) and make any changes you think necessary to improve the form. The changes you make and the time you spend working on each form’s design will be recorded automatically by the tablet. At the conclusion, you will be asked to note your age, education level and design experience on the tablet.
All the information you provide remains anonymous. I will give your information a code number; your name will not be recorded and no-one will be able to identify you from any of the recorded data. Your consent form will be held in a secure file for 6 years, at the end of this time it will be shredded. Your name will not be used in any reports arising from this study. Each participant will have a separate set of data, you will not see other participants’ data and they will not see yours. The computer is password protected to protect this data. The anonymous information collected during this study may be used in future analyses and publications and will be kept indefinitely. When it is no longer required all copies of the data will be destroyed. At the conclusion of the study, a summary of the findings will be available from the researchers upon request.
If you don’t want to participate, you don’t have to give any reason for your decision. If you do participate, you may withdraw at any time during the session and you can also ask for the information you have provided to be withdrawn at any time until 1/8/2006, without explanation and without penalty, by contacting me (details overleaf).
If you agree to participate in this study, please first complete the consent form attached to this information sheet. Your consent form will be kept separately from your data so that no-one will be able to identify you from the information you provide.
Thank you very much for your time and help in making this study possible. If you have any questions at any time you can phone me (021- [cell phone number to be inserted here ], my supervisor, Dr Brenda Lobb (09-373-7599 ext. 86870) or the Head of Department, Associate Professor Fred Seymour (3737599 ext 88414), or you can write to us at:
Department of Psychology, Tamaki CampusThe University of AucklandPrivate Bag 92019Auckland.
For any queries regarding ethical concerns, please contact The Chair, The University of Auckland Human Participants Ethics Committee, The University of Auckland, Research Office - Office of the Vice Chancellor, Private Bag 92019, Auckland. Tel. 3737599 ext 87830.
APPROVED BY THE UNIVERSITY OF AUCKLAND HUMAN PARTICIPANTS ETHICS COMMITTEE on for a period of 3 years from 2006. Reference 2006/045.
Department of PsychologyThe University of AucklandTamaki CampusPrivate Bag 92019Auckland
Tel: 09 373 7599 ext 86870
CONSENT FORM
THIS CONSENT FORM WILL BE HELD FOR A PERIOD OF AT LEAST SIX YEARS
Title: Investigation of the form design process
Researchers: Louise Yeung, Dr Brenda Lobb, Dr Beryl Plimmer, Dr Douglas Elliffe
I have been given and understood an explanation of this research project. I have had an opportunity to ask questions and have them answered. I understand that at the conclusion of the study, a summary of the findings will be available from the researchers upon request.
I understand that the anonymous data collected from the study will be held indefinitely and may be used in future analyses.
I understand that I may withdraw myself or any information traceable to me at any time up to 1stAugust, 2006 without giving a reason, without any penalty.
I agree to take part in this research by completing the laboratory session.
I confirm that my eyesight is normal or corrected-to-normal.
Signed:
Name: (please print clearly)
Date:
APPROVED BY THE UNIVERSITY OF AUCKLAND HUMAN PARTICIPANTS ETHICS COMMITTEE on 13/04/06 for a period of 3 years from 2006. Reference 2006/045.
Appendix E. Functional Aspects of Inkit
Table 1. Steps taken to demonstrate each type of change to the participant. Change DemonstrationAdd In “Drawing Mode” – a text box, a dropdown menu, a radio
button, and a label (“Name”) were drawn using pen-input on the Tablet PC screen.
Delete In “Eraser Mode” – a text box, a dropdown menu, a radio button, and a label (“Name”) were deleted by ‘brushing’ through the unwanted object.
Resize In “Selection Mode” – an element was selected by ‘drawing’ (pointing the cursor) around the element desired on the screen in a continuous motion, then let go. The rectangle box around the element indicated that it was selected. The element was then resized by dragging a (any) corner.
Relocate While still in “Selection Mode” – a text box was selected. It was dragged (by pointing to the object on the screen and moving the pen along the screen to the desired location) and dropped (by taking the pen off the screen) to the desired place (right hand corner)
Change by annotation
In “Drawing Mode” – an arrow was drawn to indicate where an element should be placed instead of the existing location. Another arrow was drawn into the space between two elements, to indicate the addition of an element to the specified space (arrow head).
Table 2.Description of the functionality of different modes, and the visual feedback of the icon selected and highlighted (in orange). Modes Icon highlighted
(visual feedback)Functionality
Inking/drawing
Ink icon Enables the user to use the pen to ‘draw’ or ‘ink’ onto the opened page (screen) as if they are drawing on paper.
Eraser Eraser icon Enables the user to erase anything that they have sketched on the page
Selection Selection icon Enables user to select elements they have drawn on the page. When an element is selected, the user can then resize or relocate the element.
Appendix F. Instruction sheets containing the requirements and scenario associated with each design
Appendix F1. Instructions including the requirements and the scenario for the low formality (on paper) design.
Instructions:
Use the requirements stated in Part A and scenarios provided in Part B to check the design you have been given. Change the design to provide a better interface to serve its purpose.
Part A: Requirements for the form
Design an interface for subscribing the International Online Magazine. The subscriber must know that it is the International Online Magazine subscription page. The online form should be easy to fill in. The following information should be obtained from the subscriber:
Login name – the user would use this login name to get to the online magazine they are subscribed to
Password – must ensure the user correctly inputs the password that they want as the password would be encrypted when as they type it in. ‘
Email address Contact number Viewing language: English, Chinese Simplified, Chinese traditional, Thai, French,
German, Spanish, Italian, Japanese Payment – credit card/cheque/bank deposit Favourite singer Favourite movie Do they want the magazine daily, monthly, or yearly (different price) Subscriber’s full name Age – for statistical purposes and to restrict viewing options if they are under 18yrs
of age. They are to indicate whether they would like a hard copy of the magazine they are
subscribed to. Mailing address – if they want a hard copy What type of things the subscriber are interested to view in their magazine
(customizing purposes): adult; movies; music; news; sports; other. Subscribers are to choose whether they want the “dating” option.
When they have finished filling in the form, they should be able to submit the form to subscribe for the magazine immediately.
Part B: Scenario
Use the scenario below to CHECK that you have a ‘control’ (e.g. textboxes, radio buttons, dropdown menus) for each item, and that each ‘control’ is of the appropriate type and size.
Login name – “pinkyWinky666” Password – “privateMary asd kjbJerry” Email address – [email protected] Contact number – +64 219237846 Viewing language: Chinese Traditional Payment – “Bank Deposit” Favourite singer – “Spice Girls” Favourite movie – “Titanic” Subscriber’s full name – “Mary Jerry” Age – “18” “Yes, I want a hard copy of the magazine”. Mailing address – “833 Crazy Avenue, Mamalada Land, New Brealand” Interested in: “adult”; “movies”; “other: children” Dating option: “Yes”
Appendix F2. Instructions including the requirements and the scenario for the low formality (on Tablet PC) design.
Instructions:
Use the requirements stated in Part A and scenarios provided in Part B to check the design you have been given. Change the design to provide a better interface to serve its purpose.
Part A: Requirements for this form
Design an interface for the $1 million Loan application (Samson’s Bank – NZ branch). The applicant must know that it is the Samson’s Bank (NZ) page. The online form should be easy to fill in. The information listed below should be obtained from the applicants accurately:
Full Name Passport No. – New Zealand Home Number Mobile Number Address – the town or city must be within the designated areas i.e. Auckland,
Hamilton, Wellington, Rototua, Dunedin and Christchurch Bankruptcy – to check if they were / are bankrupt. Occupation Status – married/single/in a relationship – if married, then what’s the partner’s
weekly income The date they want the loan Applicant’s Weekly income Other personal assets of value – e.g. house, car, stock etc Is the applicant renting a house or on Mortgage Purpose of the loan – whether applicant is suitable IRD number – security reasons Applicants must check whether they have all the items ready: i.e. person
verification, income verification (from inland revenue), past 10 years bank record, 10 official character reference
Applicants may include some questions in the applicationWhen the applicant has finished filling in the form, they should be able to submit the form.
Part B: The Scenario
Use the scenario below to CHECK that you have a ‘control’ (e.g. textboxes, radio buttons, dropdown menus) for each item, and that each ‘control’ is of the appropriate type and size.
Full Name – “Mary Carey” Passport No. – “FZ 1055872” Home Number – “09 3487533” Mobile Number – “027 13049773” Address – “10 Symonds street, Auckland CBD. Occupation – “Personal Assistant” Status – married Partner’s weekly income – “$1000” The date they want the loan – 07/011/2006 Applicant’s Weekly income – “$300” Other personal assets of value – “car” House – “Renting house” Purpose of the loan – “I want to be able to spend all my money because I have
an uncurable disease and I want to live my life happily in the time left. Then my partner will pay them off for me”
IRD number – 123-321-123 Yes, have all the items ready: person verification, income verification (from
inland revenue), past 10 years bank record, 10 official character reference Question: What is the interest rate? Because if it’s too high then I shall find
Samuel for their $2million loan”Submit application – “Next page”
Appendix F3. Instructions including the requirements and the scenario for the medium-low formality design.
Instructions:
Use the requirements stated in Part A and scenarios provided in Part B to check the design you have been given. Change the design to provide a better interface to serve its purpose.
Part A: Requirements for this form
Design an interface for the University of Strawberries (NZ) Graduation application form. The applicant must know that it is the University of Strawberries (NZ) application page. The online form should be easy to fill in. The information listed below should be obtained from the students accurately:
Full Name Preferred Name – to be called at the graduation ceremony Student ID – 10 digits ID Programme Name – e.g. Bachelor of Science, Master of Commerce etc. Department Name – e.g. Computer Science, Health Science, Psychology etc. Preferred graduation year and semester Whether he/she want to graduate in person or in absentee If absent, where they want the certificate to be sent to – address If the address is not in New Zealand, then postage fee must be paid – by
eftpos/credit card/cheque. Contact Number The last day of their course The things they want to borrow for the graduation ceremony: trencher, gown,
black suit, black dress, hood (and what colour hood)?The applicant should be able to proceed to the next page when he/she has finished filling in this page.
Part B: The Scenario
Use the scenario below to CHECK that you have a ‘control’ (e.g. textboxes, radio buttons, dropdown menus) for each item, and that each ‘control’ is of the appropriate type and size.
Full Name – “Cheng Lim Wang” Preferred Name – “Cheng Wang” Student ID – “9994443331” Programme Name – “Bachelor of Science, Bachelor of Arts” Department Name – “Psychology and philosophy.” Preferred graduation year and semester – “year 2007, semester 1” Whether he/she want to graduate in person or in absentee – “In absentee” If absent, where they want the certificate to be sent to – “19/F, XYZ Building,
XYZ road, Kowloon, Hong Kong” Postage payment – “Credit card” Contact Number – “021 1873009” The last day of their course – “June,10, 2006” Want to borrow – trencher, gown, hood (hood colours: pink and blue) Continue - “To Next page”
Appendix F4. Instructions including the requirements and the scenario for the medium-high formality design.
Instructions:
Use the requirements stated in Part A and scenarios provided in Part B to check the design you have been given. Change the design to provide a better interface to serve its purpose.
Part A: Requirements for this form
Design an interface for the New Zealand Dog Registration Online Application form. The applicant must know that it is the New Zealand Dog Registration Page. The online form should be easy to fill in. The information listed below must be obtained:
Dog’s Name Owner’s Name Address Contact Number Second Owner – if there is one Second Owner’s Address Is the Dog sterilized – for statistical and population control purposes Gender Age – dog’s age Appearance of dog – height, weight, colour Is the dog registered with a vet – for health and safety Dog’s special condition(s) – e.g. blind, disabled, violent history etc. Purpose of dog – e.g. guide dogs, police dogs, pet etc. Breed Where did they get the dog? : On the street, from friend(s) and/or family,
SPCA, own dog’s offspring, from a pet shop. – And if it’s from a shop, which shop.
An agreement to love his/her dog.The applicant should be able to proceed to the next page when he/she has finished filling in this page.
Part B: The Scenario
Use the scenario below to CHECK that you have a ‘control’ (e.g. textboxes, radio buttons, dropdown menus) for each item, and that each ‘control’ is of the appropriate type and size.
Dog’s Name – “Dinky” Owner’s Name – “Peter-Andrea Smith” Address – “8 Sunny Bay, Whangarei.” Contact Number – “027 1234567” Second Owner – “Shelly Brown” Second Owner’s Address – “14 Sunny Bay, Whangarei” Is the Dog sterilized – Don’t know Age – 4 years Appearance of dog – height = “55cm”, weight = “30kg”, colour = golden
white Is the dog registered with a vet –“No” Dog’s special condition(s) – None Purpose of dog – pet Breed – Pure Labrador Where did they get the dog? : “From SPCA” An agreement to love his/her dog. – “No”
Appendix F5. Instructions including the requirements and the scenario for the high formality design.
Instructions:
Use the both the requirements stated in Part A and the scenario provided in Part B to check the design you have been given. Change the design to provide a better interface to serve its purpose.
Part A: Requirements for this form
Design an interface for the 2007 America’s Next Top Model secured online application form. The applicant must know that it is the America’s Next Top Model application form. The online form should be easy to fill in, with minimal room for input errors from applicants. The information listed below must be obtained from the applicants:
Full Name Home Address – it must be an American Address Contact Number Email address Status – married, in a relationship, single Date of Birth – to check whether they are between 18 – 28 yrs old. Age – second check for age Height (cm) Weight (kg) Occupation – for fast grouping purposes Modelling experience – to identify those who has more experience Why do they want to enter – to screen for individuals who stands out. Where they hear about this America’s next top model form: Radio, TV,
newspaper, modelling agencies, online and/or other – for statistical and future promotional purposes.
When the applicant has finished filling in the form, they should be able to submit the form.
Part B: The Scenario
Use the scenario below to CHECK that you have a ‘control’ (e.g. textboxes, radio buttons, dropdown menus) for each item, and that each ‘control’ is of the appropriate type and size.
Full Name – “Mary-Jane Louisa Love-Hewitt” Home Address – “156, Mysterious Lane, New York, Washington D.C.” Contact Number – “09982 12345667” Email address – “[email protected]” Status – “in a relationship” Date of Birth – “06/06/1985” Age – “21” Height (cm) – “180” Weight (kg) – “47” Occupation – “student” Modelling experience – “3 years part-time modelling at Elle Magazine, 2
years part-time cat walk experience, model for a new underwear brand.” Why do they want to enter – “I want to become America’s next top model.
Modelling is my dream career, I want it so bad, I’ve been dreaming about it ever since I was like 3 years old. I am very into fashion, and I want to proof to the world that blondes are just as smart and beautiful as brunettes”
Hear about it from – TV, radio, modelling agencies, online, magazines. Submit form
Appendix G. Screen shots during font creation using My Font Tool for Tablet PC (2004).
Figure G1. Font creation using My Font Tool for the Tablet PC (2004): hand-writing input at the designated spaces for each character in the alphabet, and numbers from 0-9, as well as some common symbols such as full-stops, commas, exclamation marks etc.
Figure G2. From the original handwriting illustrated in Figure G1, the data is
compiled and the new font called “LY_Handwriting” is then created, as shown, in different sizes.
Figure G3. An illustration of the Gulim typeface in different sizes in My Font Tool for the Tablet PC
Figure G4. An illustration of the Times New Roman typeface in different sizes in My Font Tool for the Tablet PC.
Appendix H. Testing the normality assumption – Total number of changes made across levels of formality
Statistic Std. ErrorLow formality (on paper) Mean 18.733 1.1996 95% Confidence
Interval for MeanLower Bound 16.280
Upper Bound21.187
5% Trimmed Mean 18.815 Median 19.500 Variance 43.168 Std. Deviation 6.5702 Minimum 5.0 Maximum 30.0 Range 25.0 Interquartile Range 9.5 Skewness -.108 .427 Kurtosis -.680 .833Low formality (on Tablet PC) Mean 15.167 .7567 95% Confidence
Interval for MeanLower Bound 13.619
Upper Bound16.714
5% Trimmed Mean 15.074 Median 15.000 Variance 17.178 Std. Deviation 4.1447 Minimum 7.0 Maximum 26.0 Range 19.0 Interquartile Range 6.0 Skewness .381 .427 Kurtosis .525 .833Medium-low formality Mean 14.000 .7428 95% Confidence
Interval for MeanLower Bound 12.481
Upper Bound15.519
5% Trimmed Mean 13.944 Median 13.500 Variance 16.552 Std. Deviation 4.0684 Minimum 7.0 Maximum 23.0 Range 16.0 Interquartile Range 6.3 Skewness .063 .427 Kurtosis -.560 .833Medium-high formality Mean 13.133 .7042 95% Confidence
Interval for MeanLower Bound 11.693
Upper Bound14.574
5% Trimmed Mean 13.111 Median 13.500 Variance 14.878 Std. Deviation 3.8572 Minimum 7.0 Maximum 20.0 Range 13.0 Interquartile Range 6.3 Skewness -.051 .427 Kurtosis -1.165 .833High formality Mean 11.267 .6414 95% Confidence
Interval for MeanLower Bound 9.955
Upper Bound12.578
5% Trimmed Mean 11.259 Median 11.500 Variance 12.340 Std. Deviation 3.5129 Minimum 6.0 Maximum 17.0 Range 11.0 Interquartile Range 6.3 Skewness -.054 .427 Kurtosis -1.369 .833
Appendix I. Testing the normality assumption – Number of quality changes made across levels of formality
Statistic Std. ErrorLow formality (on paper) Mean 15.733 1.0045 95% Confidence
Interval for MeanLower Bound 13.679
Upper Bound17.788
5% Trimmed Mean 15.824 Median 16.500 Variance 30.271 Std. Deviation 5.5019 Minimum 4.0 Maximum 25.0 Range 21.0 Interquartile Range 9.4 Skewness -.156 .427 Kurtosis -.716 .833Low formality (on Tablet PC) Mean 13.050 .7324 95% Confidence
Interval for MeanLower Bound 11.552
Upper Bound14.548
5% Trimmed Mean 12.963 Median 13.500 Variance 16.092 Std. Deviation 4.0115 Minimum 6.0 Maximum 22.0 Range 16.0 Interquartile Range 4.3 Skewness .176 .427 Kurtosis -.200 .833Medium-low formality Mean 12.900 .6958 95% Confidence
Interval for MeanLower Bound 11.477
Upper Bound14.323
5% Trimmed Mean 12.741 Median 13.000 Variance 14.524 Std. Deviation 3.8111 Minimum 7.0 Maximum 23.0 Range 16.0 Interquartile Range 5.5 Skewness .544 .427 Kurtosis .223 .833Medium-high formality Mean 10.800 .7162 95% Confidence
Interval for MeanLower Bound 9.335
Upper Bound12.265
5% Trimmed Mean 10.722
Median 10.750 Variance 15.390 Std. Deviation 3.9230 Minimum 4.0 Maximum 19.0 Range 15.0 Interquartile Range 7.3 Skewness .153 .427 Kurtosis -.721 .833High formality Mean 9.017 .6462 95% Confidence
Interval for MeanLower Bound 7.695
Upper Bound10.338
5% Trimmed Mean 9.019 Median 9.250 Variance 12.526 Std. Deviation 3.5391 Minimum 3.5 Maximum 14.5 Range 11.0 Interquartile Range 6.0 Skewness -.042 .427 Kurtosis -1.290 .833
Appendix J: Testing the normality assumption – Number of expected changes made across levels of formality
Statistic Std. ErrorLow formality (on paper) Mean 13.550 .7736 95% Confidence
Interval for MeanLower Bound 11.968
Upper Bound15.132
5% Trimmed Mean 13.657 Median 14.750 Variance 17.954 Std. Deviation 4.2373 Minimum 4.0 Maximum 20.5 Range 16.5 Interquartile Range 7.1 Skewness -.400 .427 Kurtosis -.554 .833Low formality (on Tablet PC) Mean 11.183 .5938 95% Confidence
Interval for MeanLower Bound 9.969
Upper Bound12.398
5% Trimmed Mean 11.231 Median 12.000 Variance 10.577 Std. Deviation 3.2523 Minimum 4.5 Maximum 17.0 Range 12.5 Interquartile Range 5.1 Skewness -.225 .427 Kurtosis -.644 .833Medium-low formality Mean 10.217 .6092 95% Confidence
Interval for MeanLower Bound 8.971
Upper Bound11.463
5% Trimmed Mean 10.130 Median 9.500 Variance 11.132 Std. Deviation 3.3365 Minimum 4.5 Maximum 18.0 Range 13.5 Interquartile Range 5.6 Skewness .270 .427 Kurtosis -.707 .833Medium-high formality Mean 9.017 .6299 95% Confidence
Interval for MeanLower Bound 7.728
Upper Bound10.305
5% Trimmed Mean 8.954 Median 9.750 Variance 11.905 Std. Deviation 3.4503 Minimum 3.0 Maximum 17.0 Range 14.0 Interquartile Range 5.4 Skewness .161 .427 Kurtosis -.469 .833High formality Mean 8.000 .6027 95% Confidence
Interval for MeanLower Bound 6.767
Upper Bound9.233
5% Trimmed Mean 8.009 Median 9.000 Variance 10.897 Std. Deviation 3.3010 Minimum 2.0 Maximum 14.5 Range 12.5 Interquartile Range 5.5 Skewness -.243 .427 Kurtosis -.782 .833
Tests of Normality
Kolmogorov-Smirnov(a) Shapiro-Wilk Statistic df Sig. Statistic df Sig.Low formality (on paper) .134 30 .179 .969 30 .513Low formality (on Tablet PC) .132 30 .190 .965 30 .411Medium-low formality .130 30 .200(*) .956 30 .242Medium-high formality .112 30 .200(*) .977 30 .739High formality .152 30 .073 .957 30 .263* This is a lower bound of the true significance.a Lilliefors Significance Correction
2 17.00 1.67 3 14.40 1.82 4 16.20 1.40 5 12.00 1.48a. This level combination of factors is not observed, thus the corresponding population marginal mean is
not estimable.
Note: Formality Level: 1 = Low formality (on paper); 2 = Low formality (on Tablet PC); 3 = Medium-low formality; 4 = Medium-high formality; 5 = High formality.
Appendix M. Mean number of expected changes across each level of formality – according to between-subjects factors (design experience, major/specialization and
study level)
Design experienceMajor/
specialization Study levelFormality
level Mean Std. ErrorNone to non-CS/SE design experience
2 13.90 1.32 3 11.90 1.56 4 12.70 1.35 5 10.50 1.44a. This level combination of factors is not observed, thus the corresponding population marginal mean is
not estimable.
Note: Formality Level: 1 = Low formality (on paper); 2 = Low formality (on Tablet PC); 3 = Medium-low formality; 4 = Medium-high formality; 5 = High formality.
Appendix N. One-way ANOVA and post-hoc multiple comparisons between total, quality, and expected changes made across each level of formality
Table N1. Significant between group differences and linear trends found after conducting one-way ANOVA on total, quality and expected changes.
df F Sig.Low formality (paper) Between Groups (Combined) 2 6.67 .002* Linear Term Contrast 1 13.23 .000**Low formality (on Tablet PC) Between Groups (Combined) 2 8.15 .001* Linear Term Contrast 1 16.28 .000**Medium-low formality Between Groups (Combined) 2 8.08 .001* Linear Term Contrast 1 15.26 .000**Medium-high formality Between Groups (Combined) 2 9.10 .000* Linear Term Contrast 1 18.08 .000**High formality Between Groups (Combined) 2 7.03 .001* Linear Term Contrast 1 13.43 .000**
* The between group difference is significant at the .05 level.** The linear trend (linear contrast across levels of formality) found is significant at the .05 level.
Table N2. Post-hoc multiple comparisons between total, quality, and expected changes made across each level of formality
* The mean difference is significant at the .05 level.
Appendix O: “Extra changes” made in designs.
Appendix O1. “Extra changes” made in the Low Formality Design presented on paper: International Online Magazine Subscription Form
Types of changesT – Q(no. of
changes)
Q-E(no. of
changes)Change of text in label“Do you want a daily/weekly/yearly mag?” text changed to “How often do you want the mag?”
2 2
“Do you want a daily/weekly/yearly mag?” text changed to “I want to receive my mag…”
1 1
“Do you want a daily/weekly/yearly mag?” text changed to “How do you want it to be sent…”
1
“Do you want a daily/weekly/yearly mag?” text changed to “Do you want the magazine daily/weekly/yearly mag?”
1
At “Mailing address” item: label in first line changed to “Street no/Street” 1 1“What type of things…” text changed to “Interest…” 3“What type of things…” text changed to “Magazine content preference…” 1“What type of things…” text changed to “What genres are you interested in viewing in your magazine…?”
1
“Full Name”: text changed to “Subscriber’s Full Name” 2“Age ”: text changed to “Age years old (for statistical and content restriction purposes”
1
“Age”: label changed to “DOB” 3“Login Name” text changed to “Preferred Login Name” 1 1“Login Name” text changed to “Choose a Login Name” 1 1“Password” text changed to “Preferred password” 1 1“Contact no.” text changed to “Preferred Contact no.” 1 1“Contact no.” text changed to “Phone number” 1“Contact no.” text changed to “Number” 1“Email” text changed to “Email address” 2“Do you want the dating option?” “Would you like the dating option?” 1 1“(Please Choose….)” text changed to “(Please tick….)” 1At “Address” item set: labels changed to “Address Line1, Address Line2, Address Line3…
1
“Subscription to International Online Magazine” text changed to “Subscription Form to International Online Magazine”
1
“Payment” text changed to “Payment type” 1 1Relocation of elements/items/item setsAt “Date of loan” item set: items moved from horizontally aligned to vertically aligned
5
At “Payment” item set: items moved from horizontally aligned to vertically aligned
1 1
At “Interest” item set: items listed both horizontally and vertically instead of listing downward:
3 3
At “Dating option”: radio buttons moved from the left to the right of the label 1
“Age” (or “DOB”) item(s): moved below “Address” item set 1“Payment” item set: moved below “Daily/weekly/yearly mag” item set 4 4“Payment” item set: moved to the bottom, below “Viewing Interest” item set, above “Dating option” item.
2
“Payment” item set: moved to the bottom before the “Submit” item 1“Full Name”, “Age” and “Address” items/set below “Ethnicity” item 1“Full Name”, “Age” and “Address” items/set moved to the top as the first few items below the main heading
2
“Full Name”, “Age” and “Address” items/set below “Email” item 1
“Full Name” and “Age” items: moved to the top as the first item 3“Full Name” and “Age” items: moved below “Password” item 1“Full Name” item: moved to the top as the first item 1“Age” item: moved next to “Contact no” item 1“Email” item: moved above “Password” item (i.e. swapped location) 1“Email” item: moved below “Ethnicity” item 1“Email” item: moved to the right of “Contact no.” item 1“Contact” item: moved above “Email” item (i.e. swapped location) 2“Contact” item: moved below “Country” item within the “Address” item set 2“Favourite singer” and “Favourite movie” items: moved above “Viewing Interests” item set
3 3
“Favourite singer” and “Favourite movie” items: moved below “Viewing language” item, above “Daily/weekly/yearly mag” item set
1
“Favourite singer” and “Favourite movie” items: moved to the bottom of everything
1
“Favourite movie” item: moved to the right of “Favourite singer” item: 1“Daily/weekly/yearly mag” item set: moved below “Viewing Interests” item set. 1“Login Name” and “Password” items: moved below “Payment” item set 1“Address” item: moved below “Contact no.” item 1“Daily/weekly/yearly mag” item set: moved above “Payment” item set 1“Daily/weekly/yearly mag” item set: moved below “Country” item within the “Address” item set
1
“Daily/weekly/yearly mag” item set: moved above “Favourite singer” and “Favourite movie” items
1
“Country” item: placed first in the “Address” item set 1“Password” item: placed next to “Login Name” item 1Change of element(s) type in an item/item setAt “Dating option” item: question type changed to“ I would like the dating option”
2 2
1
1
At “Age” item: changed to 1 1At “Email” item: 1
At “Ethnicity” item: 8 8 **At “Payment option” item: items merged into one item with a dropdown menu 5At “Daily/weekly/yearly mag” item set: items merged into one item with a dropdown menu
3
At “Town/City” item within the “Address” item set:
4 4
At “Country” item within in the “Address” item set:
3
At “Viewing Interests” item set: all radio buttons merged into one item 1“Dating option” item set and “Submit” item set merged 1 1Adding an Element/Item / Item setAdded “ Required fields” at the top right corner below the main heading 1 1Added reminder “ Remember to check everything before you submit the form” 1 1Added instruction “Fill in details below” at the top below the main heading 1 1Added headings for each section e.g. “Login info”, “Payment Options”..etc 1Following the “Do you want a hard copy” item: added “If yes, mailing address” or similar at above “Mailing address” item set.
8 8
Added “Confirm Login name: ” below “Login Name” item 1Added “Submit” Question: “Do you want to submit? ”
1 1 *, **
Added “Submit” Question: “Do you want to submit? ”
1 1 *, **
Added “ Submit” item at the end of the form 1 1 *, **
Added “Verify Email ” below “Email” item 1Added group box around “Mailing Address” item set 3 3Added group box around “Login Name” and “Password” items 1 1Added heading “Address” at the “Mailing address” item set 1 1At “Mailing Address ” item within the Address” item set: split into two items: No. Street
1
Added extra line of input in “Address” item set: “Suburb ”
2 2
Added “ Credit card no. ” to the “Credit card” item within the “Payment” item set
7 7
Added “Account no. ” to the “Bank deposit” item within the “Payment” item set
1
Added “Postcode ” below “Address” item set 2 2Added “Others ” to the right of “Favourite Singer” item 5 5 *, **Added “Others ” to the right of “Favourite Movie” item 5 5 *, **Added “Other ” within the “Viewing Interests” item set 1 1 *, **Added “Date of Birth ” to the right of “Age” item 1Added “Gender ” at the top of the design below the main heading 1Added instructions “Password must be xxx characters min and xxx characters max containing xxx characters” etc at the “Password” item
2 2
Added “ Others” at the “Viewing interests” item set 2 2 *, **Added “Start date ” after “Daily/weekly/yearly mag” item set 1Added “Start date ” after “Daily/weekly/yearly mag” item set 1
Added “Are you currently a mail subscriber? ” after “Daily/weekly/yearly mag” item set
1
Added “Check availabilty” button at “Login Name” item 2 2Deleting an Element/ Item / Item setDeleted “Age ” item 1 At “Daily/weekly/yearly mag” item: textboxes combined to become one dropdown menu
1
Deleted “Full Name ”item 1Deleted “Favourite singer ” item 1
Deleted “Favourite movie ” item 1Deleted “ ” at “Weather” item within the “Viewing Interests” item set 3 3Deleted “Payment” item set 1Deleted “Dating Option” item set 1Resizing an Element(s)At “Full Name” item: longer textbox 5 5 *, **
At “Age” item: smaller textbox 2 2 *, **Note: T – Q = Total changes – Quality changes made; Q – E = Quality changes – Expected changes made
* scored 0.5 out of 1 the particular quality change due to some incorrectness according to the criteria for quality changes
** scored 0.5 out of 1 for the particular expected change due to some incorrectness according to the criteria for expected changes.
Appendix O2. “Extra changes” made in the Low Formality Design on tablet PC: Samson’s Bank $1 million Loan Application Form
Types of changesT – Q(no. of
changes)
Q-E(no. of
change)Change of text in label“Address: House/Street” text changed to “Address: Street no./Street” 1 1“Address: House/Street” text changed to “Address” 2“When do you want your loan?” text changed to “Loan date” 1 1“Any Questions?” text changed to “Any Questions? Please specify” 1 1“Any Questions?” text changed to “If you have any questions, please state”
1 1
“Any Questions?” text changed to “Questions?” 3At the “Check “ item set: “Income verification” text changed to “Income verification from Inland Revenue”
3 3
At “Submit” item set: “Yes” and “Next page” text changed to “Yes, next page” and “No, go back”
1 1 *, **
“Weekly income range” text changed to “Weekly income” 2“Weekly income range” text changed to “Your Weekly income” (with a )
1 1
“Weekly income range” text changed to “Your Weekly income range” 2 2“Status” “Marital Status” 5 5“Person verification” text changed to “Proof of ID” 1 1 *“Income verification” text changed to “Proof of income” 1 1 *“Past (10 years) bank records” text changed to “Past (10 years) bank statements”
1 1 *
“Home no.” text changed to “Home phone no.” 3 3“Reasons for loan” text changed to “Purpose of your loan: Please describe”
1 1
“Reasons for loan” text changed to “Reasons of your loan: Please describe”
1 1
“Other Personal Assets” text changed to “Personal Assets of Value” 1 1“Other Personal Assets” text changed to “Other Personal Assets of Value” 1 1“When do you want your loan” text changed to “Date of loan” 1“IRD number” text changed to “IRD number (for security reasons)” 1“Home no.” and “Mobile” text changed to “Contact (home)” and “Contact (mobile)”
1
“Mobile” text changed to “Mobile no.” 1 1“Personal Asset” item: text changed to “Personal asset worth in total” i.e. changed meaning
1
Relocation of elements/items/item set“Income range” item: moved above “Date of Loan” item set, below “Status” item
1
“IRD” item: moved below “Date of Loan” item, before “Personal Assets” item
1 1
“IRD” item: moved below “Contact numbers” items 1“IRD” item: moved below “Status” item, above “Date of loan” item 1“IRD” item: moved next to “Personal Assets” item 1“Passport no.” item: moved below “IRD” number items, above “Address” item set
1
“Passport no” item: moved below “Address” item set 1“Weekly income range” item: moved above below “Occupation” item, above “Status” item
2 2
“Address” item set, moved below “Full Name” item 2 2“Mobile no.” item: moved next to “Home no.” item horizontally 2“Mobile no.” item: moved above “Home no.” item (i.e. swapped location) 1“Date of loan” item set: moved above “Check” item set 1“Date of loan” item set: moved below “Personal Assets” item, above “Reasons for Loan” item
1 1
At “Date of loan” item set: labels below elements moved to the right hand side of the elements: dd mm yyyy
1
“Address” item set: moved to the right of “Full Name” item 1“Status” item: moved to the right of “Occupation” item 1At “Date of loan” item set: items moved from horizontally aligned vertically aligned
1
Change of element(s) type in an item/item setAt “Occupation” item: 8At “Yearly income range” item: 10 10 **At “Yearly income range” item: multiple 2 2 **At “Reasons for loan” item: 6, 12, 22At “Status” item: multiple radio buttons 6 6At “Personal Assets” item: 1 1 **At “Any Questions” item: 1 1 *Adding an Element/Item / Item setAt “Loan date” item set: added slashes / / 1 1Added “Suburb ” below “Address (House/ Street) item within the “Address” item set
3 3
Added “Renting: Yes ” at the “income range” items 1 1 **Added “$” at “Weekly income range” item:
1 1
Added “ Expiry date: ” next to “Passport Number” item 1At “Address(House/Street) ” item within the “Address” item set: split into two items: No. Street
1
Added “Email ” item at “Contact number” items 2 2Added instruction “Please fill in details below. ** are compulsory fields” at the top below the main heading
1 1
Added “Address” heading to the “Address” item set 1 1Added “Personal details” heading above the “Full name” item 1 1Added “Other ” at “Reasons for loan” item to the right of the dropdown menu (changed earlier)
3 3*, **
Added “Other ” at “Personal assets” item 2 2Added “ ” to the right of “Any Question ” item
Added “Workplace no ” 1Added below “Bankruptcy” item set:“Reason(s) for bankruptcy, please describe: “
1 1
Added instruction “Please provide hard copy of bank statements” at “Check” item set
1 1
Added numbers to each question 1 1Added instruction “If married, fill in xxxx” at the “Income” items 1 1Added “Reset” and “Clear” buttons in addition to the “Submit” item 1Added “Section lines” to separate sections in the form 1 1Added “If others, specify please: ” at “Status” item 1Deleting an Element/ Item / Item set 1Deleted “ Do not agree ” in “Bankruptcy” item set. 1 1 *Deleted “Any Questions” item 1Deleted “Date of loan” item set (but maybe due to redoing the design from scratch)
1
Delete “Loan Agreement contract” item at “Check” item 1Added “Status” item below “Address” item set, in addition to the an existing item further down
1
Deleted “ ” and “ ” at “Submit” item set (left with “Submit Application” label)
1 1 *, **
Resizing an Element(s)Resized “Day” and “Month” items only within the “Date of Loan” item set.
Other changes made (not counted as functional changes)Aligned all on left (indicated) and indention 1 1Note: T – Q = Total changes – Quality changes made; Q – E = Quality changes – Expected changes made
* scored 0.5 out of 1 the particular quality change due to some incorrectness according to the criteria for quality changes
** scored 0.5 out of 1 for the particular expected change due to some incorrectness according to the criteria for expected changes
Appendix O3. “Extra changes” made in the Medium-Low Formality Design: University of Strawberries Graduation Form
Types of changesT – Q(no. of
changes)
Q-E(no. of
changes)Change of text in label“Mailing address: House/Street” text changed to “Mailing address: No./Street”
1 1
“Student ID” text changed to “University of Strawberry Student ID” 1“Contact No” text changed to “Contact No. (NZ)” 1 1At “Continue to next page” item set: “Home” and “Next page” text changed to “Yes, next page” and “No”
4 4*, **
At “Last day of your course” item set: “dd”, “mm”, and “yyyy” text changed to “Day”, “Month”, and “Year”
1 1*
At “Last day of your course” item set: “dd”, “mm”, and “yyyy” text changed to “dd”, “month”, and “yyyy”
1 1
“Degree Name” text changed to “Program/Degree Name” 1 1“Degree Name” text changed to “Degree” 1 1At “Borrow items” item set: “You want to borrow: ” text changed to “You want to borrow for the graduation ceremony: ”
1 1
“Preferred Name” text changed to “Preferred name (for the graduation ceremony)
1 1
“Preferred Name” text changed to “Preferred name (called at the graduation ceremony)”
1 1
“Graduation Year/Semester” text changed to “Preferred Graduation Year/Semester”
1 1
“Graduation Year/Semester” text changed to “Preferred Graduation time” 1 1*“Contact No.” text changed to “Number you preferred to be contacted” 1“Degree Name” text changed to “Program name” 1 1At the main heading: “….graduation…..” text changed to “…….Graduation…..”
1 1
Relocation of elements/items/item set“Gender” item moved below “Age” and “Appearance” 1Moving “Graduating in” item set above “Mailing Address” item set (or vice versa i.e. moving “Mailing Address” below Graduating in” item set
14 14
“Degree name” and “Department” items swapped location 2“Graduation year/semester” item: moved below “Graduating in” item set 1“Graduation year/semester” item: moved below “Ethnicity” item and above “Last day of your course” item set
1
“Contact No.” and “Ethnicity” items moved away so that graduation items next to above and below each other
2 2
“Contact No.” item: moved below “Mailing Address” item set. 2 2“Contact No” item: moved below “Last day of your course” item set. 2“Last day of your course” item set: moved below “Department” item and above “Graduation year/semester” items
1
At “Last day of your course” item set: labels below elements moved to the top of the elements dd mm yyyy dd mm yyyy
1
“Mailing Address” item set: moved up to “Graduating in” item set 2 2“Hood colour” item moved to the right of “Hood” item within the “Borrow items”
3 3
“Preferred Name” item: moved to the right of “Full Name” item 1“Degree Name” item: moved to the right of “Student ID” item 1“Graduation Year/Semester” item: moved to the right of “Department” item 1“Ethnicity” item: moved to the right of “Contact No.” item 1Change of element(s) typeAt “Hood colour” item within the “Things to borrow” item set:
1 1*
At “Ethnicity” item: 1 1**
At “Speech” item set: combined items changed to“I want to give a speech: ”
1 1**
At “Speech” item set: changed to (radio button) 12 12**At “Speech” item set: changed (tick box) 2 2At “Country” item within the “Mailing Address” item set: changed to
1
At “Town/City” item within the “Mailing Address” item set:
2 2
At “Ethnicity” item: multiple radio buttons 1 1**At “Country” item within the “Mailing Address” item set:
1
Adding an Element/Item / Item setAdded heading “Degree Name” 1 1Added “Other, please specify: ” to the right of “Ethnicity” item. 2 2**Added “If absent” to the left to the “Mailing address” label 15 15Added group box around “Address” item set 1 1Added “Hood Colour” item “auto-filled” 1 1Added “Pronunciation of preferred name ” below “Preferred Name” item
1 1
Added “Account Number” item in addition to the “Postage Payment” item required
2 2
At “Country” item within the “Mailing address” item set: split into: NZ
1 1
Added “Suburb ” at “Mailing Address” item set 1 1Added another “Address” item set in addition to the “Mailing Address” item. 1Added control properties i.e. “Speech” item set kept ‘hidden’ if the item before (“Graduating in” item) is not chosen
1 1
Deleting an Element/ Item / Item setAt “Speech” item set: deleted “ No, I don’t want to give a speech” item
1 1*, **
At “Borrow items” item set: deleted “Hood colour” item 1Deleted “Graduating in” item set (indicated no need but not counted) 1Indicated no need to have “Borrow items” item 1Resizing an Element(s)“Student ID” item: smaller textbox or 8 8“Last day of your course” item: smaller textboxes 1 1Larger main heading 1 1At “Full Name” item: longer textbox 1 1*, **
At “Preferred Name” item: shorter (smaller) textbox 1
Other changes made (not counted as functional changes)Alignment of elements, especially in the first half of the design (not counted as change)
6 6
Note: T – Q = Total changes – Quality changes made; Q – E = Quality changes – Expected changes made
* scored 0.5 out of 1 the particular quality change due to some incorrectness according to the criteria for quality changes
** scored 0.5 out of 1 for the particular expected change due to some incorrectness according to the criteria for expected changes
Appendix O4. “Extra changes” made in the Medium-High Formality Design: Dog Registration Online Form
Types of changesT – Q(no. of
changes)
Q-E(no. of
changes)Change of text in labelAt “Appearance” item set: “Height” and “Weight” text changed to “Height (cm)” and “Weight (kg)”
13 13
“I love my dog with all my heart” text changed to “Love the dog?” 1At “Second owners address” item set : “Address” changed to “Second owners address”
4 4
“Address” text changed to “Address: no./street” 1 1“Address” text changed to “Address: Street” 1 1 “Dog’s name” text changed to “Name of the dog” 1Relocation of elements/items/item setAt “Appearance” item set: items moved from horizontally aligned vertically aligned
3 3
At “Appearance” item set: labels below elements moved to the left of the elements: “Height Weight Colour ”
1
At “Agree to Love dog” item: changed sides of radio buttons from to
1
“Dog name” item moved down to dog information area above “Age” and “Appearance”
2 2
“Gender” item moved below “Age” and “Appearance” 1“Breed” item moved below “Age” and above “Appearance” 1 1“The Shop is” item within “Where did you get your dog” item set moved next to “Brought from pet shop” item
3
Second owners item sets moved to the right side to First owner’s item sets
2
“Age” item moved to the right of “Gender” item 1“Register with vet” item moved to the right of “Appearance” item set 1“Register with Dog lovers’ society” moved below “Register with vet” item
1
“Purpose of dog” item moved next to “Dog’s special conditions” item
1
At “I Agree to love dog” item set: “ Yes No ” items moved to the right of “I agree to love my dog…” label
1
Change of element(s) typeAt “Age” item: changed to a 10At “Registered with Dog Lovers society” item: 11 11 *, **
At “Registered with Vet” item: changed to 3 3 *, **At “Gender” item: changed to 7At “Gender” item: changed to “Male Yes” 1At “Where did you get dog” item set: all radio buttons changed to tick boxes
5
At “Where did you get your dog” item set: list of items changed to a single line (dropdown menu)
1
“I agree to love dog” item set: changed to “ I agree to love the dog with all my heart ”
1 1
At “Town/city” item within the “Address” item set: changed to
1 1
At “Purpose of dog” item: textbox changed to text area 1At “Sterilized” item: radio buttons changed to 1 “Age” item: changed to “ Date of Birth ” 1Adding an Element/Item / Item set
Added instruction “ required fields” at the top below the heading 1 1Adding “if there’s a 2nd owner” text (or equivilent and/or with a tick box or radio button) above “Second owner” item set
6 6
Added “ If other, please specify: ” next to the “And the Shop is: ” item
1
Added “ Other, specify: ” after the options available at “Where did you get the dog?” item
5 5
At “the Shop is ” item within the “Where did you get dog” item set: “Shop is: ” split into two items: “Name: Area: ”
1 1
Added group boxes to separate first and second owner 3 3Added “Appearance: ”
Separated from “Height”, “Weight” and “Colour” items
1
Added “Other: ” at the “Appearance” item set 1Added “Owners Details” heading at the top below the main heading of “Dog registration form”
3 3
Added “Suburb: ” item to “Addresss” item set 1 1Added “Postcode: ” item to “Address” item set 1 1 “Age” item split into: “Month: Years ” 1At “Dog’s special conditions” item: added extra dropdown menus 1Added “Are you moving out soon? Yes No ” and “If yes, specify ” below “Address” item set
1
Added “If registered with vet, vet name: ” at “Registration with vet” item
1
Added “Proceed to next page: Yes No ” at the end of the page 1 1 *, **Added “Up-Load a picture of dog” item set 1Added numbers to each items e.g. 1, 2, 3a, 3b, 4, 5…etc 2 2Added “ If others, please specify ” next to “Purpose of dog ” item
1 1
Added a heading at each section (in addition to “Dog’s information” and “Owner’s information” headings )
1 1
Deleting an Element/ Item / Item setDeleted “Appearance” item set 1Deleted “Second Owner’s Address” item set 1At “Sterilized” item set: deleted “ ” item 1At “I agree to love dog” item set: deleted “ ” item 1 1At “Where did you get the dog” item set: deleted “And the shop is” item 1Resizing an Element(s)“Address” item longer textbox 1 1Other changes made (not counted as functional changes)Aligned everything (but not counted) 3 3
Note: T – Q = Total changes – Quality changes made; Q – E = Quality changes – Expected changes made
* scored 0.5 out of 1 the particular quality change due to some incorrectness according to the criteria for quality changes
** scored 0.5 out of 1 for the particular expected change due to some incorrectness according to the criteria for expected changes
Appendix O5. “Extra changes” made in the High Formality Design: 2007 America’s Next Top Model Online Application Form
Types of changesT – Q(no. of
changes)
Q-E(no. of
changes)Change of text in label“Status” text changed to “Title” 3At “Submit” item: changed text from “I agree” and “I do not agree”; to a variation of “Yes, submit” and “No”
5 5 **
“Why Enter” text changed to “Reasons for entry” 1“Status” text changed to “Marital Status” 3 3 “Where did you hear this?” changed to “Where did you hear about this competition?”
1
“Date of Birth” text changed to “DOB” 1“Address” text changed to “Home address” 1“Address: Street” text changed to“Address” 1“Email” text changed to “Email address” 1Relocation of items/item set“Address” item set moved below email & contact number 1“Email” item moved above contact number (swapped places) 1“Date of birth” item moved above “Gender” & “Age” items 1Items(set): “Status”, “gender”, “age” and “date of birth” moved below the “name” item
1
“Weight item” moved below “Height” item 1“Why enter” item moved below “Occupation” and “Modeling experience” items
9 9
“Why enter” item moved below “Occupation” item (i.e. swapped locations)
1
“Date of birth” item moved above “Age” item 3“Status” item below “Age” item 1At “Date of birth” item set: items moved from horizontally aligned vertically aligned
1
Change of element typeAt “Age” item: the dropdown menu changed to textbox 1At “Gender” item: dropdown menu changed to radio buttons associated with “male” and female”
5 5 **
At “Status” item: radio buttons changed to a textbox 1At “Email” item: the textbox changed to: 1
At “Submit” item: radio buttons associated to “I agree” and “I do not agree” changed to buttons
2 2 *, **
At “Why enter” item: textbox changed to a textbox 6 6 *, **At “Experience” item: dropdown menu changed to a textbox 3 3 *, **Adding an Item / Item setAdded “Others ” item to the right of “Occupation” item 1 1Added “Others, please specify ” item to the right of “Occupation ” item.
1 1
Added a group box around “Address” item set 1 1Added a group box around “Name”, “Address”, “Contact number” and “Email” items(sets)
Added “Title: ” item at “Status” item 1At “Street” item within the “Address” item set: “Street split into two items:
1
Added: “ I agree with the terms and conditions” item set at the end (including a “terms and conditions link”)
1 1
Added: “Conditions link” item at the “Submit” item 1 1Added “Country ” in “Address” item set 1
Added one more item within the “Address” item: “Address 2: ”
1
Added text “(must be in America)” at “Address” item set. 1Added “Personal detail” heading at the top below the main heading “America’s next top model”
1 1
Added instruction “ important details to fill in” at the top right hand corner
1 1
Added “ ” to the right of “Modeling experience” item
1
At “Modeling experience” item: added next to the element
1
Added “Years of related experience” item to the right of “Modeling Experience” item
1 1
Added “Other ” to the right of “Status” item 1Added “Please confirm age ” item at “Age” item 1At “Modeling Experience” item: added“Describe ” (text area) next to it
1 1 *
Added “Or other reasons: ” next to “Why enter” item 1Deleting an element/ Item / Item setAt “Other reason” item within the “Heard from” item set: deleted the radio button / check box on the left associated to the item
2
At “Other reason” item within the “Heard from” item set: deleted the label “Other reason” left with “ ” only
1
At “Submit” item set: deleted the item “ I do not agree” 1Deleted “Age” item 1Resizing an Element(s) in an item / item set“Date of birth” item set smaller textboxes 1 1“Gender” item a smaller dropdown menu 1“Town/City” item within the “Address” item set smaller textbox 2“Email” item longer textbox 1“Contact No.” item smaller textbox 2“Full name” item longer textbox 2 2*, **Larger heading “2007 America’s Next Top Model” 2 2Other changes madeAligned everything (counted as a functional change – easier to follow) to the left
2
Note: T – Q = Total changes – Quality changes made; Q – E = Quality changes – Expected changes made.* scored 0.5 out of 1 the particular quality change due to some incorrectness according to the
criteria for quality changes** scored 0.5 out of 1 for the particular expected change due to some incorrectness according to
the criteria for expected changes
Appendix P. “Overall Enjoyment” rankings of the five designs across each level of formality
Table Z1. Percentage and frequency of rankings of designs with different levels of formalityLevel of formailty Ranking from 1 – 5 Frequency PercentLow formality (on paper) 1 - Most liked 5 16.7 2 - 3 10.0 3 - 3 10.0 4 - 14 46.7 5 - Least liked 5 16.7Low formality (on Tablet PC) 1 - Most liked 3 10.0 2 - 1 3.3 3 - 3 10.0 4 - 3 10.0 5 - Least liked 20 66.7Medium-low formality 1 - Most liked 3 10.0 2 - 4 13.3 3 - 12 40.0 4 - 7 23.3 5 - Least liked 4 13.3Medium-high formality 1 - Most liked 3 10.0 2 - 13 43.3 3 - 9 30.0 4 - 5 16.7
Table Z2. Individual subjects’ rankings of the five designs in the order from 1 (“most-liked design”) to 5 (“least-liked design”). Underlying reasons for the rankings (indicated by the subjects) are also shown: (A) = Aesthetics; (B) = Perceived effort required; and (C) Fun/stimulating level.
Appendix R. Number of changes made in the five designs across levels of formality by subjects whose “Overall Enjoyment” ranks was dependent on
perceived effort required
Table R1. Total number of changes made across levels of formality by subjects who ranked according to percieved effort required to improve designs (n = 11).
Table R2. Number of quality changes made across levels of formality, by subjects who ranked according to percieved effort required to improve designs (n = 11).
Table R3. Number of expected changes made across levels of formality, by subjects who ranked according to percieved effort required to improve designs (n = 11).
Appendix S. Number of changes made in the five designs across levels of formality by subjects whose “Overall Enjoyment” ranks was dependent on
the level of fun/stimulation when working on the designs.
Table S1. Total number of changes made across levels of formality by subjects who ranked according to the level of fun/stimulation when working on the designs (n = 7).
Table S2. Number of quality changes made across levels of formality by subjects who ranked according to the level of fun/stimulation when working on the designs (n = 7).
Table S3. Number of expected changes made across levels of formality by subjects who ranked according to the level of fun/stimulation when working on the designs (n = 7).
Appendix T. Subjects reasons for design tool preference during the experiment
Table T1. Reasons by participants who preferred using the tablet (Inkit)CS/SE major
CS/SE design experience Reasons for preference
* * can make changes quickly and easily without having to erase things manually: 1) just select and drag; 2) easy to use because easy to make changes, can move things around, resize, erase etc
* * easier to make changes but need more practice on it* easy to use, simple interface* * quicker changes and can move things around, resize.
can move things, fun, tidier* like computers & faster changes
because its easier to show where you want to move things to, compared to paper – needs arrows & crossing out etc
* * fast changes, and move things around and transform thingseasier to change
* * Liked working on the tablet (but no particular preference if have to choose between 2, I don’t mind)
* * easier to make changes but need more practice on it* it was fun, and easy to make changes, rather than crossing out* faster, easier, click instead of writing* was good but with more practice - would prefer a lot more than paper
Table T2 Reasons by participants who preferred using the tablet (Inkit)CS/SE major
CS/SE design experience Reasons for preference
* * liked paper more coz tablet = cumbersome and annoying to click on select/draw/erase buttons
* * Not really like working with tablet (but I suspect it's lack of familiarity) so preferred paper
* * easier to make changes, faster, just crossing out stuff, and done.* * Preferred/liked paper more than tablet coz easier to draw and write on
paper, right in front of you, no scrolling etceasier but maybe computer is good to make things look good, and move things aroundNo preference after getting used to the pen on tablet, but would prefer paper before
* Liked working on paper better – not used to pen as my handwriting is terrible to follownice to change from computer to paper but preferred paper so it's in front of you - easier visualizationeasier to make changes using a pen on paper, tablet = hard to do selecting, erase, draw etc
* * faster coz can draw faster, and crossing out, mental iteration etc* because Tablet's interface was bad, and the changing of modes was
annoying (draw, erase, select), doesn't have the freedom like paper* because on tablet, had to change modes between draw, select, erase,
tiring, and annoying. If short cut, then better* easier with pen and paper
Table T3 Reasons by participants who had no particular preference in design tool CS/SE major
CS/SE design experience Reasons for preference
* * For Dog and Model - wouldn't mind which to design with - good to use tablet so can move things with (reordering) but with designs that required lots of changes[Online Magazine and Loan Application], I would prefer to use pen and paper
* * Paper = more accurate, can draw things better compared to the tablet. But with Tablet - it's easier to change, delete and modify.
Appendix U. Subjects reasons for design tool preference in real life design situations
Table U1. Reasons by participants who had no preference(s)
CS/SE majorCS/SE design Experience Reasons
^ Depends: weather design is complex or simple
Table U2. Reasons by participants who preferred using paper and pen
CS/SE majorCS/SE design Experience Reasons
Paper: more control, faster at drawing and writingProbably paper
^ * Paper - faster, pen and paper infront of you, more freedom
Table U3. Reasons by participants who preferred using tablet (Inkit)
CS/SE majorCS/SE design Experience Reasons
^ * Tablet, definitely coz got the parts from both worlds - comp and paper^ * tablet everytime - easier to work with + easier to erase mistakes
completely cf paper - even when rubbed out, deisgn mistakes are often sill visable and can be distracting
^ tablet, coz easier to edit stuff
Table U4. Reasons by participants who preferred using computer (other tools)
CS/SE majorCS/SE design Experience Reasons
^ * PC programs but tablet if it's easier to use - eg not to move too much with hand
^ * On PC: .Net - just need to drag and drop, tablet is too clumpsy, draw, select, erase, etc)
^ * PC: other programs such as fireworks, photoshop etc, - easy editing and not tablet coz not user friendly, hands moving too much.
^ * ComputersComputer - software, but easier than inkit (depend on software). Don't use paper.
^ * Computer more than paper and tablet* Will Computer (something like VB.net, drag and drop) wouldn't use
paper first, straight to comp. * Compter - can do direct changes on the computer.
Table U5. Reasons by participants who preferred using pen and paper, then tablet
CS/SE majorCS/SE design Experience Reasons
^ * If working on designs like dog or antm, then would use tablet. If beginning of a design, then would use pen-paper. Paper if still draft like, tablet if near finished.
Table U6. Reasons by participants who preferred using paper and pen, then continue on PC
CS/SE majorCS/SE design Experience Reasons
paper - faster and easier with paper and pen, then to PC^ * paper over tablet at the moment, coz like using paper and pen/ use PC.
Would choose to use mouse and keyboard coz not used to tablet's pen-inputpaper - coz technology (inkit) is hard to use, not user friendly, paper, then put it on computer
^ * Computer but not necessary a tablet - because not used to tablet, and have different modes: select, erase, draw. Would use paper first then computer.
^ Paper first then computer^ Paper first, then computer. And probably not Tablet, coz hard to draw
on…unless user friendly programPaper first, then computer. Need to get used to tablet if going to usePorbably paper to do a draft, then put onto computerDraw on paper first to sketch out idea first then computer
^ * Would use paper if start on scratch, if got everything there, would use computer - would be good if can just transform into nice looking after you've drawn it
^ * Draft on paper first (from rough to finalized) then use the final version from paper and transfer to computer and then some more finalizing on computer
^ Paper and pen, then transfer to computer^ Paper first then computer^ Paper first for rough copy because so many modifications are made,