Page 1
April 2009 Journal of Engineering Education 145
Assessing General Creativity and CreativeEngineering Design in First YearEngineering Students
CHRISTINE CHARYTON
Department of PsychologyOhio State University
JOHN A. MERRILL
College of EngineeringOhio State University
ABSTRACT
Creativity is a vital tool for innovation in engineering. Psychologyand engineering faculty developed the Creative EngineeringDesign Assessment (CEDA) because existing tools are limited.This measure was administered with general creativity measuresin 63 engineering (57 males, six females) and 21 non-engineering(six males, 15 females) students in five week intervals. Inter-raterreliability showed high consistency overall and between the testand retest administrations. Only engineering males and femalessignificantly differed on the retest. Engineering students withlow, medium, and high creative engineering design did not statis-tically differ in their general creativity, not domain specific toengineering; however, only high scorers were significantly higheron the retest from the other groups. Future research is neededwith larger samples.
Keywords: creativity, engineering design, innovation
I. INTRODUCTION
Creativity research in engineering began to blossom in the
1950s (Ferguson, 1992). The recommendations of Vannevar Bush,
an electrical engineer from MIT, led to the establishment of the
National Science Foundation in 1950. In the early 1960s, the
National Science Foundation sponsored conferences on “scientific
creativity.” Yet, “as interest in engineering design faded in most
engineering schools, creativity was put on a back burner” (Ferguson,
1992, p. 57). Many engineering education programs appreciate
and value creativity, but few offer courses that teach about creativi-
ty. Creativity is defined as a preference for thinking in novel ways
and the ability to produce work that is novel and appropriate
(Sternberg, 1999; Weisberg, 1986) (see also Charyton, 2005). This
type of creativity can also be defined as “general creativity” that is
not domain specific (Charyton, 2005; Charyton and Snelbecker,
2007). Torrance (1974) stated that creativity has been formulated
in terms of a product (invention and discovery), process, a type of
person, and the production of something new to an individual or
culture. Creativity has been explored in relation to process, product,
personality, and press or environment (Sternberg, 1999; Sternberg
and Dess, 2001). Central themes specific to engineering creativity
include novelty and usefulness (Larson, Thomas, and Leviness,
1999; Nickerson, 1999; Thompson and Lordan, 1999). Engineer-
ing is an applied science where diagnostic procedures and problem-
solutions are derived (Schon, 1983). Innovation is a process to
place new ideas into practice where creativity acts as a vital tool
(Thompson and Lordan, 1999).
This study builds upon the current creativity research in psy-
chology and engineering education. The research project goals are
to provide a useful tool that can effectively assess creative engineer-
ing design at the university level in engineering education.
A. Practicality of CreativityMore recently, creativity has received greater attention as a
necessity, rather than an accessory in engineering design. “Creativity
is important to society, but it traditionally has been one of psychol-
ogy’s orphans” (Sternberg, 1999, p. 4). Even within psychology,
creativity has often been neglected. Czikszentmihalyi (1999) sug-
gested that the person, domain, and field are relevant to under-
standing creativity and innovation. Problem posing has been em-
phasized by the minds of many disciplines in art and science
(Smilansky and Halberstadt, 1986). Highly creative people rede-
fine problems, analyze ideas, persuade others and make reasonable
risks to help generate ideas (Sternberg and Dess, 2001). “Creativity
is certainly among the most important and pervasive of all human
activities. Homes and offices are filled with furniture, appliances,
and other conveniences that are products of human inventiveness”
(Simonton, 2000, p. 151). Engineering is a creative profession that
may be misunderstood. The ability to measure creativity would not
only facilitate the identification of talented individuals, but also
would allow the measurement of baseline information necessary to
track the progress of educational and training programs aimed at
enhancing creativity (Treffinger, 2003). “The need for people
skilled in helping others use creative problem solving is increasing”
(Isaksen, 1983, p. 18). This need is evident in both engineering and
the practice of psychology.
B. Engineering Creativity“Creativity, problem solving, and innovation are of increasing
concern to organizations in these times of accelerating change”
(Basadur and Finkbeiner, 1985, p. 37). The need for creativity,
problem solving, and innovation is becoming a global need. A
growing interest in the need and utilization of creativity in engi-
neering design is evident. Shirley interviewed by Elliott stated,
Page 2
“I think if engineers are not creative, they’re not engineers” (Elliott,
2001, p. 22). How can an engineer be effective without creativity?
Designers have been engaged for centuries in engineering design;
however, only in the past few decades has there been a systematic
process capable of comprehensive analysis and improvement
(Soibelman and Pena-Mora, 2000). The need to assess and en-
hance creativity in engineering design is evident in many university
programs. There is a crucial need to teach about “real-world” engi-
neering design and operations that call for critical judgment and
creativity (Felder et al., 2000, p. 26). Creativity education is critical
in engineering education as well as general education (Ishii and
Miwa, 2005). Psychology is helpful to address creativity in educa-
tion by promoting learning through meta-cognition and self reflec-
tive activities (Ishii and Miwa, 2005). Students can gain confidence
to exercise reflection-in-action with a supportive learning environ-
ment that does not hinder their creativity (Green and Kennedy,
2001). Stimulating activities can encourage creativity and innova-
tion. Through incorporating knowledge from psychology into
engineering education, students can experience creative activities,
reflection, and their own awareness of their cognitive processes
(Ishii et al., 2006).
In the past, educational psychologists have assisted engineering
education faculty with enhancing learning for engineering students
(Felder, 1998). Empirical studies in educational and cognitive psy-
chology literature address methods for learning. These methods
have been implemented successively in engineering classes. Real-
world applications, cooperative learning, active learning, deductive,
and inductive learning are important for developing creativity.
Reflection-in-action is learning by doing (Schon, 1983). Students
also need to practice skills before they are assessed. Furthermore,
experiential learning provides students with opportunities to select
assignments and promotes deeper learning.
The goals and objectives in engineering education need to be
defined, clear and measurable (Felder and Brent, 2003). Creative
engineers are needed to solve technological problems. “It would
seem to be our responsibility to produce some creative engineers—
or at least not to extinguish the creative spark in our students”
(Felder, 1987, p. 222). Is it possible that engineering education de-
creases creativity and originality? To develop and nurture critical
and creative problem solving skills, we must provide opportunities
for students to exercise these skills. Open-ended questions, problem
finding, fluency (quantity of solutions), flexibility (variety of solu-
tions), and originality (novelty) are vital components toward
enhancing analysis and synthesis of information learned (Felder,
1987; Isaksen and Parnes, 1985). Shah, Smith, and Vargas-
Hernandez (2003) also describe these three constructs (fluency,
flexibility, and originality) under different terms such as quantity
(total number of ideas generated) variety (areas of the solution
space), and quality (feasibility of an idea to meet the design specifi-
cations). According to Shah et al., novelty is an approach to mea-
sure effectiveness that relates to quality.
Central themes specific to engineering creativity include origi-
nality (novelty) (Shah, Smith, and Vargas-Hernandez, 2003;
Thompson and Lordan, 1999; Weisberg, 1999) and usefulness
(applicability) (Larson, Thomas, and Leviness, 1999; Shah, Smith,
and Vargas-Hernandez, 2003; Thompson and Lordan, 1999).
Engineers not only need to address aesthetics like artists, but also
need to solve problems, prevent potential problems, and address
utility within the constraints and parameters that are designated.
Furthermore, creativity as an aspect of engineering can be referred
to as “functional creativity” (Cropley and Cropley, 2005). Functional
creativity means that products designed by engineers typically serve
a functional and useful purpose, unlike fine art. Creative products
emphasize novelty, resolution, elaboration, and synthesis (Cropley
and Cropley, 2005). Furthermore, problem finding may offer
another avenue to increase creative production (Nickerson, 1999).
Problem finding is a skill often found in art, yet is also necessary in
science and engineering. Both problem finding and problem solv-
ing are relevant to an engineer’s creativity; however, these attributes
have not been measured in great depth in engineering creativity
specifically. Such attributes need to be assessed and further devel-
oped by appropriate educational intervention activities (Cropley
and Cropley, 2005). The need to measure these attributes in indi-
viduals and teams would be appropriate and beneficial.
Measuring creativity in engineering design is necessary to assess
how these skills are demonstrated and developed in engineering
programs. Engineering students may profit by understanding con-
straints through reflective learning that are necessary to be creative
in the engineering field. Students may also become aware of their
own meta-cognitive processes to enhance their skills in engineering
design (Ishii and Miwa, 2005; Ishii et al., 2006).
Charyton (2005) and Charyton and Snelbecker (2007) investi-
gated measures to assess creativity in engineering students. Like the
work of Basadur and Finkbeiner (1985), Charyton and Snelbecker
(2007) explored much of the literature in creativity assessment and
consulted psychologists and engineers assessing creativity. Conclu-
sions indicated that measures such as the Myers Briggs Type Indi-
cator (MBTI) did not specifically address creativity in engineering
or engineering design. The MBTI has limitations and does not
specifically assess creativity (Larson, Thomas, and Leviness, 1999).
To date, existing engineering creativity measures are limited.
According to the literature available, there are few measures to as-
sess creative abilities in engineering design. The Owens Creativity
Test (Owens, 1960) was developed to assess mechanical engineer-
ing design. Test takers list possible solutions to mechanical prob-
lems. Its reliability ranged from 0.38 to 0.91 and its validity ranged
from 0.60 to 0.72, when applied to engineers in mechanically
related occupations. This assessment tool is out of print and is no
longer used.
Lawshe and Harris (1960) developed the Purdue Creativity Test
as an engineering personnel test, for identifying creative engineers
and their occupational potential. Participants are instructed to list as
many possible uses for one or two shapes that are provided. This as-
sessment has adequate reliability (0.86 to 0.95) and modest validity
(29 percent to 73 percent for low scorers and high scorers, respec-
tively). Validity was determined by assessing professional engineers
(product, process, and product engineers) working in industry.
Participants are instructed to generate original and novel possible
uses for single objects or pairs of objects. Scoring is based on fluency
(number of uses), and flexibility (differing categories of uses). Their
test does not directly assess originality. Although a reliable and valid
measure, limitations include little use in the field of engineering.
This assessment measures engineering creativity by assessing fluency
(number of responses) and flexibility (categories of responses). Both
the Owens Test and the Purdue Creativity Test only measure
divergent thinking.
Traditional divergent tests (e.g., the Purdue Creativity Test, and
the Owens Creativity Test) only measure lists of possible uses. The
146 Journal of Engineering Education April 2009
Page 3
April 2009 Journal of Engineering Education 147
Creative Engineering Design Assessment (CEDA) offers a new
method for assessing creative engineering design. Participants are
asked to sketch designs that incorporate one or several three-
dimensional objects, list potential users (people), and perform prob-
lem finding (generate alternative uses for their design) as well as
problem solving in response to specific functional goals. Sketching
is instrumental in design problem solving (Goldschmidt and
Smolkov, 2006) and results in creative solutions. Sketching is useful
for more creative results due to experience for spatial manipulations
that are domain specific. Design is crucial for creativity and innova-
tion for users and customers (Cockton, 2008). Engineering creativity
involves both convergent and divergent thinking. The CEDA
measures both convergent (generating a solution to the problem
posed) and divergent thinking (generating multiple solutions to
problems posed). Constraint satisfaction is assessed by measuring
the amount of shapes used as well as the materials added to each de-
sign. Schon (1983) also reported that Schein segregates convergent
science from divergent practice. Furthermore, Schein relegated
divergence to a residual category as a skill that is present in minor
professions, compared to convergence in major professions. He
stated that major professions include medicine and engineering
while other professions, such as education and social science are
attracted by the major professions as models. The study of creativity
in psychology has traditionally emphasized divergent thinking skills
(Torrance, 1974; Guilford, 1984). In the CEDA model, conver-
gent science and divergent practices are integrated as necessary
functions of cognitive processes that are assessed for creative
engineering design.
The CEDA measures constructs that are a part of the design
process as a step by step process, beginning with sketching a design
for each problem. Figure 1 shows the theoretical rationale of con-
structs necessary for the creative design process and for the selection
of instruments to assess creativity in engineering design. This figure
is based on previous studies to assess creativity as defined by the
person, process, product, and environment (Clapham, 2001;
Sternberg, 1999) and conceptualization of necessary creative
processes in engineering design (Cropley and Cropley, 2005; Finke,
Ward, and Smith, 1992; Stokes, 2006) that are domain specific
(Kaufman and Baer, 2005; Nickerson, 1999). Problem finding,
problem solving, divergent thinking, convergent thinking, and
constraint satisfaction are necessary in the creative process of engi-
neering design (see also Charyton, Jagacinski, and Merrill, 2008).
Our model, depicted in Figure 1 illustrates that personal attributes
of the individual defined as personality, temperament, and cognitive
risk tolerance influence one’s creative process. The environment of
the individual, defined as the engineering classroom or industrial
setting, also influences the creative process. The creative process is
defined as using divergent thinking, convergent thinking, con-
straint satisfaction, problem solving, and problem finding to create
a design. The creative process directly affects the product design. At
the same time, the product design dialectically influences the cre-
ative process. The product is shaped through the creative process.
At the same time, product development influences the creative
process. Last, the CEDA is a measure of the product design that is
developed by the creative process. Each portion of each problem
directly relates to psychological constructs of engineering creativity
that are described from the literature.
Figure 2 describes the theoretical rationale for the test construc-
tion of the CEDA, based on creativity literature specific to engi-
neering creativity shown in Figure 1. Figure 2 shows how each item
on the CEDA addresses these theoretical constructs. Divergent
thinking is assessed by generating multiple solutions. Convergent
Figure 1. Conceptualization of measures addressing creative mechanisms in engineering design (Charyton, Jagacinski, andMerrill, 2008).
Page 4
148 Journal of Engineering Education April 2009
thinking is assessed by solving the problem posed. Constraint satis-
faction is assessed by complying with the parameters of the direc-
tions and also adding additional materials and manipulating the
objects as desired. Problem finding is assessed by identifying other
uses for the design. Problem solving is assessed by deriving a novel
design to solve the problem posed.
The readability and comprehension of the CEDA is appropriate
for college students. The Simple Measure of Gobbledygook
(SMOG) (McLaughlin, 1969) online program was used to assess the
reading and comprehension level of the CEDA, using established
readability formulas, available at: http://www.harrymclaughlin.com/
SMOG.htm. This formula provides the widest range of educational
level and ability to match scores to actual education level. The
online SMOG calculator uses McLaughlin’s formula yielding a
0.985 correlation with the grade level of readers having 100 percent
comprehension of the tested materials. The SMOG is designed for
evaluating the reading level of materials that can be read indepen-
dently by a person without assistance from a teacher or instructor
(Richardson and Morgan, 1990). Readability is recommended at
the sixth to seventh grade level for educational materials for the
general public, being equivalent to junior high school. The SMOG
Grade for the CEDA was 8.81, being the 8th grade level, equiva-
lent to a junior high school reading and comprehension level.
Therefore the CEDA would also be appropriate and useful for
pre-college students at the high school level.
Our goals were to assess creativity through the newly developed
Creative Engineering Design Assessment (CEDA) in male and
female engineering students for comparison with male and female
non-engineering students. Our objectives were to establish test-
retest reliability of this assessment tool. Our research questions
included: (1) what is the relationship between the CEDA and other
general creativity measures in terms of a) test-retest consistency and
b) what is the relationship between creative engineering design
compare and general creativity; (2) what is the inter-rater agreement
between judges when using the new assessment tool on fluency,
flexibility, originality, and the overall CEDA; (3) what are the simi-
larities and differences between male and female engineering
students and male and female non-engineering students in terms of
both general creativity and creative engineering design; and (4) how
do engineering students with higher creative engineering design
compare with others on general creativity and creative engineering
creativity within a five week interval?
II. METHODOLOGY
Sixty one first year engineering students (49 percent were
freshmen or sophomores) consisting of 57 males and six females;
and 21 one non-engineering students (95 percent of students were
freshmen or sophomores) consisting of six males, and 15 females at
a mid-western U.S. university were administered the newly devel-
oped Creative Engineering Design Assessment (CEDA) and a
demographic questionnaire with established general creativity (not
specific to engineering creativity) measures including: (1) the Creative
Personality Scale (CPS), (2) the Creativity Temperament Scale
(CT), and (3) the Cognitive Risk Tolerance Scale (CRT) during a
fundamentals of engineering five week summer course. The engi-
neering course consisted of basic skills and a laboratory design
course to engage students in a quarter long design project. This
course is a part of the First Year Engineering Education curriculum.
The course contained hands on design activities, where the final
project objective was to design a functional roller coaster. The
course also consisted of sketching exercises and journal assign-
ments. Learning outcomes were to gain sketching and design skills
necessary to engineering.
Non-engineering students were from various majors and were
recruited as volunteers from an introductory psychology course
where they were encouraged to participate in research studies. Non-
engineering students did not have the same roller coaster project,
the same sketching exercises, or the same journal assignments.
Learning objectives and outcomes of the psychology course were to
gain knowledge about the various areas and fields in psychology.
Students were not matched based on performance, grade point
average, Scholastic Achievement Test scores, or other performance
variables. Both groups were assessed in a five week interval. Partici-
pants were given a number for identification purposes according to
the Institutional Review Board approved procedures.
III. INSTRUMENTS
A questionnaire was administered requesting demographic
information. This included age, gender, ethnicity, major, and spe-
cialization area within the major. Additional questions included
“How often does an engineer use creativity in a given day?” and
“How often do you use creativity in a given day?”
Figure 2. Creative engineering design assessment meta-cognitive processes measured.
Page 5
April 2009 Journal of Engineering Education 149
1) CPS: Creative Personality Scale: The Creativity Personality
Scale (CPS) of the Adjective Checklist (ACL) (Gough, 1979) was
administered to assess creativity attributes. This test for creative
thinking was chosen because it is highly regarded, reliable, and
widely used as a creativity test. For example, Plucker and Renzulli
(1999, p. 46) stated that, “ Oldham and Cummings (1996, p. 609),
in a comparison of personality traits, environmental characteristics,
and product ratings, found evidence that people with specific per-
sonality traits (i.e., as judged by Gough’s (1979) Creativity Person-
ality Scale) produced creative products when challenged by their
work and supervised in a supportive ‘non-controlling fashion’.”
Furthermore, normative data for the mean scores of 66 engineering
students was 3.88 with a standard deviation of 3.94 compared to
256 males 3.57 with a standard deviation of 3.99 (Gough, 1979).
2) CT: Creative Temperament Scale: The Creative Temperament
Scale (Gough, 2000) adapted from the California Psychological
Inventory (CPI) was designed to assess personality characteristics
and predict what people will say and do in specific contexts. The
Creative Temperament Scale is one of the special purpose scales of
the CPI. Gough, gave permission to extract the CT scale from the
CPI (H. Gough, personal communication, June 28, 2003). Norma-
tive data for the mean scores of 66 engineering students was 24.55
with a standard deviation of 4.98 compared to 22.65 and 5.46,
respectively, for the general normative data (n � 3,235) (Gough,
2000).
3) CRT: Cognitive Risk Tolerance Survey: The Cognitive Risk
Tolerance Survey (Snelbecker, McConologue, and Feldman) “con-
sists of 35 self report items designed to assess an individual’s ability
to formulate and express one’s ideas despite the threat of negative
assessment regarding: reputation, integrity, credibility, honor and
intelligence” ( J. Feldman [formerly J. Teitlebaum], personal com-
munication, May 27, 2003). Responses are on a Likert Scale rang-
ing from 0 (Very Strongly Disagree) to 9 (Very Strongly Agree).
Higher scores indicate higher levels of cognitive risk tolerance. The
Cognitive Risk Tolerance Survey was developed as an extension of
an earlier risk tolerance model developed by Snelbecker and col-
leagues (Roszkowski, Snelbecker, and Leimberg, 1989; Snelbecker,
Roszkowski, and Cutler, 1990). Reliability of this psychometric
instrument has proven to be adequate with a Cronbach’s alpha coef-
ficient of 0.76 during Feldman’s (2004) pilot study with 78 respon-
dents, and a Cronbach’s alpha coefficient of 0.78 during the main
section of her dissertation study with 84 respondents.
4) Creativity Engineering Design Creativity (CEDA): The new
assessment tool—the Creativity Engineering Design Assessment
(see also Charyton, Jagacinski, and Merrill, 2008) consists of five
design problems with five parts each to assess an individual’s ability
to formulate and express design ideas through sketching, providing
descriptions, identifying materials, and identifying problems that
the design solves and its potential users. Instructions are to generate
as many designs with at least one design per problem. Additionally,
at least one response should be indicated for each of the five ques-
tions for each design. Total time for this assessment is 25 minutes
for five pages, or about five minutes per page. Appendix A contains
a sample problem (see also Charyton, Jagacinski, and Merrill,
2008).
Dimensions of the assessment tool included both problem solving
and problem finding exercises. Participants were evaluated according
to their originality (0–10), fluency (amount of ideas), and flexibility
(differing types of ideas) for each design problem. Appendix B
contains scoring criteria (see also Charyton, Jagacinski, and Merrill,
2008).
Two judges were selected: one from engineering and one from
psychology. Two of the CEDA test developers trained the judges.
Judges practiced scoring in a team environment; however, each
judge evaluated the CEDAs separately. Judges evaluated: fluency
(amount of items listed), flexibility (categories of responses per
problem), and originality (on an eleven point scale found in
Appendix B). Student identification numbers were hidden from
the judges.
IV. RESULTS
1) The CEDA in comparison with other general creativity measures:(a) Correlations between the test and retest in the five week
interval were conducted to establish the reliability of the
CEDA in comparison with other established creativity
measures. The CEDA was consistent for the test and retest
(r � 0.563) like the other general creativity measures such
as the CPS (r � 0.569), CT (r � 0.512) and CRT
(r � 0.428), p � 0.01 for all comparisons.
(b) Correlations among the instruments were conducted to
identify their relationships with the CEDA. The CEDA
has low correlations with the CPS (r � �0.007), CT
(r � �0.131), and the CRT (r � �0.187) for both the test
and the retest CPS (r � �0.119), CT (r � �0.032), and
CRT (r � �0.159), p � 0.05 for all comparisons.
2) Interrater reliability: Correlations among the CEDA scores of
two judges were conducted to identify their relationships with each
other and establish reliability. The judges were in agreement
(r � 0.98) with their overall test and retest scoring. Inter-rater relia-
bility for flexibility (r � 0.90, r � 0.98) and originality (r � 0.80;
r � 0.85) indicated consistency in both test and retest measures,
respectively. Therefore, both judges’ scores were average for com-
parison and analysis.
3) Similarities and differences in creativity assessments: Similarities
and differences in general creativity in engineering students and
non-engineering students were examined. A MANOVA was used
to detect differences in three general creativity measures (creative
personality, creative temperament, and cognitive risk tolerance).
Figure 3 contains a depiction of the means for male and female
engineering, and male and female non-engineering students on the
Creative Engineering Design Assessment (CEDA) compared with
the following general creativity variables: (1) Creativity Personality
Scale (CPS), (2) Creativity Temperament Scale (CT), (3) Cognitive
Risk Tolerance Scale (CRT). No significant differences were found
between engineering students and non-engineering students.
Figure 4 contains a depiction of the mean retest scores for low
(1–84.5), medium (85–136.5), and high (137–300) creative engi-
neering design (based on the CEDA test scores) for engineering
students on the CEDA retest compared with the following general
creativity variables: (1) Creativity Personality Scale (CPS), (2)
Creativity Temperament Scale (CT), (3) Cognitive Risk Tolerance
Scale (CRT). No significant differences were found for engineering
students.
a) General creativity: A two-way MANOVA was calculated
examining the effect of class and gender on general creativity
measures before and after their class. No significant differences
Page 6
150 Journal of Engineering Education April 2009
were found in terms of general creativity for engineering and
non-engineering students (F(6) � 0.718, p � 0.636), gender
(F(6) � 0.633, p � 0.704) or their interaction (F(6) � 0.831,
p � 0.550).
b) Creative Engineering Design: Kruskal Wallis analyses were
calculated examining the effect of class and gender on creative engi-
neering design. A significant difference was found for creative
engineering design on the CEDA test �2 � 8.392, p � 0.039, and
CEDA retest �2 � 24.70, p � 0.000 for engineering students.
Follow-up analyses were completed through Mann-Whitney
analyses. A significant difference was found only for male and
female engineering students for the retest only, p = 0.008 indicating
that male engineering students tended to perform at lower levels
in comparison with the female engineering students who had
significantly higher creative engineering design in the CEDA
retest only. No significant differences were found between male and
female engineering students and male and female non-engineering
students.
4) High creative engineering design compared with low and mediumcreative design based on CEDA test scores: Based on creative engineer-
ing design for high creative engineering design engineering
students, low, medium, and high scorers were compared on the
retest. No significant differences were found for general creativity
(F(6) � 1.30, p � 0.261). Since ANOVA results did not indicate
homogeneity, Welch and Brown-Forsythe analyses were performed
to compare creative engineering design. A significant difference was
Figure 3. Descriptive statistics of means for creative engineering design and general creativity measures by group and gender.�indicates statistically significant differences
Figure 4. Descriptive statistics of means for engineering students for creative engineering design and general creativity measures bylow, medium, and high overall CEDA test scores.�indicates statistically significant differences
Page 7
found Welch (F(2) � 5.56, p = 0.009); Brown-Forsythe (F(2) �6.22, p � 0.005). Tukey posthoc analyses revealed that high creative
engineering design scorers significantly demonstrated higher cre-
ative engineering design than low scorers (p � 0.001) and medium
scorers (p � 0.038). Paired samples t-tests were performed to com-
pare test and retest differences of all measures. Differences were not
detected on the CPS (p � �0.206, p � 0.838), CT (t � �1.166,
p � 0.255), CRT (t � �0.884, p � 0.385), or CEDA (t � 0.361,
p � 0.721) for low CEDA scorers. Differences were not detected
on the CPS (t � �1.349, p � 0.192), CT (t � 0.452, p � 0.656),
or CRT (t � �0.377, p � 0.710) for medium or high scorers on
the CPS (t � 0.431, p � 0.673), CT (t � 2.082, p � 0.056), or
CRT (t � �1.102, p � 0.289). However, significant differences
were found on the CEDA test and retest for medium CEDA
scorers (t � 3.653, p � 0.001), and high CEDA scorers (t � 3.443,
p � 0.004).
V. DISCUSSION
Our first research question concerned reliability of the CEDA
in comparison with established general creativity measures. The
CEDA demonstrated statistically significant consistency, like the
established general creativity measures between test and retest
scores (Charyton, Jagacinski, and Merrill, 2008). Furthermore, the
CEDA was not related to the general creativity measures. This
finding supports the notion that creative engineering design is dif-
ferent than general creativity. Other studies (Charyton and Snel-
becker, 2007) found that music improvisation (music creativity)
was found to be unrelated to general creativity. The Purdue Cre-
ativity Test, a test of engineering creativity, was found to be mod-
estly related to general creativity measures, suggesting that the
Purdue Creativity Test may have been evaluating similar con-
structs to general creativity. Furthermore, other divergent thinking
tests, like the Purdue Creativity Test, the Owens Creativity Test,
the Torrance Test for Creative Thinking, and the Structure of In-
tellect Model measure divergent thinking, which is a component
of creativity. The CEDA measures divergent thinking, but also
measures convergent thinking—both are necessary for creativity in
engineering design. This study demonstrated how creative engi-
neering design is different from general creativity constructs; how-
ever, more research is needed to assess how creative engineering
design is similar and or different than other divergent thinking
tests in engineering.
Agreement among judges was high for fluency, flexibility, origi-
nality, and overall scores. Fluency reliability was high, due to the na-
ture of the construct being counting responses. Flexibility was still
high relating to categories of responses. Last, originality was high
measuring the quality of originality that the designs produced
solved the given problem. Some responses were more typical,
indicating a lower rating, while other designs demonstrated higher
novelty and unusual responses, indicating a higher rating. We
speculate that high reliability for originality may be due to the com-
bination of descriptors and numeric values in the scoring rubric.
Judges were trained to assess the designs and generate their own
reaction to the designs, then select the word and appropriate num-
ber to describe their analysis. It is plausible that the combination of
verbal and numeric ratings is a potential reason for the high inter-
rater agreement. The high agreement of judges was encouraging
since both the engineer and psychologist were in high agreement
for originality (novelty) as well as fluency, flexibility, and overall
scores. The reliability for the CEDA was similar to previous stud-
ies, indicating high inter-rater agreement (Charyton, Jagacinski,
and Merrill, 2008).
Similarities and differences were found among engineering stu-
dents in terms of general creativity and creative engineering design.
First, engineering students did not differ in their overall levels of
general creativity compared with non-engineering students. General
creativity was measured by creative personality, creative tempera-
ment, and cognitive risk tolerance in five week test and retest inter-
vals. Although we anticipated that both male and female engineers
may have higher levels of creative engineering design on the retest
due to the nature of the engineering course and factors related to
retesting, male engineering students had significantly lower creative
engineering design only during the retest in comparison with only
the female engineering students. There were no significant differ-
ences between the male and female engineering and male and female
non-engineering students. We do not know why only engineering
males decreased; however, lower scores for male engineering
students was also found in a previous study (Charyton, Jagacinski,
and Merrill, 2008). This trend will be examined in further research.
We speculate that a decreased score may be due to students learning
new design techniques, since only the engineering students were
exposed to sketching and a rollercoaster project in their design
course. However, this finding did not affect the female engineering
students in the same manner. We plan to investigate this phe-
nomenon in a larger sample size to determine if there are gender
differences in engineering students or if acquiring new design skills
interacts with creative engineering skills. When some students learn
a new skill, there may be a refractory period when reproducing that
skill, since they are applying a new method (Bransford, Brown, and
Cocking, 2000; Kidder 1981). This may explain a decrease in their
retest CEDA scores. However, medium and high engineering stu-
dent scorers on the CEDA also tended to have significantly lower
scores in the second retest administration. Therefore, it may be likely
that the combination of time constraints and re-exposure to generate
novel and original designs to the same problems were difficult for
some students such as the engineering male students, medium, and
high scorers. For future studies assessing creative engineering design,
longer time durations between administrations with less time
constraints may be more helpful to engineering students.
Limitations of this study include a small sample size. In our
study, we had fewer female engineering students compared to
male engineering students. However, females account for less than
20 percent of overall engineering students. Future research will
investigate similarities and differences among engineering students
in larger sample sizes. With a larger sample size, we may be able to
investigate similarities and differences among engineering students
by gender in greater detail. This study did not compare the CEDA
with other engineering creativity measures, such as the Purdue
Creativity Test. Further research is needed to compare the CEDA
with the Purdue Creativity Test. Future studies will assess creativity
outcomes in longer time durations.
High creative engineering design students statistically differed
from low and medium scorers in creative engineering design,
despite scores decreasing for medium and high scorers in the retest
administration. Various authors have contended that creativity has
been neglected or dismissed as being less important in curriculum.
April 2009 Journal of Engineering Education 151
Page 8
152 Journal of Engineering Education April 2009
This is of particular interest since we speculated that if students
are given the exposure, such as hands on design training, they may
develop and hone creativity and innovation skills. Furthermore, it
is plausible that once creative skills are learned, students may con-
tinue to demonstrate these skills in comparison with those who do
not receive hands on activities in engineering design. Further-
more, this assessment tool may offer an option to assess creativity
in engineering university programs (Soibelman and Pena-Mora,
2000). Through teaching and measuring creativity in the engi-
neering curriculum, students may have the opportunity to hone
creative engineering design skills by practicing. Like some creativ-
ity researchers (Torrance 1974), the test developers of the CEDA
believe that creativity is a skill that can be developed through a
supportive learning environment. This viewpoint is consistent
with other available literature in engineering education, indicating
that if students are expected to demonstrate skills, they need the
opportunity to practice (Felder, 1988; 1987; Felder et al., 2000;
Felder and Brent, 2003; Isaksen and Parnes, 1985). Thus, both
creativity researchers and engineering educators are in agreement
for the need to support and facilitate creativity in education
through a supportive environment with opportunities for reflective
practice (Ishii and Miwa, 2005; Schon, 1983). Furthermore, these
necessary skills for innovation could be assessed for instructor
feedback of student learning.
The newly developed assessment tool differed from the other
general creativity measures, like previous studies (Charyton,
Jagacinski, and Merrill, 2008) indicating differences in the CEDA
and similarities in general creativity. Attributes such as tempera-
ment and personality may tend to be more stable traits (Charyton
and Snelbecker, 2007; Gough, 1979; Oldham and Cummings,
1996). Charyton and Snelbecker (2007) proposed that cognitive
risk tolerance may be a component of general creativity that is likely
to be moderately related to, but different from, other general
creativity measures that may be more flexible, depending on an in-
dividual’s comfort level in various situations or conditions of uncer-
tainty. Furthermore, it is plausible that there may also be variation
in creative engineering design, since students can learn and develop
design skills in the engineering classroom. Students may learn
creativity through hands on activities that foster meta-cognition
processes about creativity and design (Ishii and Miwa, 2005).
CONCLUSIONS AND IMPLICATIONS
Students need the opportunity to practice creative design skills.
Faculty need to offer opportunities to develop this skill and assess
students on their progress. Perhaps once this skill has been devel-
oped, students can continue to demonstrate this skill. The CEDA
offers a tool of assessment to measure the challenging construct of
creativity in engineering design, which is a necessary skill for inno-
vation. To date, few tools exist to assess this construct. Since cre-
ativity is a necessary skill to be innovative and many engineering
programs strive to produce innovative engineers, this tool has a
practical application that could benefit engineering education.
Furthermore, skills that foster innovation, such as creativity, should
be a component of the curriculum so that students can practice and
develop these skills.
Continued research development on this instrument by engi-
neering and psychology faculty can benefit students within our
university and at other universities. Some researchers have indicated
the need to assess creativity in engineering classes (Cropley and
Cropley, 2005; Felder, 1987; Felder et al., 2000). The CEDA tool
could be utilized for educational purposes. Furthermore, creativity
as a component of the engineering curriculum could be an interven-
tion to provide students with more opportunities to develop these
necessary skills that are a part of being an engineer. Through under-
standing the value and nature of usefulness (Larson, Thomas, and
Leviness, 1999; Nickerson, 1999) as a key component of engineer-
ing creativity, we can enhance our understanding of creative
processes in engineering and in other domains. The understanding
of meta-cognition and the engineering creativity process will lead
engineering educators and creativity researchers to complement
each other toward increasing both the frequency and quality of
inventiveness. Creativity is key for innovation in industry. Inven-
tiveness can benefit many human conveniences (Simonton, 2000).
Thus, by providing a method for assessing creativity in engineering
design, educators can enable students to develop their talents as
future innovative engineers.
ACKNOWLEDGMENTS
The authors would like to thank Dan Wisniewski, who assist-
ed as a judge; Glenn Elliott for assisting with graphic design tech-
nology for our assessment tool; John Elliott for his critical reading
of the manuscript and critical assistance with data management;
Richard Jagacinski for his valuable feedback on the CEDA design
and Mohammed A. Rahman for statistical consultation. Prelimi-
nary data were presented as invited talks at the National Science
Foundation in Arlington, Virginia and the American Psychologi-
cal Association Convention in New Orleans, Louisiana. Prelimi-
nary data were also presented as a symposium on Creativity and
Innovation at the Midwestern Psychological Association conven-
tion in Chicago, Illinois.
REFERENCES
Basadur, M., and C.T. Finkbeiner. 1985. Measuring preference for
ideation in creative problem-solving training. Journal of Applied Behavioral
Science 21 (1): 37–49.
Bransford, J.D., A.L. Brown, and R.R. Cocking. 2000. How people
learn: Brain, mind, experience, and school. Washington, DC: National
Academies Press.
Charyton, C. 2005. Creativity (scientific, artistic, general) and risk tolerance
among engineering and music students. Ph.D. dissertation. Philadelphia, PA:
Temple University.
Charyton, C., R.J. Jagacinski, and J.A. Merrill. 2008. CEDA: A re-
search instrument for creative engineering design assessment. Psychology of
Aesthetics and Creativity in the Arts 2 (3): 147–54.
Charyton, C., and G.E. Snelbecker. 2007. General, artistic and scien-
tific creativity attributes of engineering and music students. Creativity
Research Journal 19 (2–3): 213–225.
Clapham, M.M. 2001. The effects of affect manipulation and informa-
tion exposure on divergent thinking. Creativity Research Journal 13 (3–4):
335–50.
Cockton, G. 2008. Designing worth- Connecting preferred means to
desired ends. Interactions 15 (4): 54–57.
Page 9
Cropley, D., and A. Cropley. 2005. Engineering creativity: A sys-
tems concept of functional creativity. Mahwah, NJ: Lawrence Erlbaum
Associates.
Csikszentmihalyi, M. 1999. Implications of a systems perspective for
the study of creativity. In Handbook of Creativity, ed. R.J. Sternberg.
Cambridge, MA: MIT Press.
Elliott, M. 2001. The well-rounded IE: Breakthrough thinking. IE
Solutions. Oct: 22–25.
Felder, R.M. 1987. On creating creative engineers. Engineering Educa-
tion 77 (4): 222–27.
Felder, R.M., and L. Silverman. 1988. Learning and teaching styles in
engineering education. Engineering Education 78 (7): 674–81.
Felder, R.M., G. Felder, and J. Dietz. 1998. A longitudinal study of
engineering student performance and retention. V. Comparisons with
traditionally-taught students. Journal of Engineering Education 87 (4):
469–80.
Felder, R.M., and R. Brent. 2003. Designing and teaching courses to
satisfy the ABET engineering criteria. Journal of Engineering Education
92 (1): 7–25.
Felder, R.M., D.R. Woods, J.E. Stice, and A. Rugarcia. 2000. The
future of engineering education. II. Teaching methods that work. Chemical
Engineering Education 34 (1): 26–39.
Feldman, J.M. 2004. The relationship among college freshmen’s cognitive risk
tolerance, academic hardiness, and emotional intelligence and their usefulness in
predicting academic outcomes. Ph.D. dissertation. Philadelphia, PA: Temple
University.
Ferguson, E.S. 1992. Engineering and the mind’s eye. Cambridge, MA:
MIT Press.
Finke, R.A., T.B. Ward, and S.M. Smith. 1992. Creative cognition:
Theory, research, and applications. Cambridge, MA: MIT Press.
Goldschmidt, G., and M. Smolkov. 2006. Variances in the impact
of visual stimuli on design problem solving performance. Design Studies
27 (5): 549–69.
Gough, H.G. 2000. The California Psychological Inventory. Mahwah,
NJ: Lawrence Erlbaum Associates.
Gough, H.G. 1979. A creative personality scale for the Adjective
Check List. Journal of Personality and Social Psychology 37 (8): 1398–1405.
Green, G., and P. Kennedy. 2001. Redefining engineering education:
The reflective practice of product design engineering. International Journal
of Engineering Education 17 (1): 3–9.
Guilford, J.P. 1984. Varieties of divergent production. Journal of
Creative Behavior 18 (1): 1–10.
Isaksen, S.G. 1983. Toward a model for the facilitation of creative
problem solving. Journal of Creative Behavior 17 (1): 18–31.
Isaksen, S.G., and S.J. Parnes. 1985. Curriculum planning for cre-
ative thinking and problem solving. Journal of Creative Behavior 19 (1):
1–29.
Ishii, N., and K. Miwa. 2005. Supporting reflective practice in creativity
education. In Proceedings of the 5th conference on Creativity & Cognition.
London, England.
Ishii, N., Y. Suzuki, H. Fujiyoshi, T. Fujii, and M. Kozawa. 2006. A
framework for designing learning environments fostering creativity. In
Current developments in technology-assisted education, eds. A. Méndez-Vilas,
A. Solano Martín, J.A. Mesa González, and J. Mesa González, 228–232.
Badajoz, Spain: FORMATEX.
Kaufman, J.C., and J. Baer, eds. 2005. Creativity across domains: Faces of
the muse. Mahwah, NJ: Lawrence Erlbaum Associates.
Kidder, Tracy. 1981. The soul of a new machine. 1st ed. Boston, MA:
Little, Brown.
Larson, M.C., B.H. Thomas, and P.O. Leviness. 1999. Assessing
creativity in engineers. American Society of Mechanical Engineers, Design
Engineering Division (Publication) DE 102: 1–6.
Lawshe, C.H., and D.H. Harris. 1960. Manual of instructions to
accompany Purdue creativity test forms G and H. Princeton, NJ: Educational
Testing Services.
McLaughlin, G.H. 1969. SMOG grading: A new reading formula.
Journal of Reading 12 (8): 639–40.
Nickerson, R.S. 1999. Enhancing creativity. In Handbook of creativity,
ed. Robert J. Sternberg. Cambridge, MA: Cambridge University Press.
Oldham, G.R., and A. Cummings. 1996. Employee creativity: Personal
and contextual factors at work. Academy of Management Journal 39 (3):
607–34.
Owens, W.A. 1960. The Owens’ creativity test. Ames: Iowa State
University Press.
Plucker, J.A., and J.S. Renzulli. 1999. Psychometric approaches to the study
of human creativity. New York: Cambridge University Press.
Richardson, J.S., and R.F. Morgan. 1990. Reading to learn in the content
areas. Belmont, CA: Wadsworth Publishing Co.
Roszkowski, M.J., G.E. Snelbecker, and S.R. Leimberg. 1989. Risk
tolerance and risk aversion. In The tools and techniques of financial planning,
ed. Stephan R. Leimberg. Cincinnati, OH: The National Underwriter
Company.
Schon, D.A. 1983. The reflective practitioner: How professionals think in
action. New York: Basic Books.
Shah, J.J., S.M. Smith, and N. Vargas-Hernandez. 2003. Metrics for
measuring ideation effectiveness. Design Studies 24 (2): 111–34.
Simonton, D.K. 2000. Creativity: Cognitive, personal, developmental,
and social aspects. American Psychologist 55 (1): 151–58.
Smilansky, J., and N. Halberstadt. 1986. Inventors versus problem
solvers an empirical investigation. The Journal of Creative Behavior 20 (3):
183–201.
Snelbecker, G.E., T. McConologue, and J.M. Feldman. Cognitive risk
tolerance survey. Unpublished manuscript.
Snelbecker, G.E., M.J. Roszkowski, and N.E. Cutler. 1990. Investors’
risk tolerance and return aspirations, and financial advisors’ interpretations:
A conceptual model and exploratory data. Journal of Behavioral Economics
19 (4): 377–93.
Soibelman, L., and F. Pena-Mora. 2000. Distributed multi-reasoning
mechanism to support conceptual structural design. Journal of Structural
Engineering 126 (6): 733.
Sternberg, R.J. 1999. Handbook of creativity. New York: Cambridge
University Press.
Sternberg, R.J., and N.K. Dess. 2001. Creativity for the new millennium.
American Psychologist 56 (4): 332.
Stokes, P.D. 2006. Creativity from constraints: The psychology of break-
through. New York: Springer Publishing Co.
Thompson, G., and M. Lordan. 1999. Review of creativity principles
applied to engineering design. Proceedings of the Institution of Mechanical
Engineers, Part E: Journal of Process Mechanical Engineering 213 (1): 17–31.
Torrance, E.P. 1974. Torrance tests of creative thinking. Lexington, MA:
Personnel Press/Ginn and Co./Xerox Education Co.
Treffinger, D.J. 2003. Assessment and measurement in creativity and
creative problem solving. In The educational psychology of creativity, ed. J.C.
Houtz. Cresskill, NJ: Hampton Press.
Weisberg, R. 1986. Creativity: Genius and other myths. New York:
W.H. Freeman/Times Books/ Henry Holt & Co.
Weisberg, R.W. 1999. Creativity and knowledge: A challenge to theories.
New York: Cambridge University Press.
April 2009 Journal of Engineering Education 153
Page 10
AUTHORS’ BIOGRAPHIES
Christine Charyton was previously a visiting assistant professor
in clinical/counseling psychology at Ohio State University, Newark
and is currently a lecturer in the Department of Psychology at Ohio
State University and a psychologist at the Cognitive Behavioral
Center of Greater Columbus. Dr. Charyton studies the aesthetic
science of creativity and innovation in engineering and music. Her
research emphasizes the necessity of creativity as a vehicle for inno-
vation in engineering education and industry. Her dissertation in-
vestigated general, artistic, and scientific creativity and risk toler-
ance in engineering and music students. Dr. Charyton also
conducts research using nonlinear dynamics and studies neurologi-
cal conditions such as epilepsy. She provides psychotherapy to chil-
dren, adolescents, and adults with a variety of psychological condi-
tions. She has postdoctoral expertise in treating children with
autism, pervasive developmental disorders, ADHD and other
conditions and has co-facilitated parenting groups.
Address: Ohio State University, 1827 Neil Avenue, 130 Lazenby
Hall, Columbus, OH 43210; e-mail: [email protected] . Contact
the authors by e-mail for additional detailed material.
John A. Merrill is the director for the First-Year Engineering
Program at The Ohio State University College of Engineering,
and has worked in this capacity for over nine years. The program
serves approximately 1,300 students annually in courses orga-
nized to ensure student success through rigorous academics in a
team-based environment. His responsibilities include opera-
tions, faculty recruiting, curriculum management, student reten-
tion, and program assessment. Dr. Merrill received his Ph.D. in
Instructional Design and Technology from The Ohio State
University in 1985, and has an extensive background in public
education, corporate training, and contract research. He has
made frequent presentations at conferences held by the Ameri-
can Society for Engineering Education (ASEE) and its affiliate
conference, Frontiers in Education (FIE). He is part of the re-
search team that was recently awarded an NSF grant to study
strategies for maximizing success among students with learning
disabilities.
Address: The Ohio State University, 2070 Neil Ave., 244C
Hitchcock Hall, Columbus, OH 43210-1278; telephone: (�1)
614.292.0650; fax: (�1) 614.247.6255; e-mail: merrill.25@
osu.edu.
154 Journal of Engineering Education April 2009
Page 11
APPENDIX
April 2009 Journal of Engineering Education 155
Appendix A. CEDA Sample Problem (see also Charyton, Jagacinski, and Merrill, 2008).
Page 12
156 Journal of Engineering Education April 2009
Appendix B. Scoring the CEDA. (see also Charyton, Jagacinski, and Merrill, 2008).