Proceedings of the ASME 2018 International Design ...167.99.111.205/wp-content/uploads/2018/12/DETC2018-86291.pdf · 2016 IDETC conference and graduate engineering students at Stanford.

1 Copyright © 2018 by ASME

Proceedings of the ASME 2018 International Design Engineering Technical Conference & Computers and Information in Engineering Conference

IDETC/CIE 2018 August 26-29, 2018, Quebec City, Quebec, Canada

DETC2018-86291

EFFECTS OF COLLAGE PRIMING ON SUSTAINABLE DESIGN IDEA CREATION AND ASSESSMENT

Ting Liao Graduate Student

Mechanical Engineering Stanford University Stanford, CA, USA

[email protected]

Erin F. MacDonald Assistant Professor

Mechanical Engineering Stanford University Stanford, CA, USA

[email protected]

ABSTRACT Priming is a psychological technique that can alter

designers’ mindsets prior to conceptual design exercises [1]. For

example, priming the five senses enhanced designers’ abilities to

communicate sustainability through the product features they

designed [2,3]. Although the three pillars of sustainable design—

social desirability, economic competitiveness, and

environmental friendliness—are all important, they are not

necessarily equally accessible or salient during the design

process. This paper applies the collage priming method of [2] to

(1) increase/improve ideas related to the sustainability pillars, in

the eyes of users, and (2) reduce ownership bias and cause a more

favorable judgment of others’ ideas, when compared to one’s

own ideas. An experiment tests (1) and (2) for the collage

priming method versus a reading preparation activity and no

prime/activity for effectiveness in these two applications. The

participants included graduate design student attendees at the

2016 IDETC conference and graduate engineering students at

Stanford. For (1), collage priming is proven to be successful in

helping designers to generate ideas that are more

environmentally friendly but less successful in helping designers

generate ideas related to social desirability and economic

competitiveness, as judged by potential users; no more

successful than a reading exercise. For (2), we find evidence that

the collage priming reduces ownership bias in designers, as

measured in their judgment of other (simulated) designers’ ideas,

and in this case the reading exercise does not have the same

effect.

1 INTRODUCTION Sustainability has three important pillars: social/user

desirability, economic competitiveness, and environmental

friendliness [4]. Social/user desirability refers to the well-being

of a community and its members, as a sustainable system is

expected to have a positive influence on human equality, social

justice, members’ happiness, etc. [5-6]. In this study, as it is hard

to fully investigate the well-being of a community, we focus on

the local level of the social pillar, namely user desirability.

Economic competitiveness means that a sustainable business or

product must allocate its resources efficiently to be consistently

profitable. Again, to focus on a local level, we interpret

economic competitiveness as the manufacturing cost and use

cost of a product. Of the three pillars, environmental friendliness

relates most closely to the training of engineers, such as in

material selection, resource consumption, design of assembly

and disassembly, etc. Thus, it makes sense that when engineers

approach sustainable product design, they may tend to gravitate

toward environmental friendliness and pay less attention to the

aspects of social impact/user desirability and economic

competitiveness.

A designer’s mindset has a strong impact on the outcome of

the design process, especially their mindset in the early

conceptual stages of the design process, such as ideation. This

mindset is a combination of longstanding knowledge as well as

recently acquired stimuli and new information. One method to

alter a designer’s mindset is through priming—a psychological

technique that aims to affect performance on a task via exposure

to a stimulus that activates a particular idea, contextualization, or

feeling [7]. Priming alters a designer’s mindset by not only

bringing background knowledge into the foreground, but also by

introducing new information that subsequently influences, for

example, ideas generated. The psychological mechanism of

priming is rooted in the concept of “perceptual readiness”

proposed by Bruner [8], which claims that the information and

feelings that are currently cognitively accessible lead to

corresponding thoughts and behavior. Priming, which activates

a specific set of information and/or feelings in the brain,

increases the accessibility of thoughts, memories, and feelings


associated with this activation, and thus motivates related

perspectives, decisions, and behavior [1,9]. For numerous

examples, refer to [1]. Priming, which is subconscious, is

different from directly, consciously engaging a topic, such as a

reading an article about environmental design.

She and MacDonald found that their collage priming

method—where participants physically handled images of

products to create a paper picture collage—enhanced student

engineers’ abilities to generate features that communicate

environmental friendliness to the user [2]. This collage priming

method was developed based on the work of Guyton, who

demonstrated collage activity an effective approach to establish

product semantics for sustainable products and to inspire

sustainability concerns [2, 10]; for more information on the

motivation and origin of the method, refer to [2]. Subsequent

testing of prototypes built to include the best of the features

thought-up with this method demonstrated that potential

customers increased their consideration of sustainability during

hypothetical purchase decisions [3]. Thus, the collage priming

method, with respect to designing improved communication of

sustainability, seems effective.

Here, we tested to see if the collage priming method,

adapted from [2], can effectively increase/improve the ideas

related to all three pillars of sustainable design during ideation,

especially for social and economic factors. We also wondered if

performing the priming collage task could better prepare

designers for the next step of the design process in a

collaborative environment, namely idea evaluation, as priming

has been known to affect decision-making [1]. While we were

not able to fully-investigate this question, we investigated how

designers, with and without collage priming, judged their own

ideas during a mock-up of an idea selection process. Onarheim

and Christensen [11] found that every participant proved to be

more likely to select his or her own ideas over other ideas when

individually screening ideas in engineering design. Designers

tend to unconsciously favor their own ideas, known as ownership

bias, which can affect their objectivity during idea selection and

ultimately the final product [11].

We ask: (1) Does the collage prime increase/improve ideas

related to the sustainability pillars, in the eyes of users? and (2)

Does the collage prime reduce ownership bias and cause a more

favorable judgment of others’ ideas, when compared to one’s

own ideas?

In this study, we divided designers into five treatment

conditions. There were three treatment categories: (A) a priming

stimulus of a collage activity relating to either social, economic,

or environmental aspects of sustainability; (B) a reading activity

that directly and consciously engaged the reader on the

environmental aspect of sustainability; or (C) no stimuli. The

designers, working alone, generated ideas for washing machines,

and then made judgments about their ideas and the ideas of other

designers (which were simulated). Then, we had potential users

on Amazon Mechanical Turk assess all ideas generated. Next,

we analyzed all results to test the ability of the primes to increase

the number of effective design ideas generated by designers, to

improve the quality (defined by originality and feasibility) of

ideas, and to reduce ownership bias during idea evaluation.

This paper is organized as follows: Section 2 provides a

brief review of sustainable design and of priming techniques in

psychology, marketing, and engineering design. Section 3 lists

our propositions and hypotheses. Section 4 gives an overview of

the experiment, while Section 5 is an explanation of how we

prepared the experiment. Section 6 describes the experiment

itself in detail. Data and statistical analysis is shown in Section

7, and Section 8 provides our discussion. Finally, Section 9 is the

conclusion and a description of future work.

2 BACKGROUND

2.1 Sustainable Design and Framing of Sustainability Product design has a huge effect on global sustainability

because many design decisions, such as choice of materials or

manufacturing plan, extensively impact a product’s entire life

cycle [12]. To create more sustainable products and more

sustainable processes, engineers have investigated novel design

tools, methods, and models to guide industry and business

[2,12,13]. Objective tools, such as life cycle assessment [12,14],

the House of Ecology [14], and quality function deployment [14],

are widely included in design processes; so too are strategic

methods, like sustainable product and service development [13].

Designers sometimes equate “environmentally friendly”

products with a fully-sustainable product. This is likely due, in

part, to their training in engineering, in subjects such as material

selection and energy usage. They may also be influenced by the

users’ familiarity with environmental friendliness [13,15], but

unfamiliarity with the broader definition of sustainability. In

addition, environmental friendliness is easily quantified with

existing metrics [2], while manufacturing processes and

economic issues are usually considered beyond the scope of

product design.

The social pillar of sustainable design is somewhat

malleable in its meaning. For example, Cuthill [16] writes that it

involves social policy and community development, while Littig

and Griessler [17] describe it as “a quality of societies that

satisfies an extended set of human needs.” In this study, we focus

on a consumer product for the United States. Therefore, the

social pillar definition(s) that relate more to the developing world

do not apply. Although a product such as a washing machine

does have a societal impact, the U.S. is equipped to handle many

problems with a well-developed community system, such as a

sewer system to process dirty water and electrical grid to allow

for variable electricity usage. In this study, we interpret the social

pillar of sustainability at an extreme local level, namely the user

desirability of the product. An environmentally friendly product

does not benefit the environment if no sales are made, and in

order to sell, user desires must be balanced with costs and

environmental concerns.

The economic pillar of sustainability has been defined as

“cost of supply chain” [18] and “total net profits for a product”

[19]. The full concerns of economic sustainability are too broad

for a designer to consider in a single ideation setting. However,


manufacturing and use costs are a large component of economic

sustainability and are addressable during ideation; thus, we chose

them to represent the economic pillar for this experiment.

2.2 Priming Techniques Psychological priming is a robust method used extensively

in psychology, behavioral economics, and organizational

behavior to activate specific mindsets, thereby making relevant

knowledge more accessible to participants [2,20,21]. Priming

has been shown to affect purchase decisions [20], increase

awareness of a social problem, or change behavior [21]. Priming

has inspired sustainability concerns [2,10], pro-social behavior

[22-24], rudeness toward the experimenter [9], mimicking of

advanced age [25], competitiveness and selfishness [26], and

overcoming of racial stereotypes [27]. Various mental and

physical activities are used as a priming method, including

unscrambling words [28], placing objects [29], creating sensory

collages [2,10], simulating a user scenario [9], writing stories

[29-31], answering questions [9], and playing games [24]. The

priming method functions without the awareness or intent of the

individual, and its effect can last up to 24 hours or more [32].

Priming also works in real-world situations. For example, voters

whose polling places are located in schools are more likely to

support educational propositions on the ballot; this is known as

contextual priming [33].

Unscrambling sentences containing harsh words lead to

participants drawing more hostile features, such as spikes and

claws, in their sketches of hypothetical aliens [28]; answering

questions about nutrition habits improved participants’

productivity and the quality of their ideas on how to improve

health [31]; positive affective priming—participants were shown

a picture of a laughing baby—increased the quality of the ideas

generated in an alternative uses task [34]; mimicking limited

mobility, by wearing gloves, increased participants’ empathy for

elderly users, improved the originality of their concepts, and

alleviated their design fixation [9]; and counterfactual priming—

participants consider events that happened as well as events that

almost happened—increased participants’ performance on the

Drucker candle problem by triggering more consideration of

alternative uses of the given supplies [35].

2.3 Ownership Bias Ownership bias refers to preferring one’s own ideas over the

ideas of others during concept selection in engineering design

[36]. This unintentional tendency potentially introduces

partiality to the decision-making process and challenges team

collaboration. Since idea selection is considered one of the most

crucial steps in engineering design in a collaborative

environment, this bias can have a large impact on the success of

the design process [11].

To understand the causes of bias, behavioral economists

investigated the phenomenon directly and found that the feeling

of ownership originated from the appreciation of solutions

developed by individuals [37] and perception of themselves [38-

39]. Thus, the bias can be affected by personality traits and

gender [39]. To facilitate the idea selection process and eliminate

ownership bias, many systematic selection methods have been

developed, such as the Analytic Hierarchy Process (AHP) [40]

and Pugh method [41]. However, many designers still prefer

informal selection methods to screen a myriad of early-stage

ideas [42]. Based on the explanations of the reasons for

ownership bias, we suspect that changing the mindset of

designers via priming my change the level of ownership bias they

experience when evaluating ideas.

3 RESEARCH PROPOSITIONS AND HYPOTHESES Proposition 1: The collage priming activity can increase the

number of effective ideas per designer from design ideation in

terms of as-rated user desirability, manufacturing/use costs, and

environmental friendliness. To determine if priming successfully

activates one of these mindsets/perspectives, we compare the

number of effective ideas per designer from the primed

conditions with the number of effective ideas per designer of a

control group, and additionally with the number of effective

ideas per designer of a group who engaged in a reading activity

(a conscious engagement with the topic of environmental

sustainability). We hypothesize:

H1(a,b,c,d): Priming for (User Desirability, Cost, Cost,

Environmental priming) results in a larger number of effective

ideas generated by each designer, when judged on (User

desirability, Manufacturing Cost, Use Cost, Environmental

Impact) by novice users, than those generated by the control

condition.

The collage priming activity can also improve the quality of

outcomes of individual ideation by priming. We hypothesize that

priming that focuses on one of three aspects of sustainability

helps designers in general by increasing the originality and

feasibility (two characteristics of quality) of their design ideas

compared to ideas generated by the control group. Section 7.1.2

gives detailed information on the rationale behind the metric

selection.

H1(e,f): Priming of any aspect of sustainability results in a

larger number of effective ideas, when judged on (originality,

feasibility) by novice users than ideas generated by the control

group.

Proposition 2: Priming can reduce ownership bias in

designers for other designers’ ideas. Priming potentially makes

latent information related to stimuli more accessible, not only

improving the designers’ design ability in an active way, but also

potentially changing the designers’ subconscious attitude toward

the targeted perspective and goals of the design process. We

investigated if designers who were primed were more likely to

embrace ideas generated by hypothetical others when evaluating

a mixed set of ideas that included their own ideas.

H2: Collage priming for any aspect of sustainability results

in a higher instance of a designer changing their mind about the

importance of their own ideas when exposed to ideas generated

by hypothetical others, than the control group.


4 EXPERINMENT OVERVIEW In this section we give highlights of the experiment, with

more detail in the following sections. We designed a test-versus-

control experiment to test the hypotheses of Section 3. Figure 1

shows the five conditions: user desirability prime (A1), cost

prime (A2), and environmental prime (A3), reading education

activity (B); and control (C) of no activity (neither priming nor

reading).

FIGURE 2 TESTING CONDITIONS

Figure 2 gives an overview of the experiment’s six major

steps. All activities were performed alone, not in condition

groups. The collage activities assessed coffee cups. The main

design task of the experiment was to design a washing machine.

Explanations for these choices are included in Sections 5.1.1 and

5.3.

I. Pre-ideation Activity: Participants in conditions A did the

collage prime and B did the reading activity. Those in the

control conditions began in Step II. The collage and

reading activities are described in Sections 5.1 and 5.2.

II. Ideation 1: Participants generated design solutions for a

washing machine. Selection of this product is described in

Section 5.3.

III. Video Watching: Participants watched a montage of

videos that highlighted sustainability aspects (user

desirability, manufacturing, and environmental

friendliness) of washing machines. This step was included

to further stimulate design ideas, as described in Section

5.4.

IV. Ideation 2: Participants generated design solutions again

for the same product.

V. Idea Evaluation: Participants first evaluated a set of their

own design ideas. Then they evaluated a mixed set of ideas,

which included the same set of their ideas and a number

of predetermined, sustainability-related ideas, presented

as coming from other designers. The selection of those

ideas and more details are described in Section 5.5.

VI. Questionnaire: All participants filled out a questionnaire

about their demographics, knowledge of sustainable

design, and awareness of characteristics in sustainable

design.

5 ACTIVITY AND STIMULI PREPARATION AND DESCRIPTION

5.1 Collage priming activity In this activity, participants rated their preference for

various types of coffee cups by creating a collage. First, they

physically placed product images on a piece of butcher paper

which already had a set of labeled axes drawn on it, as illustrated

in Figure 3. Second, participants described their selections by

choosing words from a list we provided and writing these words

FIGURE 1 ILLUSTRATION OF EXPERIMENT PROCEDURE


next to the product images. Each word had to be used at least

once, and each cup had to be described by using at least one word.

Two aspects of the activity created the priming condition: 1)

The Y-axis directly related to the aspect of sustainability that was

being primed; for User Desirability we used “boring” to

“delightful,” for Cost we used “inexpensive to make” to

“expensive to make,” and for Environmental Friendliness we

used “low environmental impact” to “high environmental

impact.” 2) Our list of descriptive words were chosen based on

the strength of each word’s association with the concepts “social,”

“economic”, and “environmental” (described further in Section

5.1.3).

FIGURE 3 ILLUSTRATION OF THE COLLAGE PRIMING ACTIVITY

FIGURE 4 EXAMPLE OF COLLAGE FROM THE EXPERIMENT

5.1.1 Selecting a Product for the Collage Activity It was important that the collage activity of experiment Step

I did not include the same product as in experiment Steps II–V.

In other words, participants must be primed using a different

product than the one they will design. This was to ensure that it

was the priming that caused the effects we saw in the ideations,

and not, for example, the effects of assessing/comparing a

variety of washing machines (the focal product of the design

exercises).

After pilot-testing various options, we chose coffee cups as

the product category for collage priming because: coffee cups

are a common product that most participants are familiar with;

they represent a large variety of appearances, prices, and

materials, which demonstrate different levels of user desirability,

cost, and environmental impact; and the coffee cup incorporates

social elements because coffee is a part of many social

experiences. They are also very different from the product in the

main design task, washing machines. We also considered other

products, for example transportation modes (planes, cars), but

post-pilot-test interviews showed that environmental impact and

cost, but not user desirability, were driving assessments even in

the user desirability prime.

5.1.2 Choosing Axes Labels All three collage activities used the same x-axis labels

(“like” / “dislike”) so that participants could cognitively load as

much as they want (e.g., opinion, features, aesthetics) on it. The

y-axis labels varied according to the priming conditions (see

Table 1).

5.1.3 Descriptive Words To determine our list of descriptive words, we conducted a

pilot study in which graduate students at Stanford University first

generated a list of words by consulting design documents and

thesauri. Then they sorted the words into 4 categories (Social,

Economic, Environmental, and Other), and then rated the

strength of a word’s association with that concept. We used the

12 highest-rated words in each category for our lists (Table 1).

In pilot testing, students that were given the task to generate

Economic words produced fewer ideas than other students. The

words in the cost prime could have stifled ideation if they

focused too much on the negative consequences of spending

money; therefore, we re-evaluated this list to make sure none of

the words were too negative.

TABLE 1 AXES LABELS AND DESCRIPTIVE WORDS FOR THE PRIMING COLLAGE ACTIVITY

User

Desirability Cost

Environmental

Friendliness

Horizontal

Axis

Labels

Dislike/Like Dislike/like Dislike/Like

Vertical

Axis

Labels

Boring/

Delightful

Inexpensive/

Expensive to

make

Low/High

environmental

impact

Collage

Words

Sociable

Friendly

Warm

Helpful

Ergonomic

Modern

Unfriendly

Useless

Useful

Solitary

Playful

Stylish

Affordable

Budget

Cheap

Economical

Bargain

Valuable

Inexpensive

Indulgent

Overpriced

Expensive

Rich

Costly

Recyclable

Natural

Efficient

Organic

Conserving

Polluting

Disposable

Synthetic

Inefficient

Wasteful

Reusable

Consumable


5.2 Educational Reading Activity Participants assigned to this condition did not engage in the

collage priming activity but instead read a document about eco-

design drawn from the educational materials of the Sustainable

Minds website [43], with edits. The material focused mostly on

the environmental aspect of sustainability. Our goal was to

compare the effectiveness of the reading activity (B) with the

environmental collage priming (A3) to see if directly engaging

with a conscious, educational activity (which is not considered

“priming”) was more effective than subconsciously activating a

certain mindset (priming).

5.3 Ideation Activity In this activity, all participants were prompted to generate

ideas for a next-generation sustainable washing machine within

a given time. We chose washing machines because they are

common consumer products that are familiar to most designers

and students in the U.S. Washing machines have much room

for improvement in user desirability, cost, and environmental

impact.

The prompt given to designers was “Imagine that you are

working on a product design team at Smithfield Appliances, and

you have been asked to design the next generation of sustainable

washing machine. Product ideas may be related to reducing

water and energy usage, improving manufacturing, and

increasing the product’s appeal to users. Generate ideas that

address this design challenge. You may use scratch paper if

necessary but please type all individual ideas in the boxes below.

You do not have to fill in all of the boxes. You will have 8 minutes

to complete this task.”

We included explicit instructions, e.g. reducing water, to

help trigger participants’ thoughts of sustainability, because She

and MacDonald [2] found that participants perform much better

if more explicit design directions are provided.

5.4 Video Montage

In between Ideation 1 and 2, all participants watched a

montage of videos (available at https://youtu.be/7RKSZxOv2g4)

for approximately 10 minutes. The videos included balanced

mentions of user desirability (a video describing a sleek, modern

washing machine by Samsung with extensive user-centered

features, approximately 4 minutes), environmental impact (two

news segments on highly environmentally-friendly washing

machines, approximately 2.5 minutes), and manufacturing (a

segment from the television show “How It’s Made” on front-

loading washing machines, approximately 4 minutes). This

video montage was added in response to pilot testing, which

showed that participants needed more inspiration to generate an

analyzable number of ideas. Pilot testing honed the video for

balance and demonstrated that watching the video increased the

number of ideas generated in Ideation 2. All participants saw the

same video. It is possible that the collage priming activity

enhanced the effect of the video, allowing primed participants to

obtain more information from the video than non-primed

participants, but this is difficult to specifically test.

5.5 Idea Evaluation This activity occurred after Ideation 2, and its purpose is to

see if priming reduced ownership bias, as articulated in Research

Proposition 2 in Section 3. To design the evaluation activity, we

consulted with an expert on team behavior. First, the activity

creates an anchor by asking participants to rate 5 of their own

ideas (randomly drawn from their previous answers using a

JavaScript in Qualtrics). They classify them into 3 categories:

“definitely want to explore,” “possibly want to explore,” and “do

not want to explore,” as shown in Figure 5. On the next screen,

the participants see these five ideas again, now randomly

interspersed with 12 preselected ideas from hypothetical others.

All participants see the same 12 other ideas, which represent

varying sustainability aspects (user desirability, cost,

environmental impact) and quality characteristics (originality

and feasibility). The participants classified all 17 ideas using the

same categories as before. The analysis below in Section 7.2

checks to see if participants demoted the classification of any of

their five ideas when exposed to the ideas of others.

6 EXPERIMENT PROCEDURE Seventy-three participants were recruited at the

International Design and Technical Conference (IDETC) 2016 in

Charlotte, North Carolina, and on Stanford University’s campus.

To control the experimental environment, we used similar desk

set-ups in both locations. Most of the participants were graduate

students in engineering. Participants were contacted by e-mail

and reminded by a text message. They received $50.00 for

participation. The experiment took on average 82.4 minutes to

complete. Participants were encouraged to sketch or draw on

scratch paper, but only typed responses were processed in this

study; because the quality of drawings might affect the judges’

perception of the quality of the idea [44], we wanted to avoid

such influence.

All activities except the collage priming activity took place

on an iPad, and all data was collected using Qualtrics.

Participants in conditions A1–A3 and B each spent 12 minutes

interacting with the collage or reading material before designing,

while participants in the control condition started with Ideation

I, which lasted for 8 minutes for all participants. Then, all

participants watched the video for 10.5 minutes. Next,

participants had another design session to generate additional

ideas (Step IV, Ideation 2). If a participant generated fewer than

5 ideas total from both sessions, the participant would see a

design prompt for additional encouragement, but in practice,

only one participants required this prompt. Lastly, in Step VI

Questionnaire, participants answered a survey about their

https://youtu.be/7RKSZxOv2g4


demographics, their past experience designing sustainable

products, and their appreciation of various qualities of the

washing machine.

7 DATA AND ANALYSIS We collected responses from 73 participants in three

separate sections: Ideation, Idea Evaluation, and Questionnaire.

In Ideation (Ideation 1 and 2), participants generated 914 design

ideas in total and 670 unique ideas. Because ratings of the

number of effective ideas in Ideation 1 and 2 are not significantly

different for all conditions, all ideas were treated as one data set

for analysis in this paper.

For the Questionnaire (Step VI), participants reported their

previous experience of sustainable products and awareness of

different characteristics of sustainable designs, from extremely

important to not at all important (Figure 6). 49.3% of participants

reported having no experience working on sustainability-related

projects, and 86% of participants reported not being familiar

with the three pillars of sustainability. Figure 6 shows that

environmental impact, cost, and user desirability all were ranked

FIGURE 5 ILLUSTRATION OF THE INTERFACE IN STEP V, IDEA EVALUTION

FIGURE 6 SELF-REPORTED AWARENESS OF CHARACTERISTICS OF SUSTAINABLE DESIGNS, AT END OF EXPERIMENT


among the most important aspects of design. Note that this is

after priming or educational reading, and video exposure.

7.1 Judging Ideas

7.1.1 Preparation and Procedure We first reviewed all ideas generated in Ideation 1 and 2 to

correct typos and grammatical errors, so that the type of

expression and the clarity of the language would not affect the

assessment of ideas. Next, very similar ideas were identified and

grouped to a single idea for rating by the judges. This (1) ensured

that two very similar ideas would not get different ratings; (2)

prevented confusion, should judges be presented with two

similar ideas within their randomly-drawn set; and (3) reduced

the number of ideas from 927 to 670, requiring less judging

work. The cleaned ideas were entered into a Qualtrics survey

designed to present a judge with 15 randomly-drawn ideas for

judging, such that each idea was rated on each metric at least 5

times by 5 separate judges.

Judges were recruited on Amazon Mechanical Turk

(AMT)—a platform that efficiently crowdsources users [45]—

and, in this paper, are referred to as “AMT judges” or simply

“judges.” They had minimum design experience but were

familiar with washing machines. Typical screening procedures

were applied, using a test to qualify judges, and dropping the

responses with response times shorter than 1 standard deviation

of the mean response time (35 respondents). Judges were paid $4

for completing the survey, which took on-average 18 minutes to

complete.

She and MacDonald found that the ratings per designer by

experts and novices aligned with a Pearson correlation

coefficient of 0.7 when evaluating the quality of the design ideas

[2]. In addition, novice judges who passed the screening were

expected to understand rating criteria and to be qualified to give

rating scores. Additionally, using professional judges on 670

features brings its own issues related to internal consistency and

judgment fatigue. Therefore, only AMT judges were recruited in

this study.

The rating survey consisted of two sections: one for judging

ideas related to aspects of sustainability (user desirability,

manufacturing cost, use cost, and environmental impact) and

another section for ideas related to characteristics of intrinsic

quality (originality and feasibility). In each section, AMT judges

received a task overview and then training about the rating

categories, including score descriptions, example ideas, and

corresponding rating scores (see Table 2). Judges first took

quizzes to test if they paid attention to the training, and those

who passed rated 15 randomly ordered ideas in each section, for

a total of 30 ideas.

7.1.2 Idea Metrics The rating scales used to judge the effectiveness of User

Desirability, Manufacturing Cost, Use Cost, and Environmental

Impact are shown in Table 2. Cost was broken into

“manufacturing cost” and “use cost” based on pilot judging,

which indicated a confusion in rating cost as a whole. Separately,

we measured originality and feasibility to assess the intrinsic

quality of a design. A global definition of design quality does not

exist, and research yielded different definitions and metrics

under different circumstances [46]: Shal et al. [47-48] proposed

four effectiveness measures: quality, quantity, novelty, and

variety; Kudrowitz et al. suggested three attributes to measure

early-stage product ideas: novelty, usefulness, and feasibility

[36]. In this study, it is appropriate to adopt a subset of the

existing metrics based on the hypotheses at hand, and originality

and feasibility were chosen to represent how well a design

expands the design space. These metrics are discussed further in

[36,47-48], on which we based our rating scale. All ratings were

on a scale of 1 to 5, with 5 being the most positive rating. This

5-point scale is adopted from the previous study by She and

MacDonald [2] for the comparability of the results.

7.1.3 Judging Results for Ideation 73 participants generated 914 design features, including

non-unique ones. Table 3 shows the total number of ideas

generated per experimental condition and the average number of

ideas generated per participant. Ratings of ideas were averaged

across judges; Table 4 summarizes the average rating scores

across each condition for all rating categories, and Table 5

summarizes the median rating scores for all rating categories.

To test H1(a,b,c,d), we counted the number of above-

median-rating ideas generated by each designer for each

sustainability-related rating category. For brevity we call these

above-median ideas “effective ideas.” We compared the average

count of effective ideas per designer for each experimental

condition against the average count in the control.

Figure 7 plots the average count of effective ideas generated

per designer across all conditions and rating categories, and

Table 6 gives the statistical details.

To determine statistical significance, we used the Kruskal-

Wallis test [49] for comparison of two non-normally distributed

samples from the sample distribution. This test is similar to one-

way ANOVA but is used when the measurement data does not

meet the normality assumption or has a small size. Results:

• Manufacturing Cost (Fig. 7b): The mean value of the

number of effective ideas generated by designers in the

reading activity condition is significantly higher than that of

the control condition (8.29 vs. 4.80, p = 0.05*).

• Environmental Impact (Fig. 7d): The mean values of the

number of effective ideas generated by designers in the cost

and in the environmental collage priming conditions are

significantly higher than the control (cost: 7.07 vs. 4.27, p =

0.007*; environmental: 6.13 vs. 4.27, p = 0.02**). The mean

value of numbers of effective ideas generated by designers

in the reading activity condition (B) is significantly higher

than that of the control condition (7.57 vs 4.27, p = 0.05*).

• In general, the average count of effective “user desirability”

and “use cost” ideas generated per designer in three collage

priming conditions and the reading activity condition are

higher than the average count of effective ideas in the

control condition, but the differences are not significant.

To test H1(e,f), we counted the number of effective (above-

median-rating) ideas generated by each designer for each


quality-related rating category. We compared the average count

of effective ideas per designer for each experimental condition

against the average count in the control, again using Kruskal-

Wallis. No significant effects were found, although “feasibility”

average count was higher for all conditions versus the control.

TABLE 2 RATING CATEGORIES AND DESCRIPTIONS OF SCORES 5, 3, AND 1

Rating

Category

Description of

Score 5

Description of

Score 3

Description of

Score 1

Use

r

Des

irab

ilit

y Desired by a

reasonable

number of

customers;

pleasing;

attractive

Okay to have;

somewhat

desired by

customers

Unpleasant or

should be

avoided

Man

ufa

ctu

rin

g

Cost

Manufactured

with feasible

process and

cost saving

Neither

decreasing nor

increasing

process time or

manufacturing

cost compared

to the current

design

Manufactured

with high cost

or tremendous

amount of

effort

Use

Cost

Operated at a

lower cost or

helpful at

reducing the

cost of using

(electricity, gas,

water etc.)

compared to

the

conventional

design

Having no

significant

effect on the

cost of using,

compared to

the

conventional

design

Operated at a

higher cost or

increasing the

cost of using,

compared to

the

conventional

design

En

vir

on

men

tal

Imp

act

Good for the

environment

Neutral for the

environment

Bad for the

environment

Orig

inali

ty

Not expressed

before and

ingenious

Not typical,

and show some

imagination

Common and

boring

Feasi

bil

ity

Easy to

implement

without major

changes or

violation of

known

constraints

(financially and

physically)

Could be

implemented

with minor

changes to

existing

conditions

Very hard to

implement

given the

existing

conditions

and/or

requiring

significant

research and

development

TABLE 3 NUMBER OF IDEAS GENERATED PER CONDITION AND AVERAGE PER DESIGNER

Condition

Number of

participants

Total number

of ideas

generated

(including non-

unique ones)

Average number

of ideas

generated per

participant

User

desirability

prime 13 159 12.23

Cost prime 15 195 13.00

Environmental

prime 16 191 11.93

Reading 14 204 14.57

Control 15 165 11.00

TOTAL 73 914 12.50

TABLE 4 MEAN SCORE (AND STANDARD DEVIATION) OF RATING

CATEGORIES PER CONDITION

Con

dit

ion

Rating Category

User

desira

bility

Manuf

acturin

g cost

Use

cost

Enviro

nment

al

impact

Origin

ality

Feasibi

lity

User

desirabi

lity

prime

3.64 2.50 3.42 3.82 2.95 3.47

(0.74) (0.57) (0.52) (0.66) (0.88) (0.79)

Cost

prime

3.57 2.54 3.44 3.90 3.03 3.34

(0.75) (0.65) (0.55) (0.71) (0.88) (0.77)

Environ

mental

prime

3.55 2.58 3.46 3.86 2.86 3.470

(0.73) (0.62) 0.561 0.72 0.86 0.782

Reading 3.38 2.67 3.42 3.85 2.95 3.33

(0.80) (0.69) (0.53) (0.68) (0.84) (0.81)

Control 3.59 2.54 3.40 3.69 3.08 3.25

(0.78) (0.59) (0.53) (0.71) (0.80) (0.81)

Grand 3.54 2.57 3.43 3.83 2.97 3.37

(0.76) (0.63) (0.54) (0.70) (0.86) (0.80)


TABLE 5 MEDIAN RATING SCORES FOR EACH CATEGORY

User

desirability

Manufacturing

cost Use cost

Environmental

impact Originality Feasibility

3.66 2.52 3.32 3.89 3.03 3.40

TABLE 6 AVERAGE NUMBER OF EFFECTIVE IDEAS GENERATED BY EACH DESIGNER (An “effective idea” is one

that was rated above the median score of that rating category; **p<0.05 and *p<0.1.)

Rating Category

User

desirability

Manufacturing

Cost Use Cost

Environmental

Impact Originality Feasibility

Con

dit

ion

s

User

Desirability

Prime

6.92 5.77 5.77 6.00 6.31 6.69

Cost Prime 6.80 6.27 6.73 7.07 *

(p=0.07) 6.80

5.93

Environmental

Prime

6.13

6.13 6.31

6.31 **

(p=0.02) 5.38 6.31

Reading

Activity 6.00

8.29 **

(p=0.04)

7.21

7.57 **

(p=0.04)

6.71

7.00

Control 5.53 4.80 5.27 4.27 6.20 4.93

FIGURE 7 COUNT OF EFFECTIVE IDEAS GENERATED BY EACH DESIGNER ACROSS ALL CONDITIONS (An “effective idea” is one that was rated above the median score of that rating category; error bars show the standard deviations.)


7.2 Idea Evaluation

This analysis tests Proposition 2: It is possible to reduce

ownership bias of designers by using the collage prime.

Specifically, it investigates if designers re-classify the

importance of further investigating their own ideas when also

asked about the importance of other designers’ ideas, see Section

5.5.

The analysis focuses on the classification exercise of the five

randomly-drawn ideas from the design participants responses in

the ideation phase of the experiment. This mimics idea selection

in a collaborative environment, which is one of the crucial

components in the design process. The classification categories

were “Definitely,” “Possibly,” and “Do Not” want to explore

(this idea further), and a fourth classification, “Untouched,” of

not dragging the idea into any of the three boxes. The participants

were asked to evaluate these ideas twice, first alone, which we

call evaluation section ES1, and then in the presence of 12 ideas

from hypothetical other designers, which we call ES2.

7.2.1 Binomial Proportion Test The proportion of changing the classifications of

participants’ own ideas from ES1 to ES2 for each testing

condition is quantified as Eq. 1.

𝑝𝑐ℎ𝑎𝑛𝑔𝑒,𝑖 = ∑ 𝐶𝑜𝑢𝑛𝑡 𝑜𝑓 𝑅𝑒𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑒𝑑 𝐼𝑑𝑒𝑎𝑠𝑖 𝑖𝑛 𝐸𝑆2

∑ 𝐶𝑜𝑢𝑛𝑡 𝑜𝑓 𝐼𝑑𝑒𝑎𝑠 𝑃𝑙𝑎𝑐𝑒𝑑 𝑖𝑛 𝐵𝑜𝑥𝑒𝑠 𝑖𝑛 𝐸𝑆1𝑖 (1)

Table 7 shows the proportions of changing for all conditions

and the p-values when comparing each value of the experimental

groups (A1–A3, B) to the control group by binomial proportion

test. The cost collage prime (A2) has a dramatic and statistically

significant effect, with 40% of classified ideas changing

positions. The other conditions have no significance, although all

are greater than the control.

TABLE 7 PROPORTIONS OF RECLASSICATIONS

Condition

ES2: Count of

reclassified ideas

ES1: Count of Ideas

Placed in Boxes

% Reclassified

User

Desirability 17 63 27% (p=0.28)

Cost 30 75 40% (p =2e-4*)

Environment 20 71 28% (p=0.15)

Reading 20 68 29% (p=0.10)

Control 15 71 21% -

7.2.2 Wilcoxon Signed Rank Test To explore the change of classifications at the individual

level from ES1 to ES2, for example, an idea was classified as

“possibly want to explore” in ES1 and “definitely want to

explore” in ES2, the non-parametric Wilcoxon signed rank test

was used to compare the paired ES1-to-ES2 classification results

without and with the predetermined ideas for each condition.

Specifically, higher rank indicates the direction of changes is

towards higher importance.

TABLE 8 SIGNED-RANKS BETWEEN ES1 AND ES2

Condition Test statistics

User Desirability 36 (p = 0.03*)

Cost 30 (p = 0.09*)

Environment 140 (p = 0.20)

Reading 100 (p = 0.9)

Control 70 (p = 0.5)

Table 8 shows the test statistics along with the p-values for

five conditions by the Wilcoxon signed rank test. The results

show that both user desirability collage priming and cost collage

priming have a significant effect on encouraging designers to

classify their own ideas as less important and potentially to

explore others’ ideas more in a collaborative environment.

8 DISCUSSION The experiment demonstrates that exposing designers to a

collage priming activity associated with sustainable design (user

desirability, economic competitiveness, environmental

friendliness) is partially effective in generating effective ideas

and in reducing ownership bias. A comparison reading activity is

just as or more effective than the priming activities in generating

effective environmentally friendly ideas and manufacturing cost

ideas, but not effective in reducing ownership bias. This suggests

that priming is an effective way to produce environmentally

friendly design ideas while keeping an open mind to ideas of

others. Unfortunately, priming did not provide a method for

activating the less-readily-accessible pillars of sustainability—

social/user desirability and economic competitiveness—during

ideation. Reading material on sustainable design was effective in

producing a greater number of effective manufacturing cost

ideas, but not in reducing ownership bias in decision evaluation,

compared to control.

There were 670 unique ideas generated—a huge amount—

so we used novice AMT judges with limited in-survey training,

and averaged multiple evaluations from separate judges for the

statistical testing of effectiveness of ideas. As the AMT judges

were imperfect and inexperienced raters, we defined an effective

idea as one that was ranked higher than the median rating for

each rating category, instead of relying on a pure rating of greater

than 3 to determine effectiveness, as we may have with expert

judges. While it can be argued that using novice judges

diminishes the findings of the study, as compared to using expert

judges, the argument can also be made that as a product’s

sustainability and quality will be judged in the end by novice

users, it makes sense to use them for research judgments as well.

In previous research, we found good alignment, with a Pearson

correlation of 0.7 [2], between expert judges and AMT judges on

a similar rating task.

The effectiveness of the reading education activity is

notable. In terms of pure numbers, not with significance, this

condition had the most ideas generated per designer, and had the

highest average number of effective ideas per design for

Manufacturing Cost, Use Cost, Environmental Impact, and


Feasibility—four out of six rated categories. This suggests that

the student design participants, on the whole, did not have much

latent content in their brains that could be activated by priming

to address sustainable design. The fact that 63 out of 73 designers

had no concept of the three pillars of sustainability supports this

explanation. Instead, providing basic sustainable design

education was more effective. We did not combine the reading

activity with a collage priming activity, but this could have been

even more effective. Possibly, priming more-experienced

designers, who are accustomed to addressing cost and user

desirability concerns, would have been more effective than basic

education (which they would already have). Or a better approach

might be to combine a priming activity and an education activity

for student designers.

Our specific hypotheses produced mixed results. The

collage priming activity that was associated with the

environmental pillar significantly increased the number of

above-median-rating (effective) ideas generated per designer

when judged on “environmental impact,” which supports

hypothesis H1d. This also supports the thoughts of the paragraph

above, as it is likely that of the three pillars, student designers

have the most experience with environmental impact design,

such as reducing energy or resources used during manufacture

and/or use of the product. The results also show that the collage

priming activity associated with cost and the reading education

activity also resulted in a significant increase in the number of

effective “environmental impact” ideas. Reducing cost is closely

linked with reducing resource use and consumption, so this is

logical.

Hypotheses H1a, b, and c are rejected; that is, the collage

priming activity associated with user desirability had no

significant effect on increasing the average number of effective

“user desirability” ideas, and the collage priming activity

associated with cost had no significant effect on increasing the

average number of effective “manufacturing cost” and “use cost”

ideas.

Hypotheses H1e and f are also rejected. There were no

significant effects on improving quality, as measured by

feasibility and originality. While other priming activities have

been demonstrated to increase novelty, the activities in this

experiment were ineffective.

Hypotheses H2 tested if priming activities could lead to

more receptivity and less ownership bias when evaluating one’s

own ideas in the context of others’ design ideas. It is supported

for the Cost priming activity by a binomial proportion test and

Wilcoxon Signed Rank test, and for the User Desirability

priming activity by only the Wilcoxon Signed Rank test. The

Cost priming resulted in a substantial (40%) reclassification of

the importance of further investigating one’s own ideas when

exposed to the ideas of others. H2 is not supported for the

Environmental collage priming activity. It may be that this

prime, instead of underscoring ownership bias, triggers social

desirability bias, or doing what is morally right, and creates an

overemphasis on the importance of environmental impact as

compared to the other two pillars of sustainability. Thus, when a

variety of ideas are presented, designers stick to their own

environmentally-oriented ideas in favor of ideas that address the

other pillars of sustainability. The reading activity also did not

reduce ownership bias.

This study has several limitations. The first is that the study

used student designers instead of practicing designers. Students

have less experience and education with the broader concerns of

engineering, and thus less latent mental content can be triggered

by sustainability primes. The success of the education activity

supports this limitation. Ideation outcomes were assessed by

multiple online judges, who are potential users of the focal

product, but who may have very different purchase habits and

life goals from each other. This setting reflects the situation of

sustainable products in the real market; yet the individual

differences between judges is not taken into account in this study.

Expert judgments may yield different results, although a past

study indicated a good correlation between expert and novice

judges. In addition, participants in the priming conditions

performed activities, either physically placing pictures or

reading a document, while participants in the control condition

directly started to ideate. The acceptance of others’ ideas was

only performed with simulated other designers. There were too

many possible factors, such as personality fit, familiarity,

language differences, variability in creativity and

communication, etc., that were not controlled for, to get an

accurate read on change in ownership bias in a real team setting.

Also, the participants in our study were unable to do design

research; perhaps in a more involved exercise, with a washing

machine on-hand, users to interview, and Life Cycle Assessment

analysis available, the priming activities could have triggered

design actions, which may have led to even better ideation

sessions.

There are also a number of potential sources of technical

error in the study. For example, the data is not normally

distributed, and although the Kruskal-Wallis test was used to

compensate for this, it can still result in incorrect predictions.

The sample size was small, at approximately 15 designers per

condition. Doubling this number could improve the distribution

of results and also the robustness of the tests. Another potential

source of error is that only one design task was performed on one

target product. Testing multiple products would increase the

assuredness of the findings. While the reliability of AMT judges

has been proven trustworthy, if automatic assessments where

recorded instead of those of actual novice users, this could also

impact the results.

9 CONCLUSION AND FUTURE WORK The experiment further illustrates the difficulty designers

have when tackling social and economic aspects in ideation

processes. As stated in the introduction, designers naturally

gravitate toward the environmental aspect in the early stage of

sustainable design, even though social and economic aspects are

also essential for sustainable products to succeed in the market.

This issue is well demonstrated in this experiment: designers

who were exposed to the user desirability prime did not improve

significantly on creating a greater number of effective user-

desirable ideas, and designers who were exposed to the cost


prime did not generate a greater number of effective designs that

improved manufacturing cost or use cost of the product, as

judged by novice users. As mentioned above, this is likely due to

a lack of education on these participants—without embedded

knowledge in the brain to “activate,” priming cannot work.

A good follow-up study would be to test the effectiveness of

priming activities in combination with education activities.

Questions of the order of the two activities, and the best pairings

of types of activities should be explored. Effort to expose student

designers to broader design concerns, such as manufacturing,

marketing, and human-centered design, must continue in order

to create available mental content that can be accessed during

ideation. Life Cycle Assessment and other environmentally

motivated engineering training is important, but without design

attention to the other two pillars of sustainability, the product will

not be profitable and popular with users and may ultimately fail

in the market and in use.

Overall, the study contributes to growing evidence that

priming is an effective tool to improve ideation, with mixed

results for the experiment at-hand. We demonstrate that collage

priming activities can help designers generate a greater number

of effective ideas that address the environmental pillar of

sustainability, but not that address cost (representing the

economic pillar) or user desirability (representing the social

pillar). We believe lack of education and experience is to blame

for the ineffectiveness of the priming activities for these pillars.

A reading education activity in this experiment generated a larger

number of effective sustainable design ideas but did not reduce

ownership bias. The Economic and Social pillar collage priming

activities are effective in reducing ownership bias to others’ ideas

when evaluating design ideas to pursue further, but the

Environmental collage priming activity may trigger a social bias

for environmentally driven ideas and enhance ownership bias.

Combining these two findings, the Economic priming activity

was the most effect prime overall, although not in the manner

that we predicted: it was not effective toward the economic pillar

of sustainability, but rather the environmental one, and also

showed evidence of reducing ownership bias over the ideas of

other designers.

ACKNOWLEDGMENTS This work was funded by the Hasso Plattner Design

Thinking Research Program. We thank Hasso Plattner Insitut for

supporting this project. We thank Ufuoma Ovienmhada for her

assistance in data collection. The premise of this study was

conceived with Carolyn Conner Seepersad at UT Austin. We

thank Dylan Moore and Professor Pamela Hinds for their

thoughtful contributions to the design of the experiment.

REFERENCES [1] She, J., Seepersad, C. C., Holtta-Otto, K., & MacDonald, E.

F., 2018, “Priming Designers Leads to Prime Designs,”

Design Thinking Research, pp. 251-273, Springer, Cham.

[2] She, J., & MacDonald, E., 2014, “Priming Designers To

Communicate Sustainability,” ASME J. Mech. Des., 136(1),

p. 011001.

[3] She, J., & MacDonald, E. F., 2018, “Exploring the Effects of a

Product’s Sustainability Triggers on Pro-environmental

Decision-making,” ASME J. Mech. Des., 140(1), p. 011102.

[4] Keeble, B. R., 1988, “The Brundtland Report: ‘Our Common

Future’,” Medicine and War, 4(1), pp. 17–25.

[5] Cuthill, M., 2009, “Strengthening the ‘Social’ in Sustainable

Development: Developing A Conceptual Framework for Social

Sustainability and a Rapid Urban Growth Region an

Australia,” Sustain. Dev., 18(6), pp. 362–373.

[6] Littig, B., & Griessler, E., 2005, “Social Sustainability: A

Catchword Between Political Pragmatism and Social Theory,”

Inter. J. Sustain. Dev., 8(1-2), pp. 65–79.

[7] Friedman, R. S., Fishbach, A., Forster, J., & Werth, L., 2003,

“Attentional Priming Effects On Creativity,” Creat. Res. J.,

15(2–3), pp. 277–286.

[8] Bruner, J. S., 1957, “On Perceptual Readiness,” Psychol.

Rev., 64(2), p. 123.

[9] Bargh, J. A., Chen, M., and Burrows, L., 1996, “Automaticity

Of Social Behavior: Direct Effects Of Trait Construct And

Stereotype Activation On Action,” J. Pers. Soc. Psychol.,

71(2), pp. 230–244.

[10] Guyton, A. A., 2006, “Developing Sustainable Product

Semantics for Consumer Products: A Sustainable Designer’s

Guide,” Doctoral dissertation, Georgia Institute of

Technology.

[11] Onarheim, B., and Christensen, B. T., 2012, "Distributed

idea screening in stage-gate development processes," J. Eng.

Des., 23(9), pp. 660-673.

[12] Ramani, K., Ramanujan, D., Bernstein, W. Z., Zhao, F.,

Sutherland, J., Handwerker, C., Choi, J. K., Kim, H., and

Thurston, D., 2010, “Integrated Sustainable Life Cycle

Design: A Review,” ASME J. Mech. Des., 132(9), p. 091004.

[13] Maxwell, D., & Van der Vorst, R., 2003, “Developing

Sustainable Products And Services,” J. Cleaner Prod., 11(8),

pp. 883-895.

[14] Halog, A., 2001, “Using Quality Function Deployment for

Technique Selection for Optimum Environmental

Performance Improvement,” J. Cleaner Prod., 9, pp. 387–394.

[15] Grunert, K. G., Hieke, S., & Wills, J., 2014, “Sustainability

Labels On Food Products: Consumer Motivation,

Understanding And Use,” Food Policy, 44, pp. 177–189.

[16] Cuthill, M., 2009, “Strengthening the ‘Social’ in Sustainable

Development: Developing A Conceptual Framework for Social

Sustainability and a Rapid Urban Growth Region an

Australia,” Sustain. Dev., 18(6), pp. 362–373.

[17] Littig, B., & Griessler, E., 2005, “Social Sustainability: A

Catchword Between Political Pragmatism and Social Theory,”

Inter. J. Sustain. Dev., 8(1-2), pp. 65–79.

[18] Mota, B., Gomes, M. I., Carvalho, A., & Barbosa-Povoa, A. P.,

2015, “Towards Supply Chain Sustainability: Economic,

Environmental and Social Design and Planning,” J. Clean.

Prod., 105, pp. 14-27.

[19] United Nations, Department of Economic., 2007, “Indicators

of Sustainable Development: Guidelines and Methodologies,”

United Nations Publications.

[20] Mandel, N., and Johnson, E. J., 2002, “When Web Pages

Influence Choice: Effects of Visual Primes on Experts and

Novices,” J. Consum. Res., 29(2), pp. 235–245.

[21] Stapel, D. A., 2011, “Priming as Proxy: Understanding the

Subjectivity of Social Life,” Cognitive Methods in Social

Psychology, pp. 148–183.

[22] Shariff, A. F., and Norenzayan, A., 2007, “God Is Watching


You: Priming God Concepts Increases Prosocial Behavior In

An Anonymous Economic Game,” Psychol. Sci., 18(9), pp.

803–809.

[23] Pfeffer, J., and DeVoe, S. E., 2009, “Economic Evaluation:

The Effect Of Money And Economics On Attitudes About

Volunteering,” J. Econ. Psychol., 30(3), pp. 500–508.

[24] Martin, L. J., Hathaway, G., Isbester, K., Mirali, S., Acland,

E. L., Niederstrasser, N., Slepian, P. M., Trost, Z., Bartz, J. A.,

Sapolsky, R. M., Sternberg, W. F., Levitin, D. J., and Mogil, J.

S., 2015, “Reducing Social Stress Elicits Emotional

Contagion Of Pain In Mouse And Human Strangers,” Curr.

Biol., 25(3), pp. 326–332.

[25] Bargh, J. A., Chen, M., and Burrows, L., 1996, “Automaticity

Of Social Behavior: Direct Effects Of Trait Construct And

Stereotype Activation On Action,” J. Pers. Soc. Psychol.,

71(2), pp. 230–244.

[26] Kay, A. C., Wheeler, S. C., Bargh, J. A., and Ross, L., 2004,

“Material Priming: The Influence Of Mundane Physical

Objects on Situational Construal and Competitive Behavioral

Choice,” Organ. Behav. Hum. Decis. Process., 95(1), pp. 83–

96.

[27] Sassenberg, K., and Moskowitz, G. B., 2005, “Don’t

Stereotype, Think Different! Overcoming Automatic

Stereotype Activation By Mindset Priming,” J. Exp. Soc.

Psychol., 41(5), pp. 506–514.

[28] Marsh, R. L., Bink, M. L., and Hicks, J. L., 1999,

“Conceptual Priming In A Generative Problem-Solving Task,”

Mem. Cognit., 27(2), pp. 355–363.

[29] Reed, A., Aquino, K., Levy, E., and Ii, R., 2007, “Moral

Identity and Judgments of Charitable Behaviors,” J. Mark.,

71(1), pp. 178–193.

[30] Molinsky, A. L., Grant, A. M., and Margolis, J. D., 2012,

“The Bedside Manner Of Homo Economicus: How And Why

Priming An Economic Schema Reduces Compassion,” Organ.

Behav. Hum. Decis. Process., 119(1), pp. 27–37.

[31] Rietzschel, E. F., Nijstad, B. A., and Stroebe, W., 2007,

“Relative Accessibility Of Domain Knowledge and

Creativity: The Effects Of Knowledge Activation On The

Quantity And Originality Of Generated Ideas,” J. Exp. Soc.

Psychol., 43(6), pp. 933–946.

[32] Srull, T. K., and Wyer, R. S., 1979, “The Role Of Category

Accessibility In The Interpretation Of Information About

Persons: Some Determinants And Implications,” J. Pers. Soc.

Psychol., 37(10), pp. 1660–1672.

[33] Berger, J., Meredith, M., and Wheeler, S. C., 2008,

“Contextual Priming: Where People Vote Affects How They

Vote,” Proc. Natl. Acad. Sci. U. S. A., 105(26), pp. 8846–

8849.

[34] Lewis, S., Dontcheva, M., and Gerber, E., 201, May,

“Affective Computational Priming And Creativity,” In

Proceedings of the SIGCHI Conference on Human Factors in

Computing Systems (pp. 735-744). ACM.

[35] Galinsky, A. D., and Moskowitz, G. B., 2000,

“Counterfactuals as Behavioral Primes: Priming the

Simulation Heuristic and Consideration of Alternatives,” J.

Exp. Soc. Psychol., 36(4), pp. 384–409.

[36] Kudrowitz, B. M., and Wallace, D., 2013, “Assessing the

quality of ideas from prolific, early-stage product ideation,” J.

Eng. Design, 24(2), pp. 120-139.

[37] Kahneman, D., Knetsch, J. L., and Thaler, R. H., 1990,

"Experimental Tests of the Endowment Effect and the

Coase Theorem," J. Political Econ., 98(6), pp.

1325-1348.

[38] Alicke, M. D., 1985, "Global Self-Evaluation as

Determined by the Desirability and Controllability of Trait

Adjectives," J. Pers. Soc. Psychol., 49(6), pp. 1621-1630.

[39] Toh, C. A., Patel, A. H., Strohmetz, A. A., and Miller, S. R.,

2015, “My Idea Is Best! Ownership Bias and its Influence on

Engineering Concept Selection,” ASME 2015 IDETC/CIE,

Aug. 2-5, Boston, MA.

[40] Marsh, E. R., Slocum, A. H., and Otto, K. N., 1993,

"Hierarchical decision making in machine design," MIT

Precision Engineering Research Center.

[41] Pugh, S., 1991, Total design: integrated methods for

successful product engineering, Addison-Wesley,

Wokingham.

[42] López-Mesa, B., and Bylund, N., 2011, "A study of the use

of concept selection methods from inside a company," Res.

Eng. Des., 22(1), pp. 7-27.

[43] “Ecodesign overview” [Online]. Available:

https://app.sustainableminds.com/learning-center/ecodesign-

and-lca/ecodesign-overview desirability.

[44] Yang, M. C., 2009, “Observations on concept generation and

sketching in engineering design,” Res. Eng. Des., 20(1), pp.

1-11.

[45] Kittur, A., Chi, E. H., and Suh, B., 2008, “Crowdsourcing

User Studies with Mechanical Turk,” CHI 2008, Apr. 5–10,

Florence, Italy.

[46] Reeves, C. A., and Bednar, D. A., 1994, “Defining quality:

alternatives and implications,” Acad. Manag. Rev., 19(3), pp.

419-445.

[47] Shah, J. J., Smith, S. M., and Vargas-Hernandez, N., 2003,

“Metrics for Measuring Ideation Effectiveness,” Des. Stud.,

24(2), pp. 111-134.

[48] Shah, J. J., Kulkarni, S. V., and Vargas-Hernandez, N., 2000,

“Evaluation of Idea Generation Methods for Conceptual

Design: Effectiveness Metrics and Design of Experiments,”

ASME J. Mech. Des., 122(4), pp. 377-384.

[49] Kruskal, W. H., & Wallis, W. A., 1952, “Use of Ranks in One-

Criterion Variance Analysis,” J. Am. Stat. Assoc., 47(260), pp.

583-621.

[50] Howell, D. C., 2012, “Statistical methods for psychology,” pp.

127-131, Cengage Learning.

[51] Sidney, S., 1957, “Nonparametric Statistics for The Behavioral

Sciences,” J. Nerv. Ment. Dis., 125(3), p. 497.

Proceedings of the ASME 2018 International Design ...167.99.111.205/wp-content/uploads/2018/12/DETC2018-86291.pdf · 2016 IDETC conference and graduate engineering students at Stanford.

Documents