The transition to a responsive website: A user study1040865/FULLTEXT01.pdf · Responsive design Web design used to be a simpler task with only a few screen sizes to have in mind while

IN DEGREE PROJECT MEDIA TECHNOLOGY,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2016

The transition to a responsive website: A user study

JOSEFINE LINDQVIST

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF COMPUTER SCIENCE AND COMMUNICATION

ABSTACT

An anonymous company has been redesigning its website to a responsive site and during this process the users have the

option to stay on either site. This study examined the reasons as to why users stay on the non-responsive site and how the

responsive site could be improved to make them switch to it. In order to understand how users were effected by first

impressions and age as well as how adaption to technology works, previous research in these areas have been presented. A

survey and user test were conducted to find the answers to the research question.

Because the transformation to a responsive site meant that the visual structure changed, it affected the users’ existing habits.

The results and conclusion were that there were three major groups stating different main reasons on what would make them

go to the responsive site. The strategy for making users on the non-responsive site more satisfied with the responsive site

would be to first implement missing functionality. The next step would be to invite all users and adding guides to help them

to learn how to find what they usually look for.

SAMMMANFATTNING

Ett anonymt företag har arbetat med att designa om sin hemsida till att bli responsiv och under den här processen har

användarna haft möjlighet att stanna på båda websidorna. Den här studien undersökte varför vissa användare stannade på den

icke responsiva sidan och hur den responsiva sidan kan förbättras för att få dem att byta till den. För att förstå hur användare

påverkades av första intryck och ålder samt hur anpassning till teknik fungerar har tidigare forskning inom dessa områden

presenterats. En enkät och användartest utfördes för att svara på frågeställningen.

Eftersom en övergång till en responsiv sida innebar att den visuella strukturen ändrades påverkades användares redan

existerande vanor. Resultatet och slutsatsen var att det fanns tre huvudgrupper som angav olika anledningar till vad som

skulle få dem att gå över till den responsiva sidan. Strategin för att få användarna på den icke responsiva sidan mer nöjda

med den responsiva sidan skulle vara att först implementera de funktioner som saknas. Nästa steg skulle vara att bjuda in alla

användare och att skapa guider för att hjälpa dem att lära sig hur de hittar det de oftast letar efter.

The transition to a responsive website: A user study Josefine Lindqvist

Royal Institute of Technology

Stockholm, Sweden

[email protected]

ABSTRACT

An anonymous company has been redesigning its website

to a responsive site and during this process the users have

the option to stay on either site. This study examined the

reasons as to why users stay on the non-responsive site and

how the responsive site could be improved to make them

switch to it. In order to understand how users were effected

by first impressions and age as well as how adaption to

technology works, previous research in these areas have

been presented. A survey and user test were conducted to

find the answers to the research question.

Because the transformation to a responsive site meant that

the visual structure changed, it affected the users’ existing

habits. The results and conclusion were that there were

three major groups stating different main reasons on what

would make them go to the responsive site. The strategy for

making users on the non-responsive site more satisfied with

the responsive site would be to first implement missing

functionality. The next step would be to invite all users and

adding guides to help them to learn how to find what they

usually look for.

Author Keywords

User study; user satisfaction; HCI; website redesign;

technological adaption

INTRODUCTION

Responsive design

Web design used to be a simpler task with only a few

screen sizes to have in mind while developing, but over

time it has become more complicated. Today, there are

multiple different screen sizes and devices to adjust a

website to. According to Fox [11] responsive design can be

used to counter this problem, which means that the website

adjusts the content based on the resolution of the screen.

This approach makes websites fit all kinds of screen sizes,

even on tablets and phones.

Glassman and Shen [8] explains that responsive design also

highlights priorities when designing a website. Instead of

doing a fully formed website developed for desktop

practices, this approach begins with finding the most

essential part of a website. This is to fit content and services

to a small internet accessible device, such as phone or

tablet. This approach makes the content contain core

functionality in its simplest form, while scaled up versions

could have extended features. With this, heavy or broad

content that is non-essential can be avoided on mobile

devices but could be enabled on desktop to give a richer

user experience. [8]

The project

The anonymous company’s product is a Swedish online

marketplace available on both desktop and mobile devices.

The users are the ones selling and buying on the site and the

sellers can offer both auctions and fixed priced items. Each

week one million unique users visit the site to browse, bid,

buy or sell items online. Not only private customers access

this site but companies also use it to drive their businesses.

The age range of the user base varies with active users from

18 years old to around 85 years old.

The company has worked on transforming their website to a

responsive site, including new design and features. The

reason behind this redesign was mainly because the mobile

traffic has increased in recent years and because the website

was designed for desktops they were not well displayed in a

mobile phone. This responsive site has been in development

for some time. On mobile devices this version of the site is

available to everyone. However, on desktop the site is

currently divided into two parts: My account and the rest of

the site. At My account the members can for example deal

with their bids, bought items and sold items. At the rest of

the site the member can for example search for items and

upload new auctions to sell. Because the responsive version

is currently in development, only half of the users on

desktop can see the option to go to the responsive version

of My account while the rest of the site is accessible to

everyone. Except these two parts, the old non-responsive

site is accessible as well and the desktop users have the

option to stay on the non-responsive site during the

transition. The different version can be seen in Figures 1-4.

Even though there are only a few functions missing on the

responsive site, around 10% of the users who can see the

link are choosing to stay on the non-responsive site. This

study explored why they are staying there and how the

responsive site can be improved to make them switch to it.

To do this, the field of Human-Computer Interaction (HCI)

will be in the center. The HCI area focuses on

understanding what creates a good experience for the users

and how products can become more user-friendly [10]. To

be user-friendly, the users must feel that a product is useful

Figure 1: This figure shows how My account looks at the

non-responsive site. This is shown to the users who

cannot access the responsive version.

Figure 2: This figure shows how search results looks

like at the non-responsive site. Users can access this site

by clicking a link or by going to a specific URL.

Figure 3: This figure shows how My account looks at the

responsive site. This version is only shown to half of the

desktop users the rest sees the non-responsive version.

Figure 4: This figure shows how search results looks like

at the responsive site. This is accessible to all users.

and that it would enhance their job performance. It is also

important for the product to be perceived as easy to use,

which means that the user can interact with a system

without effort. By exploring emotional aspects when users

interact with products and by adapting products to their

needs and capabilities, the experience can be improved and

better usability can be achieved. Because the user has an

increasingly larger role in design development, this study

will focus on their experience and satisfaction. [19]

RELATED WORK

Responsive design and website redesign methods

Earlier related work in the area of responsive web design

highlights how best practices should be defined in this area

as well as that research are needed to find out how

responsive design performance could be improved. [2,17]

User studies have had its focus on testing responsive design

practices on mobile devices [4,5] rather than testing how

existing users are effected by the visual change and

structure a new site can have.

Several case studies have been published describing library

websites redesign process with the support of user tests

[3,20]. These kinds of tests can be particularly good to

identify and understand how people use a certain site. In

one study [20] the test was conducted with prepared tasks

that represent the actions a user could take on the

website. To cover the possible actions there can be between

six to twelve tasks to perform on the website. Furthermore,

the number of participants can be small, with between five

to fourteen users. Tests with five users often identifies 85

percent of the problems. That is why it is more cost-

efficient to iterate multiple times to find all problems rather

than to do only one test with many users. [20]

In another study [3], focused on user-centered design when

redesigning a library website, iterative user tests were

conducted to include the users’ needs and wants into the

design. The researchers asked the users to think aloud while

doing pre-determined tasks on both the old and the new

site. This gave them the opportunity to incorporate the

users’ opinions into the product rather than base it on

assumptions.

Barnes and Vidgen [1] highlights that when benchmarking

the process of a redesign, surveys can be used. Surveys

should be conducted regularly, especially when a larger

change is done to a website. Also, websites are often

complex and have multiple user groups. If an improvement

is done to one group, it affects the other groups as well and

they might perceive it differently. Perhaps the change will

lessening the ease of use for them. [1]

Technology adoption

When redesigning a website, it is important to have an

understanding of how first impressions and familiarity can

affect the user. A user makes up their mind about a website

after exposures as short as approximately 50 ms according

to Lindgaard et al [16]. They further explain that this can

result in complications. If users have a positive first

impression they may disregard problems encountered later,

i.e. errors or other issues may be overlooked. The reverse

effect occurs when the first impression is negative. Even if

the experience of the website is positive after the first

negative exposure, the first impression will make the user

reluctant to accept it. [16]

A theory named Mere Exposure Effect, explaining that

people find familiar things more attractive, has had its

research well-documented for more than fifty years

according to Ruggieri and Boca [18]. The authors clarify

that the effect is a stable and verified phenomena in social

psychology but the understanding of this effect is still

incomplete. In the same study, which was about placement

of advertising, the researchers found that even one exposure

could lead to a positive attitude towards a brand. The

processing that occurs, with a single product placement

exposure, is enough to give a person the feeling of

familiarity that is later recognized as a preference for the

brand. Other studies [13] have revealed that with every

repeated exposure of a novel object people tend to find it

more attractive. It is a positive linear relationship between

attractiveness and exposure frequency.

In another study about interaction and familiarity [9], the

result was that with each interaction the user gains a better

understanding based on experiences and learnings. This

means that the user knows how to navigate and can find

things he or she wants to do and a familiarity is created with

the site. According to Cox and Cox [6] users preference for

complexity may change if the user is exposed to a website

multiple times. Furthermore, preferences for more complex

designs often increase with repeated exposure while few

exposures lead to preference for simple design.

User acceptance is an important part when redesigning a

website because it is the users who will perform tasks at the

site. The Technology Acceptance Model (TAM) is a model

that aims to describe acceptance of users in information

systems [14,15]. This model explains underlying factors to

technology acceptance and the user’s adoption processes.

The user’s intention is a way to predict their behavior and

the intention is determined by the user’s assessment of the

system’s usage, Therefore, perceived usefulness and

perceived ease of use are two key elements in this which is

highlighted in a study by Lee and Lehto [14] as well as in

another study by Lin [15]. The first term means to what

extent a user believes a system enriches their job

performance and the second term refer to what level a user

believes a system is free of effort.

The effect of ageing

Because the company has a broad age range among the

users it is important to think about elderly people and how

they can be affected by new technology and design. A study

about age and technology [12] presented that older adults

have a lower degree of adoption towards technology.

According to the study older users do not have the

experience or training that younger people have today with

technology, which often makes them novices in the area.

Furthermore, older users often use outdated equipment

compared to the younger generations. Their computers,

mobile phones etc. can be gifts from younger adults who

have upgraded their own equipment. Because of this, the

technology is limited in terms of what kind of applications

it can run. [12]

Older adults may also have problems with age-related body

changes which is described in the same study [12]. When

users perceptual-, motor-, and cognitive-abilities decrease it

will affect their use of technology. A reduced vision is a

common age-related change, which makes it harder to e.g.

read on a screen. Also, their reduced cognitive abilities

affect the short-term memory, speed of processing and

problem solving ability, which are critical when learning

new technologies. This can cause difficulties for older

adults when dealing with web navigation and dynamic

content changes. [12]

Purpose

The purpose of this study was to identify the users on the

non-responsive site and to find the reasons as to why they

are staying there. The research question is:

Why are some users staying on the non-responsive site and

what can be done to make them use the responsive site?

This study is limited to the desktop version of the site. It

does not include studying the mobile versions because on

mobile devices the responsive site is accessible to everyone

and are therefore not relevant to this study.

METHOD

Both qualitative and quantitative data have been gathered in

this study to answer the research question. The methods

conducted was data gathering by distributing a survey and

having user tests with users from the non-responsive site.

Identifying the users

The first step was to identify the users on the non-

responsive site. For this Google Analytics (GA) was used,

which is a web analytic service that tracks and reports

website traffic. To identify the sellers, member-IDs of users

who have successfully listed at least one auction through

the non-responsive site during November 1st to January

31st were extracted with GA. In the same way the buyers’

member-IDs were exported by knowing that they had

confirmed a bought item through the non-responsive site.

However, because the number of buyers in the same time

span was unmanageable, the decision was made that it was

sufficient to extract buyer data from the most recent month,

January.

To better understand who these users were and to gather

quantitative information, their member-IDs were used to

extract demographic data, such as age, gender and location.

This was done in a program named QlikView, which is a

business intelligence & visualization software. Other

relevant factors were also extracted, e.g. their e-mails, if

they accepted to receive e-mail, how much a user sells/buys

per month, how much Gross Merchandise Volume (GMV)

they have produced etc.

Survey

To gather information on what the identified users think of

the responsive site, an online survey was conducted. The

focus of the survey was to understand the users, why they

use the non-responsive site as well as what would make

them go to the responsive site. It also included questions

about what they gain from staying the on the non-

responsive site and what their issues are with the responsive

site. The survey consisted of questions with mostly free-text

inputs, the rest was multiple-choice options. This was to

encourage the users to think freely on what they want,

instead of limiting them to predefined alternatives. In the

survey result, the sellers’ and the buyers’ answers were

separated to see if there was any difference between them.

The survey was sent to all identified sellers who accepted to

receive e-mails. On the other hand, because the buyers were

a larger number of users the decision was made to make a

selection out them to be able to handle the expected data

outcome from the survey. The survey was therefore sent to

around half of the identified buyers who had confirmed

most bought items and accepted being contacted by e-mail.

Because the survey mostly had free-text inputs the answers

needed to be analyzed. Therefore, the responses on each

question were read and categorized into different groups

depending on their inputs.

User tests

In order to gather deeper insights on design and

functionalities of the site user tests was conducted. The

purpose of these user tests was to understand how sellers

and buyers on the non-responsive site work on the site and

to find their pain points at the responsive site. Questions

and tasks on both the responsive and non-responsive site

were prepared beforehand.

To observe how users work with the site on their own

devices these user tests took place at the user's home, if

possible. To represent the users on the non-responsive site,

five different types of users were invited. As mentioned in

related work section, five users can often identify 85

percent of the problems [10]. The different types of users

were: a corporate big seller, a private big seller, a private

small seller, a private big buyer and a private small buyer.

A big seller is someone who list more than 50 times a week

and a small seller list only a few per month. A big buyer

means someone who buy around two times a week and a

small buyer purchase around one time a month. Three of

the user tests took place at the user's office or home and two

took place at the company’s office.

During the user tests the users were asked to think aloud

which requires the users to express their thoughts as they

were engaging in completing tasks. If the users stop

verbalizing for some time they were reminded to think

aloud. This data gathered from this is qualitative and

therefore it is preferred to have a smaller number of users

performing this method during user tests [7]. The user tests

were recorded using a video camera during home visits

because then the users did not need to have certain

recording software installed on their private computers.

During the user tests at the office, screen recording software

was used on the company owned computer. The video

recording was to support taking notes and making sure

everything that was said or done could be followed up later.

Graph 1: Age distribution in intervals for both sellers and buyers. Total number of buyers and sellers were 13247 respectively

39341.

All users got to decide if they wanted to be recorded or not

after the purpose of the video was explained.

RESULT

Identifying users

The results of the initial data gathering from GA identified

13247 sellers who used the old site to upload new auctions.

2,9% of them were corporate sellers, meaning they were

companies using the site to sell items, and the rest was

private users. The gender distribution was 51% female and

49% male among both the private and corporate sellers. The

number of identified buyers was 39341 who had bought

items from the non-responsive site. Out of these, 99,7%

were private and the rest were corporate users. The gender

distribution was 53% male and 47% female.

When looking closer at age intervals the distribution was

rather even among the sellers. All intervals contained

between 12% and 22% of the sellers, with the most users in

the age range 48-57 years old, as seen in Graph 1. The

intervals with the fewest users were the younger users

under 27 and older users above 68 years old. Among the

buyers it is a more uneven spread compared to the sellers.

The age intervals contained between 9% and 25% of the

buyers, but it is still the age range 48-57 that contain the

most buyers. The interval with fewest users was users under

27 years old and over 68 years old.

Survey

Sellers

The seller survey got a total of 2015 responses in the five-

day time span the survey was open, which represented

15,2% of the identified sellers. The majority of the sellers

answered that they were missing key functionalities on the

responsive site and that the older design fit them and their

usage. A fewer number of users said that they used the site

that appeared first.

Most of the feedback on the functionality was about the

lists in My account, where the sellers said they missed

information and actions to take that existed in the lists on

the non-responsive site. One other major functionality the

sellers wanted to be improved was the page where they

leave feedback to buyers, e.g. because of smaller bugs and

that they wanted to leave feedback on multiple items on one

page to save time.

One of the reasons behind why the sellers stated that the

design was better on the non-responsive site was because of

the lack of a clear overview. It can be exemplified by these

quotes:

“Text, boxes and images are smaller

compared to the new site, which

means you get a better overview

(almost everything fit on the screen so

you do not have to scroll a lot)”

“It was a better overview of sold and

not sold, I could easily choose to look

what was not paid or items I have not

left feedback on.”

Others just stated that the overview was better on the non-

responsive site and that it was easier to use but with no

further explanation of why that is.

The answers from the question “what would make you go

to the responsive site?” were grouped into three major

reasons. Reason number one was users stating that they

would never switch and would rather quit using the site

than to use the new one. Reason number two were users

who missed functions and the last reason was users who

wanted to go there but did not know how.

The largest group was those who said they would never

want to go to the responsive site. Some quotes on this was:

“Nothing! When it disappears I will

stop advertise! It will not be simpler

than that!”

“I’m happy with the old site and I

will probably quit using the site if the

old disappears”

“No I do not want to go to new design

because I’m used to the old one… I

find everything with a click. I have

been on the site since the start so I

speak with experience. My wish is to

KEEP THE OLD DESIGN!”

These people were generally upset with having to deal with

a new site for various reasons. One of the reasons was

because they have to learn something new and that will take

time. They mentioned being used to the non-responsive site

and was therefore happy with it. They were also saying that

it takes too much time to list auctions and that it is not

worth the time. It is more efficient for them to list larger

number of items on the non-responsive site compared to the

responsive one. There were also users saying that they

would only go to the responsive site if the old site is shut

down, and even then a few would rather consider quitting

then to switch.

The second largest group were those who were willing to

go to the responsive site if the functions they use are

implemented there. They said that the responsive site is

unusable right now because the functions they use are not

incorporated on the site.

The last group wanted to go to the new site but did not

know how to do it.

“If it is easier to find”

“Why aren't we being redirected

there directly?”

These quotes were typical for this group of users. They

wanted to find the site more easily by being helped to get

there by clear instructions or a guide. A few also said that

they wanted to be automatically redirected to it.

Buyers

In the buyer survey a total of 1956 users answered in five

days, which represent 24% response rate to the survey.

Most of the buyers stated in the survey that they were sent

to the non-responsive site automatically and a few did not

see the difference between them or did not even know that a

new site existed. For those who actively stayed on the non-

responsive site they were doing it because they are used to

it and they have their working habits. Some stated that this

makes it easier to use and to understand. They also

mentioned that the overview is better on the non-responsive

site, with less scrolling and clicking to navigate. In addition

to this, there are important functions that are missing which

stops them from switching to the responsive site.

The answers to what would make them go to the responsive

site were for the buyers also grouped into three major

reasons. The largest number of users stated that the entrance

to the responsive site should be easier to find. This is some

examples of quotes:

“If I knew how to do it”

“If I could find it… why are there

several sites?”

“If I could easily switch. Clear

explanation of how I change.”

A few specifically mentioned having clearer instructions

explaining how to go to the new site. Other wanted a guide

on the responsive site explaining what is new and how

things can be done.

The second largest group of users would like the overview

and the layout to be better to be able to use the responsive

site. They did not state how this can be achieved with the

exception of reducing the number of clicks required on the

responsive site and to remove white space in the auction

lists in My account to make the lists more compact. These

were typical quotes for this group:

“Hard to believe that someone can

get me to start using the new site

when there is so much clutter and so

incredibly immense. “

“Clearer overview. Easier to find.

Less messy.”

The last and third largest group wrote that they would never

go to the responsive site and that they would quit their

membership if the non-responsive site is closed. The reason

for this is that they do not believe that something can

replace the non-responsive site and that something that is

already perfect should and cannot be replaced.

User tests

During the tests, the users consisted of three males (52, 54

and 72 years old) and two females (53 and 70 years old). A

summary of the age, gender, type of user and how much the

test user sell/buy can be seen in Table 1.

AGE GENDER TYPE OF USER SELL/BUY PER WEEK

52 Male Big seller corporate Several hundred items/week

53 Female Big seller 50 items/month

72 Male Small seller 1 item/month

53 Male Big buyer 2 items/week

71 Female Small buyer Occasionally

Table 1: This table show the test users’ age, gender, what type

of user they were and how much they sell/buy at the site.

The users had varied technological knowledge which was

something they got to interpret themselves but was also

observed during the user tests. Their difference in

technological knowledge made their interactions with the

site quite different. The older people was more careful

when navigating on the site compared to the younger, even

though the age gap was not that large.

When it came to the equipment they used, all of them had

both mobile phones and computers. Their screens and

computers were relatively new and the computer screens

were between 12-27 inches. The one having a 12-inch

screen also had a bigger screen at work which he used

frequently. All except one had mobile phones that could

access internet.

Two of the users were keen to stay on the old site. One,

who was a corporate seller who sell several hundred smaller

items a week, said that:

“I have to be effective to sell a lot”

He said that he felt that the responsive site did not

encourage big sellers because it is slower to put up similar

auctions again. In the non-responsive site there is a function

where he can restart auctions with edits that it is missing on

the responsive one. He would never upload a totally new

auction because it requires filling out all information again

and that takes time. After he have sold something he use a

list called a “picklist” where all sold items are shown

grouped by buyers and it is printable. This list on the non-

responsive site can be filtered and it is all in one page,

however on the responsive site it is quite different. There is

no filtering and it has pages, which made his picklist 27

pages with 50 items on each page. He states that this is

unusable for him because it would take too much time to

look through it.

The other person, who's very keen to stay, was the big

seller. She worked differently compared to the corporate

seller. She did not use the picklist even though she sold 50

items a month, instead she used the “sold” list to manually

copy the addresses to the buyers and pasting it in her own

document, which she later can print to place on packages.

She had tried the responsive version several times but did

not like it because she has her working habits and was used

to the non-responsive way. She said that:

“Why change the old site when it

works?”

The next user was a small seller and he did not see the

reason why he should switch when they design-wise look

the same. Because he was a small seller he did not need the

functionality of the picklist for example, that is why he used

the other seller lists. He just wanted to be able to see his

active and sold auctions in the same list, which he cannot

on the responsive site. He said that it is unnecessary to

divide them into two different lists when he often only have

one or two items to view at the same time. He also

mentioned that he would like bigger images to know what

item he is dealing with which he wanted on both sites.

Except this, he did not see any functionality differences

when comparing the both sites and he did not understand

why the site was rebuilt when he said this.

The big buyer bought items around two times a week and

he used both the mobile phone and his laptop to go to the

site. This made him quite confused because the mobile

version is the same as the responsive site and the functions

that were new were only working on his mobile phone. He

believed that he was on the new site on both devices and

that they were just structured differently. He said that he

thought that the company did as other sites, where they

release updates without you having an option of choosing

what version you want. He said the mobile version was

better when viewing auctions in the “wish list”, a list

whenever you can add your favorite auctions, because he

could filter the list on ongoing and ended auctions.

However, because he was one of the users that could not go

to the new My account-page he could not use the

responsive site on his desktop. When talking about change

and to learn how to use a new site he said that everything

new is always an obstacle to overcome but you get used to

it eventually.

The last user was a small buyer who bought items

occasionally, mostly for her grandchildrens’ birthdays and

during Christmas. She used the whole non-responsive site

because it is the first that is shown when she goes to the site

in the browser. She was also one of the users who did not

have the option to go to the responsive version of My

account. After she tried the responsive site with a test

account she said that she did not understand why there are

two sites and she is uncertain if she would go to the new

one if she could. She liked the new design of viewing her

own feedback because it was clearer, but she said that she is

used to the old navigation and she knows how it works.

Then she states that:

“Of course you can get used to

anything maybe"

DISCUSSION

The purpose of this study was to first identify the ones

using the non-responsive site. The next step was to

understand why users choose to stay on the non-responsive

site and what can be done to make them more satisfied with

the responsive one.

Identified users on the non-responsive site

The results from identifying the users on the old site made

it clear that all ages and genders are using the non-

responsive site. Even though the buyers’ age distribution

was not as even as the sellers it is still even enough to point

out that all ages use it. The most common age interval was

the ages between 48-57 for both buyers and sellers, which

is when age is beginning to take its toll on the body. As

stated in the related work, body changes come with age and

perceptual-, motor-, and cognitive-abilities decrease among

those who are older. Older users often use outdated

technology and do not have the same base as younger

people when it comes to learning and to adapting to new

things. This could be seen in the user tests by the older

user’s careful behavior while navigating. However, while

visiting users their technical equipment could be observed

and the two users visiting the company’s office got to

describe their equipment. The results of this was that they

did not have outdated equipment. This was a surprising

discovery because it was not expected, however because of

the small sample this cannot be said for every older user.

The reasons for staying on the non-responsive site

The survey and user tests identified three main reasons

when it comes to leaving the non-responsive site to use the

responsive one. The largest group among the sellers and the

last of the three major reasons among the buyers said that

they would never leave and that they would rather quit

using the site then use the responsive one. Related work in

this area that was examined in this study did not highlight

the consequences of having a new visual design and

structure when transforming a site to become responsive.

The result from this study is that both old and young users

said that it is a hassle to learn something new and that it

will take them extra time to perform their tasks at the site.

This group also said that they did not see the benefits with

having a responsive site even though more and more traffic

goes through mobile devices.

The fundamentals of HCI and TAM is that users must feel

that they are gaining something from using a product and

that it enhances the user’s performance. This does not seem

to be the case right now when users have a hard time

learning to navigate on the responsive site. Two persons

also mentioned in their user tests that they did not see the

benefit of a new site when the non-responsive site works for

them and when it looks the same.

Because the user groups said that they would never use the

responsive site, they most likely had a negative experience

from the start. Therefore, it could be argued that they are

reluctant to see what the advantages are with the responsive

site. However, studies have pointed towards people getting

more familiar with each usage and exposure to a site. With

this, the users in this group needs to repeatedly use the site

in order to learn how it works and get a positive experience

from it. Still, this is complicated matter because they can

choose to go back to the non-responsive site by themselves

which means that they can avoid getting familiar with the

responsive site.

The second largest group among the buyers and the third

largest group among the sellers, said that they would like be

on the responsive site. This group also said that they would

like some guidance on how the site works and what is new

to be able to find it easier. Currently only 50% get the

opportunity to see a link to the whole site and there are both

advantages and disadvantages with dividing it like this.

Because users make up their mind after only a few

milliseconds, the first impression often stays and effect the

experience. Since the site is not completely finished there is

always a risk of people trying it out and are getting a bad

experience because certain things are missing. Therefore, it

is an advantage to show the link to a limited amount of

users first because the things that need improvement can be

identified and corrected without effecting the rest of the

users’ experience. On the other hand, a disadvantage is that

users become confused while using the responsive design

because they are sent to the non-responsive site when

functions are missing, and sometimes they cannot find their

way back again. This can happen when they want to, for

example, book shipping, contact highest bidder or use other

functionalities that is not implemented on the responsive

site yet. To be able to make everyone use the whole site, all

the known missing functionalities needs to be implemented

and the problems that have been discovered need to be

fixed as well to get the uses to be satisfied.

The last reasons was different for the sellers and the buyers.

One was because functionality was missing and the other

was that the overview was worse on the responsive site. As

previously mentioned, missing core functionality is mainly

the reason to why not everyone can go to the responsive

site. The survey and user tests identified additional

functionalities that was missing apart from known ones and

also identified problems with existing functionalities, e.g.

the picklist and restarting auctions. That the overview was

worse on the responsive site compared to the non-

responsive one was a new discovery but not many users

stated why it was worse. The known reasons such as white

space in the lists is because the design approach was doing

mobile devices first which resulted in the lists being empty

in the middle when scaled up. This is something that needs

to be considered to make it more desktop friendly.

Designing for two user groups

In this study the users from the non-responsive site are in

focus, but it is important while designing not to forget the

existing user’s behaviors and needs. If something is to be

changed on the responsive site to please the non-responsive

users, the users on the responsive site needs to accept this

change as well. It can become a delicate matter balancing

the two different user groups because of their varying

opinions and needs. The best way would be to find a

solution that would fit both of them, but the risk is that it

instead fit no one because the design is a compromise. The

same problem can occur while dealing with small and big

buyers/seller because they use the site quite differently.

Heavy sellers often use the “picklist” because the view is

more compressed which means that they can oversee more

items at a time. However, a small seller does not have the

same need as heavy sellers. As the small seller said in the

user test, he wants to have everything in the same lists both

ongoing and sold auctions. If this was to be implemented, it

would be unusable by the bigger sellers. These different

types of user need to be taken in consideration as well for

every design decision.

What can be done to increase the satisfaction on the responsive site?

While still having the different user groups in mind,

missing functionalities is something that needs to be

implemented. This will make all those who are using these

functionalities be more satisfied with the responsive site.

This study also discovered functionality that exist on both

sites but are not satisfactory enough on the responsive site.

This needs to be researched further to discover what the

problems are with these and how to solve them. When the

implementation of missing functionality is completed, the

link inviting users can be shown to all users. If this is done

earlier it can affect the user’s first impressions negatively

because of the known missing functionalities. At the same

time as showing the link to all users, the guides should be

done to help the users with finding everything they usually

use.

The hardest users to satisfy are the users who do not see a

benefit with using the responsive site and said that they

would never use it. Because the number of exposures effect

the way a site is perceived hopefully more exposures could

lead to more users to be convinced to switch site. If some

users continue to stay on the non-responsive site despite

repeated exposures, perhaps additional features need to be

implemented for them to see the value of using the

responsive site. However, this needs to be examined with

additional surveys, interviews or user tests with the users.

Method discussion

Because of the large number of users that stay the non-

responsive site the survey was an appropriate approach to

gather as much insights as possible in a limited time frame.

One advantage of using a survey is that it can reach many

users located all over Sweden in a short amount of time

compared to other methods such as workshops, telephone

interviews etc. However, surveys have their limitations as

well because it is a one-way communication and it is

unclear if the respondents understand the questions asked.

There is also no means of asking quick follow-up questions

to their responses to further understand their answers. This

is why the survey was supplemented with user tests. The

user tests were also conducted because the users’ way of

working and usage of the site could be observed while they

were performing specific tasks. While surveys can gather

large quantity of data, this method was used to dig even

deeper to gain further insights about the users and their

needs. On the other hand, user tests are time consuming and

can only be done with smaller amounts of participants. In

this case, the users taking part in the test were older because

younger users did not have the time during the day to either

participate in their home or at the company’s office. This

was unfortunate because the result from the survey showed

that the age distribution was quite even. Therefore, it is

important to have in mind that the results from the user test

might be skewed. However, because the younger users

could make their voices heard trough the survey they were

not forgotten.

Future work

This was an initial study to discover user needs and

problems with the responsive site. To keep track of how the

progress goes of transferring users to the responsive site,

more user studies need to be conducted. This is to be able to

see how the satisfaction is among the users and to be able to

adjust the site to their usage. When doing a redesign of a

complex site such as this company with its diverse user

base, it is important to be able to give value to the users. If

the users are not satisfied they will look for other forums to

buy and sell online.

The results of this study can be a start of discovering how

redesigns can be done in the future. In this specific case the

users had the option to go to the responsive site by

themselves, but other solutions could be researched to find

the most effective and satisfying one for the user when

redesigning a site.

CONCLUSION

To conclude, the reasons behind users staying on the non-

responsive site were mainly because of three things;

functionality was missing, they could not find the

responsive site or they were more comfortable with the way

the non-responsive site works.

To make user from the non-responsive site more satisfied

with the responsive site and move them over, the first step

would be to implement the functionality that are missing.

Because the user base is varied, the implementation of the

missing functionality need to be carefully planned to fit the

user’s needs on both the responsive and the non-responsive

site. When all functionality is implemented, all users can be

invited to use the responsive site with guides helping them

find the functionalities they use. However, there were users

saying that they would never switch sites and to get them

satisfied with the responsive site more research needs to be

done in order to find out what would make them satisfied or

comfortable with the responsive version.

ACKNOWLEDGMENTS

A special thank you to the company, its employees and

members for supporting this study with all their knowledge

and expertise. My gratitude also goes towards my

supervisors at KTH and the company.

REFERENCES

1. S.J. Barnes and R.T Vidgen. 2001. Assessing the

Effect of a Web Site Redesign Initiative: An SME

Case Study. International Journal of Management

Literature 1, January.

2. Meltem Huri Baturay and Murat Birtane. 2013.

Responsive Web Design: A New Type of Design

for Web-Based Instructional Content. Procedia -

Social and Behavioral Sciences 106, December:

2275–2279.

http://doi.org/10.1016/j.sbspro.2013.12.259

3. Danielle a. Becker and Lauren Yannotta. 2013.

Modeling a library website redesign process:

developing a user-centered website through

usability testing. Information Technology and

Libraries 32, 1: 6.

http://doi.org/10.6017/ital.v32i1.2311

4. Kim Bohyun. 2013. Responsive Web Design ,

Discoverability , and Mobile Challenge. Library

Technology Reports 49, 6: 29–40.

5. Henrik Bygdeman. Från desktop till responsiv

design – en fallstudie kring utformningen av

publikationsdatabasen på Skolverkets webbplats.

Csc.Kth.Se. Retrieved from

http://www.csc.kth.se/utbildning/kth/kurser/DD143

X/dkand11/Group2Mads/Rapport_Sebastian_Hew_

Toni_Huotari.pdf

6. Dena Cox and Anthony D Cox. 2002. Beyond first

impressions: The effects of repeated exposure on

consumer liking of visually conplex and simple

product designs. Spring 30, 2: 119–130.

http://doi.org/10.1177/03079459994371

7. Sharon Eveland Jr., William and Dunwoody. 2000.

Examining Information Processing on the World

Wide Web Using Think Aloud Protocols. Media

Psychology 2, 3: 219–244.

http://doi.org/10.1207/S1532785XMEP0203

8. Robert Fox. 2015. Being responsive. OCLC

Systems & Services: International digital library

perspectives, 28, 3: 119 – 125.

9. David Gefen. 2000. E-commerce: the role of

familiarity and trust. Omega 28, 6: 725–737.

http://doi.org/10.1016/S0305-0483(00)00021-9

10. Michael Glanznig. 2012. User Experience

Research: Modelling and Describing the Subjective.

Interdisciplinary Description of Complex Systems

10, 3: 235–247. http://doi.org/10.7906/indecs.10.3.3

11. Nancy R NR Glassman and Phil Shen. 2014. One

Site Fits All: Responsive Web Design. Journal of

Electronic Resources in Medical Libraries 11, 2:

78–90.

http://doi.org/10.1080/15424065.2014.908347

12. Vicki L. Hanson. 2010. Influencing technology

adoption by older adults. Interacting with

Computers 22, 6: 502–509.

http://doi.org/10.1016/j.intcom.2010.09.001

13. Paul Hekkert, Clementine Thurgood, and T. W

Allan Whitfield. 2013. The mere exposure effect for

consumer products as a consequence of existing

familiarity and controlled exposure. Acta

Psychologica 144, 2: 411–417.

http://doi.org/10.1016/j.actpsy.2013.07.015

14. Doo Young Lee and Mark R. Lehto. 2013. User

acceptance of YouTube for procedural learning: An

extension of the Technology Acceptance Model.

Computers and Education 61, 1: 193–208.

http://doi.org/10.1016/j.compedu.2012.10.001

15. Chin Chao Lin. 2013. Exploring the relationship

between technology acceptance model and usability

test. Information Technology and Management 14,

3: 243–255. http://doi.org/10.1007/s10799-013-

0162-0

16. Gitte Lindgaard, Gary Fernandes, Cathy Dudek,

and J. Brown. 2006. Attention web designers: You

have 50 milliseconds to make a good first

impression! Behaviour & Information Technology

25, 2: 115–126.

http://doi.org/10.1080/01449290500330448

17. S. Mohorovicic. 2013. Implementing responsive

web design for enhanced web presence. 36th

International Convention on Information &

Communication Technology Electronics &

Microelectronics (MIPRO): 1206–1210.

18. Stefano Ruggieri and Stefano Boca. 2013. At the

roots of product placement: The mere exposure

effect. Europe’s Journal of Psychology 9, 2: 246–

258. http://doi.org/10.5964/ejop.v9i2.522

19. Manuel Sánchez-Franco and José Roldán. 2005.

Web acceptance and usage model. International

Journal of Retail & Distribution mangement 34, 1:

21–48. http://doi.org/10.1108/02656710210415703

20. Robert L Tolliver, David S Carter, Suzanne E

Chapman, et al. 2006. Website redesign and testing

with a usability consultant: lessons learned. OCLC

Systems & Services: International digital library

perspectives 21, 3: 156 – 166.

www.kth.se