Usability Testing Jesselindsey.com website. Usability Test Report Jesse Lindsey Metropolitan State University. WRIT 574G. Usability In Technical Communication. Fall 2016.
Usability Testing Jesselindsey.com website. Usability Test Report
Jesse Lindsey Metropolitan State University. WRIT 574G. Usability In Technical Communication. Fall 2016.
Usability Test Report 1
EXECUTIVE SUMMARY
Summary
This assignment was conduct by Jesse Lindsey as a usability test for the jesselindsey.weebly.com
website to receive credit for his WRIT 574G Usability for Technical Communication Course. The
jesselindsey.weebly.com website is a professional online portfolio that is used as supplemental
document for online job application. The portfolio consists of past academic and professional
work accomplished by Jesse Lindsey. The research began by determining essential scenarios that
most employers would view when evaluating an employee’s online portfolio. There were six
scenarios created that the researcher believed would grant the most helpful data:
1. Navigating to the homepage of the website
2. Navigating to the graduate coursework webpage of the website
3. Reviewing the content on the graduate coursework webpage
4. Navigating to the homepage from the graduate coursework page
5. Navigating to the creative work page from the homepage
6. Navigating to the contact us page from the creative work page, and sending an email from
the contact us page.
Due to the extraordinary nature of the usability test all participants needed to be screened to
make sure that they had previous interviewing experience with applicants that used an online
portfolio.
After reserving conference rooms at Wells Fargo and Saint Paul College, and creating a guest
user account on the researcher’s computer, we recruited ten volunteers. After screening the ten
volunteers there were six participants that remained. The usability test lasted approximately 15-
20 minutes on average.
Key Takeaways
Participants were frustrated when they encountered links that were broken while trying to
review content on webpages.
Recommendations:
The researcher recommends that modification be made to the broken links on the graduate
work page and creative work page of the jesselindsey.com website. This will prevent potential
employers from becoming deterred from the applicant due to an unappealing webpage.
Usability Test Report 2
BACKGROUND AND INTRODUCTION
Background
The product Tested was the jesselindsey.weebly.com website. The website was created by Jesse
Leneal Lindsey to prepare for his final capstone project to earn a Master of Science in Technical
Communication Degree. After completion of his MS degree, Lindsey plans on utilizing the
website as a professional online portfolio, serving as a supplemental document for his future
online application.
Introduction
The objective of the usability test for the jesselindsey.weebly.com website was to evaluate the
user-friendliness of the website, and measure the professional appeal of the website for
potential employers. In order to evaluate the user-friendliness and professional appeal of the
website, the researcher created scenarios that mimicked actions future employers might
preform while reviewing the website. The scenarios were based on ease of navigation through
webpages, the comfort of locating necessary links on the website, and understanding the
content on the webpages.
Usability Test Report 3
PARTICIPANT ANALYSIS
Background Information
The average user of the jesselindsey.weebly.com website will be potential employers of Jesse
Lindsey. Most users will have previous experience reviewing applicant’s online portfolios.
Typically, users will focus their attention on aspects of the online portfolio such as website
esthetics, functioning links to subpages, error free content on the webpages, and ease of
navigation through the website.
Familiarity
The research selected six of ten volunteers that agreed to participate in the usability test. The
selection was based on volunteers meeting the minimum requirement to participate in the test.
Only six volunteers had previous experience interviewing job candidates who utilized online
portfolios with their application. Three participants worked at Wells Fargo Home Mortgage in
their collections department as Technical Writer Managers, and three participants worked at
Saint Paul College as department heads. The participant’s ages ranged from thirty-eight to fifty-
four years old.
Participant Analysis
Based on review of the pre-test data, which was reviewed before the usability test began, the
research was confident that the test scenarios addressed all of the necessary information
needed to accumulate appropriate data for analysis.
Usability Test Report 4
USER ANALYSIS AND PARTICIPANTS
Target Users
The target users are employers seeking applicants that utilize online portfolios in addition to
their job applications. Also Metropolitan State University Faculty.
Participant’s Demographics
Age: Participant’s ranged from age thirty-eight to fifty-four.
Gender: There were two female participants and four male participants.
Ethnicity: All participants were of Caucasian decent.
Vocational Experience: All participants had previous experience reviewing applicant’s
professional online portfolios
Usability Test Report 5
METHODOLOGY AND PROCEDURES
Consent
The day before the test began, each participant was emailed a letter of consent, and asked
to print, sign, and bring a copy to the testing location the next day. Participants were also
informed there would be copies of the consent form at the testing locations. Participants
were informed that participation in the usability test would not be permitted without
signed consent. All participants complied.
Testing Locations
The test recruiter obtained ten test volunteers. The volunteers were asked question by the
test screener to determine if their experience met the minimum qualification for the
usability test. The screener communicated with the volunteers via email several days
before the testing date. There were six initial volunteers from Wells Fargo, and four from
Saint Paul College. The screener provided the researcher with his question responses, and
the researcher determined that three volunteers from Wells Fargo, and three from Saint
Paul College met the minimum qualifications to participate in the usability test.
There were two testing locations. The first testing location was at Wells Fargo Home
Mortgage corporate office in SE Minneapolis. The test was held in a conference room
located on the second floor of the north building. The test at Wells Fargo took place on
November 21, 2016. Three participants were test at this location. Each participant was
assigned a thirty-minute slot to complete the test. The first test started at eleven thirty
AM, the second test immediately followed at twelve o’clock noon, and the last test
started at twelve thirty PM.
The second test location was at Saint Paul College in a conference room located on the
second floor of the building. The testing at Saint Paul College took place on November
22, 2016. There were three participants for the second test. The time schedule was the
same as the Wells Fargo location.
Testing
Before the test began, participants were asked to fill-out a pre-test questionnaire. The
usability test was divided into six section. Participants were allowed to continue on to the
next exercise at their own leisure, but were asked to answer a post-task survey question
after the completion of each task. There was no interruption by the test administrator. If
participants had questions, they were instructed to ask the test administrator for
assistances. After the test, participants were asked to fill out a post-test questionnaire.
Usability Test Report 6
FINDINGS
Introduction
A usability test was conducted to evaluate the user experience of the jesselindsey.weebly.com
website. The test was conducted in a lab. There were two primary locations where the test were
administered. The test administrator had feasible reason to believe that conducting the test at
different locations would not contaminate the test results due to the structure of the usability test,
and the solitary environment that the testing locations provided. The test was conducted in
conference rooms at a corporate offices and a higher education institution. The conference rooms
were made available be team members and faculty at the two locations. There were ten initial
volunteers. The volunteers were recruited from various technical writing departments and higher
education institutions. All participant’s had experience conducting interviews with applicants that
utilized online portfolios as part of their application process. The usability test consisted of four
sections: a pre-test questionnaire, a usability test integrated with post-task survey questions, and a
post-test questionnaire. The finding were as follows:
Pre-Test Finding
The Pre-Test Questionnaire concluded that employers prefer reviewing online portfolios
that are simple, have good content, and do not have any broken links. Further, the Pre-
Test Questionnaire suggested that employers are deterred from applicant’s online
portfolios because portfolios have spelling errors, and bad layouts. Furthermore, half of
the test participants said that the top feature sought after when reviewing applicant’s
online portfolios is the portfolio’s esthetics.
Usability Test Report 7
Pre-Test Questionnaire Results, 2 Part Question: Pyramid Chart
Question #1
What two features make an applicant’s online portfolio stand out to you?
Pre-Test Questionnaire Results: Pyramid Chart
Question #2
What feature deter you from considering an applicant based on his online portfolio?
83% of participant prefer reviewing online portfolios that are simple.
60% prefered portfolios with good content
30% prefered no broken links
83% of participants are detered from applicants because the applicant's
online portfolio has spelling or gramatical erros.
13% are detered because
of bad portfolio
layouts.
Usability Test Report 8
Pre-Test Questionnaire Results: Pyramid Chart
Question #3
When selecting an applicant for employment, what is the top thing that you look for in their
online portfolio, what is must haves?
Post-Task Findings
The first Post-Task Questionnaire question indicated that navigating to the homepage of
the jesselindsey.weebly.com website was easy. The second Post-Task question concluded
that navigating to the graduate course work page was also easy. However, only 15% of
participants said that reviewing content on the graduate coursework webpage was easy.
50% of participants said that navigating back to the homepage from the graduate course
work page was easy. 67% of participants said that it was easy to review content on the
creative work webpage. 100% of participants said that it was easy to navigate and send an
email from the contact us webpage.
50% of participants ranked portfolio esthethic as the top features they look for in an applicant's
online portfolio.
30% of participants ranked easy navigation through the portfolio as the top feature they look for
on an applicant's online portfolio.
20% ranked clear homepage content as
their top feature to examine.
Usability Test Report 9
Post-Task Questions
Question #1:
Please rate, how difficult it was to navigate to the homepage of the website?
Question #2:
Please rate, how difficult was it to navigate to the graduate coursework page from the webpage?
Question #3
Please rate, how difficult was it to explore content on the graduate coursework webpage?
Question #4
Please rate, how difficult was it to navigate back to the homepage from the graduate coursework
page?
Question #5
Please rate, how difficult was it to explore content on the creative work webpage?
Question #6
Please rate, how difficult was it to navigate to, and send an email from the contact us page?
Accumulative Chart: Post-Task Questionnaire
5
5
5
5
5
5
5
5
5
5
5
5
4
3
4
4
5
4
4
4
5
5
5
4
5
5
4
4
5
5
5
5
5
5
5
5
0 1 2 3 4 5 6
PARTICIPANT 1
PARTICIPANT 2
PARTICIPANT 3
PARTICIPANT 4
PARTICIPANT 5
PARTICIPANT 6
Accumulative Post-Task Chart
Question 6 Question 5 Question 4 Question 3 Question 2 Question 1
Usability Test Report 10
Post-Test Findings
Question one through four were rated on a scale of one through five, five being easy and
one being difficult. Question one indicated that 33% of participant found the website to
be easy to use. 50% found it somewhat easy to use, and 15.5% found that the website was
neither easy or difficult to use. Over 83% percent found that overall, the website was at
least somewhat easy to use. Therefore, the researcher concluded that moderate changes
need to be made to improve the usability of the webpage.
Question two indicated that 100% of participants said that overall, it was easy to navigate
through the website using the menu bar. The researcher concluded that no changes need
to be made to the menu navigation links.
Question three indicated that 100% of participants found it easy to relate the website to a
professional online portfolio. The researcher concluded that the website layout is good,
but content needs to be added.
Question four indicated that 66% of participants found it easy to relate the website to be
user-friendly. 33% found that it was somewhat easy to relate the website to be user-
friendly.
Question five indicated that participant thought the feature on the homepage that
appeared most professional was its simplicity. Participant One stated, “the website
immediately appeared professional because it was not filled with clutter.
No annoying lights were blinking, and the content gave me a clear indication of what the
website was all about”. Question six indicated that the component that seemed least
professional on the website was the broken links on the graduate course work page.
Question seven prompted a unanimous response that the feature that all participants
would change to make the website more professional is the broken links.
Usability Test Report 11
Post-Test Questions
Question #1
Overall, please rate how difficult or easy it was to use the website?
Question #2 Overall, please rate how difficult or easy it was to navigate through the website using the menu
bar?
Question #3
How difficult or easy was it to identify the website as a professional online portfolio?
Question #4
Please rate, rather it was difficult or easy to relate the online portfolio as user-friendly?
Accumulative Chart: Post-Test Question 1-4
0 1 2 3 4 5 6
Participant 1
Participant 2
Participant 3
Participant 4
Participant 5
Participant 6
Accumulative Post-Test Chart
Question 4 Question 3 Question 2 Question 1
Usability Test Report 12
APENDIX
Introduction
The usability test will be structured to evaluate the user experience of the jesselindsey.weebly.com
website. The test is a moderate field study. There are two primary locations where the test will be
conducted; however, the location is subject to change given each participants circumstance. The
researchers’ objective is to determine the user-friendliness of the website, and to also determine
the quality of how the website will appeal to future employers.
The usability test will be conducted as survey based. However, participants are going to be required
to use a computer during the test, which will be provided by the test administrator. There will be
a pre-test, post-task, and post-test questionnaire to provide data for analyzing the user-friendliness
and professionalism of the website.
All participants will be screened to determine if they meet the minimum qualification to participate
in the usability test. The first screening process will consist of a moderator asking discrete question
to determine participant’s ability to participate in the test. The second test will be conducted by
the test screener, who will ask pertinent question to determine participant’s qualification to proceed
with the test. The usability test will consist of 5-10 participants, all of which must have held
managerial position in which they have conducted recruitment of applicants who utilized online
portfolios. The usability test will be conducted on an individual basis. Only one participant will be
tested at a time; however, multiple participants may be tested simultaneous if, and, or under
extraordinary circumstances.
Usability Test Report 13
Greetings
Greeting Participants: Hi, my name is [name of test administrator]. Thank you for agreeing to
participate and accepting the terms and conditions of this survey. Your input will help our team
learn a lot of useful feedback for improving the jesselindsey.weebly.com website in order to make
it as user friendly as possible and easily accessible to employers. Before we begin the user-
experience testing, I would like survey participants to first complete a pre-test questionnaire.
Pre-Test Questionnaire: Thank you for participating in the pre-test questionnaire. It is a simple
3 question questionnaire. I will allow 5 minutes to complete the questionnaire. After you have
completed the questionnaire, please move your questionnaire to the side of the table so that the
test administrator will know that you have finished the questionnaire.
Now that the pre-test has been completed, I would like for you to sit at the table and use the
computer to complete the user-experience task. After completing each task, please answer the post
task and post-test questions.
Transitioning in the middle of the test: Now that you had an opportunity to complete the user-
experience task for the website, I ask that you take a post-test survey sheet and answer the survey
question to the best of your knowledge. Don’t worry, we are not trying to figure out how well you
navigated the website? Rather, how convenient and user-friendly the website was for you to use.
Concluding The Test: You guys were amazing. We have learned a lot from you. As promised,
you will receive a $5.00 Jimmy Johns gift card as an incentive for participating in our usability
test.
Again, thank you very much for coming in today. We really learned a lot from your feedback.
Usability Test Report 14
Screener
Script
Hello everyone. We will be screen to determine who qualifies to participate in the survey today.
We will ask a few simple questions; all you have to do is raise your hand if the question applies to
you. I will try to make this quick as possible.
First question: Please raise your hand if you have interviewer experience.
Thank you. If your hand is not up, I want to thank you for your participation, and have a nice day.
[Screener escorts participants out of the testing area].
Second question: Please raise your hand if you have conducted recruitment for applicants that
used online portfolios.
Thank you. If your hand is not up, I want to thank you for your participation, and have a nice day.
[Screener escorts participants out of the testing area].
Thank you, that concludes the screening process of the usability test. If all of you are willing to
continue on with the usability test, we can start the pre-test questionnaire shortly. Thank you.
Usability Test Report 15
Pre-Test Questionnaire
Script
Thank you for taking time out of your day to participate in our usability test. Your opinion and
participation will provide feedback that is essential to improving the jesselindsey.weebly.com
website, and how users experience it. Remember, we are not testing your knowledge of using
online portfolio website, but how easy it is for you to use and understand the content, and
terminology of the website.
Before we start, I would like you to complete, to the best of your knowledge, a pre-test
questionnaire that will provide us with insight to your computer and internet background, and more
specifically, your understanding of evaluating online portfolios for potential employees.
1.) What two features make an applicant’s online portfolio stand out to you?
a.
b.
2.) What feature deter you from considering an applicant based on his online portfolio?
a.
3.) When selecting an applicant for employment, what is the top things that you look for in
their online portfolio, what are the three must haves?
a.
Usability Test Report 16
Scenarios
Script
Thank you again for coming today to give us your feedback. Now that we have a little
understanding of your background, we will like to invite you to start the task section of the usability
test. The first scenario will be to enter the URL: jesselindsey.weebly.com into the URL bar. Than
we will have you explore some of the pages in the menu bar. We hope that you keep the ease and
difficulties levels in mind that you encounter while navigating through the webpages. This will
help you come up with the most accurate responses to the post-task and post-test surveys. Thank
you again, and here are the task assignments [hand them the assignment sheet].
Usability Test Report 17
Scenarios/Tasks
SCENARIO 1
Task: Type in the URL: jesselindsey.weebly.com, and explore home page.
Scenario: Participants should be at a computer. Participants should type in the URL:
jesselindsey.weebly.com. The participants should than explore and observe the home page for
content and terminology.
SCENARIO 2
Task: Find and select graduate coursework page.
Scenario: Participants will navigate to the graduate coursework page on the website using the
graduate course work link located at the top of the home page in the menu bar.
SCENARIO 3
Task: Find and explore content in graduate coursework page.
Scenario: Participants have navigated to the graduate course work page of the website.
Participants will randomly explore subsequent pages of the graduate coursework webpage
utilizing the link buttons on the graduate coursework page.
SCENARIO 4
Task: Navigate back to the homepage from the graduate course page.
Scenario: After exploring subsequent pages of the graduate coursework page of the website,
participants will navigate back to the homepage of the website using the homepage link in the
menu bar.
SCENARIO 5
Task: Select and explore creative work using the creative work page of the website.
Scenario: Participants have returned to the homepage. Now participants are going to navigate to
the creative work page using the link in the menu bar at the top of the homepage. After
participants have navigated to the creative work page, participants will explore content using
various buttons on the creative work page.
Usability Test Report 18
SCENARIO 6
Task: Navigate to the contact us page and send an email.
Scenario: From the creative work page, participants will navigate to the contact us page using
the contact us link in the menu bar at the top of the creative work page. Once the participants
have navigated to the contact us page, participants will fill in the necessary information and send
an email.
Usability Test Report 19
Post-Task Questionnaire
Script
Thank you for finishing the user-experience task that were assigned to you. If you are willing to
continue on with the usability testing, I would like for you to take about 5 minutes to give us your
feedback and fill out the post-task questionnaire. Remember, we are not evaluating you, but the
user-friendliness of the website. Following the post-task questionnaire there will be a post-test
questionnaire. Here are your post-task questionnaire sheets [Hand participants the questionnaires]
Usability Test Report 20
Please circle a number in response to the following questions
Scenario 1
Please rate, how difficult it was to navigate to the homepage of the website?
Difficult 1 2 3 4 5 Easy
Scenario 2
Please rate, how difficult was it to navigate to the graduate coursework page from the webpage?
Difficult 1 2 3 4 5 Easy
Scenario 3
Please rate, how difficult was it to explore content on the graduate coursework webpage?
Difficult 1 2 3 4 5 Easy
Scenario 4
Please rate, how difficult was it to navigate back to the homepage from the graduate coursework
page?
Difficult 1 2 3 4 5 Easy
Scenario 5
Please rate, how difficult was it to explore content on the creative work webpage?
Difficult 1 2 3 4 5 Easy
Scenario 6
Please rate, how difficult was it to navigate to, and send an email from the contact us page?
Difficult 1 2 3 4 5 Easy
Usability Test Report 21
Post-Test Questionnaire
Script
Thanks for all of you help and feedback. Now that you have accomplished all of the user-
experience task provided to you, and also finished the post-task questionnaire, we would like for
you to take a quick 8 question post-test questionnaire if you are willing to continue with the testing.
The post-test questionnaire is not to evaluate your knowledge, but to analysis the user-experience
of the jesselindsey.weebly.com website, and how it is depicted to interviews. Thank you again for
your feedback, and here is your post-test questionnaires sheet [Hand participants the
questionnaire].
Usability Test Report 22
1). Overall, please rate how difficult or easy it was to use the website?
Difficult 1 2 3 4 5 Easy
2.) How easy or difficult was it to navigate through the website using the menu bar?
Difficult 1 2 3 4 5 Easy
3.) How difficult or easy was it to identify the website as a professional online portfolio?
Difficult 1 2 3 4 5 Easy
4.) Please rate, rather it was difficult or easy to relate the online portfolio as user-friendly?
Difficult 1 2 3 4 5 Easy
5.) What feature appeared most professional on the homepage of the website?
6.) What feature appeared the least professional on the homepage of the website?
7.) What feature would you change on the webpage to make it appear more professional?
Usability Test Report 23
Usability Test Consent Form
Please read and sign this form.
During this usability test:
You will be asked to perform certain tasks on a computer.
We will also conduct multiple surveys with you regarding the task you performed during the test.
Participation in this usability study is voluntary. All information will remain strictly confidential.
The outcomes and finding may be used to help improve the jesselindsey.weebly.com website.
Further, this is an academic study to be used for academic purposes. All research methods will
adhere to strict ethical guidelines. At no time will your name or any other identification be used.
You can withdraw your consent to the test and stop participation at any time.
If you have any question after today, please contact me at: [email protected]
I have read and understood the information on this form, and had all of my questions answered.
Participants signature:
Usability Test Report 24
Checklist for Moderators
1. How are you doing today?
2. The weather is beautiful/dreary isn’t it?
3. Did you walk, take public transportation, or drive today?
4. What is your current vocation?
5. Are you planning on completing the usability test?
6. Have you ever personally created a portfolio?
7. Have you ever helped anyone create a portfolio?
8. Have you ever participated in a usability test before?
9. If so, how many? What was the test objective, etc.?
10. Are you excited to participate in today’s usability test?
Usability Test Report 25
Other Pertinent Information
There are no cameras, microphones, one-way mirror, or so forth used in this usability test. There
will be no observers, only test administrator. We encourage participants to ask question, but
discourage thinking-aloud procedures. We will not require participant to acknowledge when they
are finished with each task, but ask that they notify test administrator when all task is complete.
Besides for instructional information, administrator and participants will not interact during the
usability test. Notes will be conducted upon completion of the usability test.