622 Usability Testing

1

Usability Testing Report for ScrapWalls.com

Created by Group 14 Jodee Jernigan Ben Mullins Cassandra Palmer Created for: Joe Golden, ScrapWalls.com Word count: 4947

2

Executive Summary Usability testing is a method that tests actual users of a system. We conducted user tests with 5 participants, all women over the age of 25, who are new users to ScrapWalls.com. These user tests resulted in several new findings as well as reinforced findings from our previous reports. The new findings include: ● The photo upload process takes some time

Recommendation: Include thumbnails during upload to inform of progress ● The “Border” option presented some difficulty for users

Recommendation: Move the placement of the Border color option after the Border selection option and consider alternate verbiage

● Users often did not scroll down to see options that were too far down Recommendation: Include all important info near top of screen

● When users upload new photos for an additional collage, photos from their old collage are automatically selected for inclusion Recommendation: offer users a bulk delete option

● Users expressed they wanted more control in the collage creation process Recommendation: Add more user control in the placement of pictures within the collage.

● Some users wanted to delete photos from the collage by dragging the photo away Recommendation: Add an additional option for deleting the photos from the collage by moving the photo off the collage outline.

The reinforced findings include: ● Users cannot easily share their collages on Facebook

Recommendation: Include a Facebook Share button that is easy to see and understand

● Shipping information is hard for users to understand Recommendation: Streamline and clarify shipping information

● Users who have created multiple collages could not identify the collage Recommendation: Use thumbnails of collage in shopping cart and “Your Projects” to represent collages.

● Users were confused by the duplicate nature of the Products and Prices tab Recommendation: Streamline these two tabs to just one

3

Introduction ScrapWalls.com is an online web service that allows users to create photo collages by choosing from over 70 different collage shapes. Users can upload pictures from their hard drive for use in the collage, and they can also import photos from their Facebook account. ScrapWalls conveniently organizes the photos to fit inside the shape you choose, and can do this with anywhere from 2 to 300 photos. When creating a collage, ScrapWalls gives you flexibility to modify the border, title, orientation, or placement of the photos. Users can then share the collage on Facebook for free, as well as order a printed copy. ScrapWalls is meant to be a fun and convenient way for anybody to share or print their photos in a unique presentation. (Jernigan, Mullins, and Palmer, 2012a) Over the course of the semester, our team has evaluated the ScrapWalls.com site using several different methods: interviewing potential users of the site, creating personas and scenarios, competitive analysis, heuristic evaluation, and surveying existing users of the site. These various methods have uncovered many key insights into how users interact with ScrapWalls.com and possible usability issues they may encounter. No method is quite as thorough, however, as conducting usability test with actual users of the system. For this study, we conducted an in-‐depth usability test on 5 users, who were all new to using ScrapWalls.com. These test users were given 4 tasks to complete so that we could capture the most common functions on the site. Usability testing enabled us to confirm some of our previous findings as well as uncover additional usability issues we had not anticipated. From these findings we have made recommendations in order to help ScrapWalls.com improve their user experience. Our goals for for conducting this study included the following key questions we wished to answer: ● How does a new user initially react to the site and understand its main

functions? ● What are some of the main problems a real user may encounter while

creating and editing a photo collage? ● Are users able to easily share their completed collages?

4

Methods Goals Our previous studies, including heuristic evaluation and customer surveys, allowed us to make several findings about ScrapWalls’ general user experience and to uncover several potential usability problems. However, in order to confirm our findings and also uncover additional usability issues we did not anticipate, we needed to test actual users of the system. Our goal was to evaluate how a new user to ScrapWalls would navigate the system and perform common tasks and functions. Through direct observation of the user experience, we sought to evaluate the system’s overall usability, functionality, appearance, and ease of use. Methodology The first step of the process was identifying key functions of the site and designing task scenarios that would allow us to observe how actual users carry out those tasks. We also identified our target population for the user test. We then created pre-‐ and post-‐test questionnaires in order to assess user demographics and also assist us in our final de-‐briefing interview. A pilot user test with a close friend was deployed so that we could improve wording of our tasks as well as improve overall process flow for our actual users. We then recruited actual users based on our target population needs. Before they test, we secured appropriate equipment as well as a suitable physical environment in which to conduct the tests. As a team, we met after each user test and discussed and analyzed the key areas where users most had difficulties, and then later synthesized these findings for this report. More details for each of these steps can be found below. Task Generation We first identified the main functions that be accomplished at ScrapWalls.com, which include collage creation, collage editing, purchasing a collage, and sharing a collage. We designed task scenarios which involved each of those key site functions. We also asked our client about various features of the site in which he sought feedback, and we included that feedback in our task creation. We also revisited key findings from our previous studies, including heuristic evaluation and customer surveys, so that we could test various potential usability issues and confirm how users respond. We developed 4 user tasks in total. The first task was the most involved task and included several steps. Subsequent tasks were shorter in duration so that we could avoid user fatigue during the test. The first task involved creating a collage and uploading photos from the user’s desktop. The second task involved editing and purchasing a collage. The third task required users to upload photos from Facebook

5

and do more editing. The final task involved asking the user to share the collage, and tested their ease either sharing on Facebook or via an email link. Complete task descriptions and scripts can be found in Appendix A. Questionnaires In order to assess participant’s comfort with technology level and also to discover their equipment and browser needs, we created a Recruitment Questionnaire. Because the client was eager to discover users’ initial impressions of the site, we created a Pre-‐Test Questionnaire, which asked the user to look at the home page and answer a few questions regarding the basic purpose of the site and overall first impression. After the user completed all four tasks of the usability test, we also administered a post-‐test questionnaire, which asked the user general feedback questions about the site, as well as specific questions pinpointing any difficulties the user may have encountered during completion of the tasks. This post-‐test questionnaire was helpful in guiding the moderator through the de-‐brief session and to ask further follow-‐up questions based on the user’s responses during the test and given on the post-‐test questionnaire. Questionnaires can be found in Appendix B. Participant Selection and Recruitment We identified potential participants for our usability testing based upon the target population of the site. Our previous survey study indicated that a preponderance of ScrapWalls users are likely female, over the age of 25. (Jernigan, Mullins, Palmer, 2012d) Using this data, we decided to recruit users who also met this criteria. We also sought users who lacked training in usability research and methods, so we could get a lay-‐person’s opinion of the site. Potential participants were also screened out who had already had experience using the system, as we wanted to capture first impressions of the site. Because the target population is relatively broad, we were able to easily recruit participants within our respective social networks, including fellow SI students and friends of friends. A brief summary of the participant demographics: 37, Female -‐ Grad student 32, Female -‐ Grad student 33, Female -‐ Social worker 41, Female -‐ Grad student

6

34, Female -‐ Grad student Participants were offered various incentives for their time, either a Starbucks gift card or for personal friends of the group, a future lunch date treat or happy hour drinks. Physical Set-‐up In order to make users comfortable, we offered both PC and Mac for testing equipment, and asked them about their preferences before the test. We also offered a wireless mouse for those participants who were not comfortable using trackpads. Users were also able to choose the browser they were most comfortable using in order to avoid any usability issues outside of the ScrapWalls environment. In order to test users in a comfortable environment and to minimize distractions, we scheduled the usability tests in University of Michigan School of Information project rooms, which offered a closed environment. The user tests were captured using two different methods. The user test laptop was hooked up to the large monitor screen in the project room, and we filmed the screen via camcorder. This also allowed the note-‐takers to have a clear view of the user’s activity while performing the task. We also recorded the user session via the laptop web-‐cam using Camtasia software. These two recording methods, along with written notes, were an important safeguard against possible lost data during equipment failure. (See Discussion section for more details regarding an instance of equipment malfunction.)

Testing Layout

7

Prior to the test, we loaded photos needed for the test in a Folder on the desktop. Additionally, we logged in to Facebook so that the user could access a set of Facebook photos during the test and also access the Facebook connect sharing feature. All three group members were present for the user tests (with the exception of User Test 3). One group member served as moderator, and the the other two members both took notes using Logging Forms (with the exception of User test 3 which had only one note-‐taker.) The testing session began with making the user comfortable at the testing station and the moderator reading the preamble script. We then explained the consent forms and had the user sign indicating their permission to be recorded for the test. The moderator preamble script can be found in Appendix C and a copy of the consent form appears in Appendix D. Analysis Sessions Once all five user testing sessions were completed, our group met in order to review the tests and aggregate the results. The note-‐takers also combined their handwritten notes and transcribed the results for inclusion in the report. (Copies of the Logging Forms and Notes from the Usability Tests can be found in Appendix E and Appendix F respectively.) From these analysis sessions, several usability findings emerged based on common problems encountered by multiple users. We also were able to compare our findings with findings from our previous studies to notice any common results. This analysis enabled us to formulate the key findings and recommendations in this report.

Findings and Recommendations

Summary Results In general, ScrapWalls.com is an extremely usable site and users were able to successfully complete the tasks with very few errors. However, we witnessed several issues that could be improved in order to make the user experience even better. Because we were able to discover new usability issues, in addition to the previous usability issues we had uncovered, we decided to separate our findings into two distinct categories: new findings and reinforced findings. The new findings include: ● The photo upload process takes some time, and users wished they could see

the thumbnails loading while they wait ● The “Border” option presented some difficulty for users, both in

understanding the label Border as well as confusing placement for the option

8

● Users often did not scroll down to see options that were too far down on the screen

● When users upload new photos for an additional collage, photos from their old collage are automatically selected for inclusion

● Users expressed they wanted more control in the collage creation process ● Some users wanted to delete photos from the collage by dragging the photo

away The reinforced findings include: ● Users cannot easily share their collages on Facebook ● Shipping information is hard for users to understand ● Users who have created multiple collages could not identify the collage based

on the thumbnail ● Users were confused by the Products and Prices tab which both direct to the

same page

Key Findings and Recommendations New Findings and Recommendations Finding 1: The photo upload process takes some time, and users wished they could see the thumbnails loading while they wait.

Upload In Progress Window In task 1 we asked the users to upload a photos that were preselected from a file that was arranged for them. In total there were 47 pictures to upload and because of the Internet connection, uploading this number of pictures took longer than anticipated. During this time many users expressed that out of habit they would

9

probably be doing something else while they waited such as looking at other web-‐pages. They also wished they could somehow see their progress. Recommendation for Finding 1: Add thumbnails of photos as they are in the process of uploading. Since the test users were not able to surf other web-‐pages during the upload time, we posed several questions to them, one of which was if there was something else they would like to see while they are waiting for an upload. One user expressed how she would like to stay active within the collage making process, so perhaps knowing ‘next steps’ after their photos were uploaded would be helpful. To go along with that sentiment, one user expressed how they would actually like to see each picture as it uploaded. This particular user had also used Snapfish before and mentioned how this is a feature at that site. The user likes this feature because they could see the pictures as they uploaded and if there were pictures that they did not want uploaded, they could stop the process immediately and correct their pictures selections. Finding 2: The “Border” option presented some difficulty for users.

Collage Border Controls Users were confused by the “Border” option, and thought that it meant a border around the entire collage, instead of between the photos. Also, the placement of the Border color option before the Border selection option caused errors because color could not be selected first. For example, a user would indicate a border color first because it was the first option for border, but nothing would happen until the user would later select Border size (because “No Border” is the default option.)

10

Recommendation for Finding 2: Reorder the border options and consider clarifying the verbiage. A slight reordering of the border options could make things less confusing for the user. By having the user select the border width first and then selecting the color the user would immediately see the border they have chosen. Also, indicating that the border is strictly for the pictures and not for the entire collage would be helpful for some users who are more familiar with a border representing an entire collage. Changing the verbiage to “Border Around Photos” would be more clear to the users. Additionally, ScrapWalls may want to consider adding a “Border Around Collage” feature to their collage options as many users expressed interest in that feature. Finding 3: Users often did not scroll down to see options that were too far down on the screen. During our user tests, we noted many participants did not always scroll all the way down to see important options. This was apparent during the test when users did not see the option for naming the collage, which was at the bottom of the screen. This issue also cropped up when users selected the shape of their collage; although there is small Continue button at the far top, users choosing a shape in the middle of the screen did not see the that or the larger Continue button at the bottom. Recommendation for Finding 3: ScrapWalls should make sure that all vital selection options are presented in the top-‐half of the screen. Paying attention to this recommendation would also make ScrapWalls more usable for people who use netbooks and smaller laptops as these devices have even less screen space than the laptop we used for testing. Finding 4: When users upload new photos for an additional collage, photos from their old collage are automatically selected for inclusion and deleting these unwanted photos was cumbersome.

11

Photo Selection Window In both task 1 and 3, users are asked to create a collage. In task 3, users are asked to create a new collage with a different set of pictures than from task 1; however, when users go to upload the new set of pictures, the pictures from the previous collage were still there, and selected for inclusion in the new collage by default. In particular, there were about 47 pictures that were unwanted. Many of the users openly complained about whether or not they had to delete each of these photos one by one. While this is a helpful feature for users who would like to re-‐use these photos in another collage, it was a cumbersome process for a user creating a brand-‐new collage. Recommendation for Finding 4: ScrapWalls should implement a bulk delete option similar to their bulk uploading feature Since there were 47 unwanted pictures, many users wanted a way to delete a grouping of photos rather than delete them one by one. During the user testing we noticed that many of the users tried to do a ctrl+shift to highlight a grouping or photos, similar to they way they would select a group of photos to upload. This feature would be much appreciated by users who may need to delete large quantities of pictures.

12

Finding 5: Users expressed they wanted more control in the collage creating process. After users had uploaded the pictures for the first collage they created in Task 1, all of the users expressed delight when they saw that ScrapWalls automatically generates the photos into the shape that they choose. Some of the users indicated that to them, the idea of a photo collage meant the would be placing and arranging the photos manually. Although they appreciated the photos being arranged in the shape for them, they also indicated how they wanted more control over where the photos were placed. Currently, moving a photo from location A to location B results in the two photos automatically swapping locations. Recommendation for Finding 5: ScrapWalls should consider adding more user control in the placement of pictures within the collage. Simply swapping the pictures could become too labor intensive and confusing for the users. Instead of just swapping the location of two pictures, ScrapWalls could add in the ability for the user to place the pictures in a specific order. Finding 6: Some users wanted to delete photos from the collage by dragging the photo away.

Assembled Collage In Editing Window

13

Through the discovery process of using ScrapWalls the users noticed how when you hover over the collage area, you can see the outline of pictures they may be cut off by the collage. In task 2, users were required to delete a specific picture within the collage. Multiple users, when prompted to delete the picture, decided to try and drag the picture outside of the visible picture area of the collage in order to delete it. We can infer that these users may have thought moving the picture entirely outside of the collage shape would make it disappear, similar to the way certain parts of a picture may not be visible if they are outside of the collage shape. Recommendation for Finding 6: Consider adding an additional option for deleting the photos from the collage by moving the photo off the collage outline. Although it may have been easy for some of the users to notice the ‘x’ mark which is used to indicate deletion of the photos, ScrapWalls must also consider the user that defaults to their heuristics when navigating to the site for the first time. Adding this feature may be helpful for some users.

Reinforced Findings and Recommendations Finding 7 (from Heuristic Evaluation report): Users cannot easily share their collages on Facebook and do not understand they can share the collage by clicking the small “Like” button. From our Heuristic Evaluation Report (Jernigan, Mullins, Palmer, 2012c), we indicated that users may have difficulty when trying to share their collage through Facebook. This was confirmed in our user test in Task 4 when we had the users try to share their collage. Many users did not see the small Facebook “Like” button. Additionally, the Facebook “Like” button was not a clear indication that they could share the collage through Facebook; rather they thought if they clicked the button they would be “liking” ScrapWalls on Facebook. Although there is some instructions displayed on how to share their collage, it is far away from the Like button and most users did not see it. One user indicated that they completely ignored the information because it is located in an area (right side) where they are accustomed to seeing advertisements being displayed, not useful information. Recommendation for Finding 7: On the Share screen, ScrapWalls should include a Facebook Share button that is easy to see and click. By calling the button “Share on Facebook”, it would be more clear to the user than using the “Like” button. ScrapWalls should also offer instructions regarding the share icons, and we also recommend they put that information in an area within

14

proximity of the button where it will be noticed and not confused with what the user may consider clutter such as advertisements. We feel this is an important recommendation as sharing photo collages on Facebook is a prime function of the site and would also serve to drive new users to use ScrapWalls based on word of mouth. Finding 8 (from Comparative Analysis report): Shipping information is hard for users to understand. From our Comparative Analysis report (Jernigan, Mullins, Palmer, 2012b), we indicated that users may find ScrapWalls shipping information difficult to understand. This was confirmed in our user test in Task 1 where we had the user first look up the shipping information. The user indicated one price, but when they actually created the collage and added it to the cart, it ended up being a different price than they expected. In 5 user tests, only one user accurately predicted the final shipping price for the item. Recommendation for Finding 8: ScrapWalls should look at it’s competitor site, PosterMyWall.com, to get an indication of how to make their shipping information more clear. Finding 9 (from Heuristic evaluation report): Users who have created multiple collages could not identify the collage based on the thumbnail .

Blank Project Thumbnails From our Heuristic Evaluation report (Jernigan, Mullins, Palmer, 2012c), we indicated how using the blank thumbnail of just the shape of the collage within ‘Your Projects” and the shopping cart could be difficult for the users to identify. This was confirmed in Task 2 when users were required to choose between two collages with

15

the same shape but different details. Most users had to look at the timestamp to try and decide which collage was the one they wanted to add to the shopping cart. Recommendation for Finding 9: ScrapWalls should consider adding a small thumbnail of the actual collage itself rather than just the collage shape which can help users to easily identify their project. Finding 10 (from Heuristic Evaluation report): Users were confused by the Products and Prices tab which both direct to the same page. From our Heuristic Evaluation report (Jernigan, Mullins, Palmer, 2012c), we indicated how the Product and Prices tab on the ScrapWalls website go to the same page. The user in our tests also noticed it and were confused. Recommendation for Finding 10: ScrapWalls should consider combining the Products and Prices to one tab to avoid confusion from the user and streamline user options.

Discussion We were able to easily recruit users for our test based on the somewhat broad demographic of women over the age of 25. However, all of our test subjects were working on or have completed master’s degrees which might have added an educational and technological bias to our results. Although we screened out any participants who are overly familiar with usability testing and research, another study including less educated and less technologically savvy women might have yielded further findings. The fact that our test population was users that had not heard of ScrapWalls was helpful in gauging first impressions and how intuitive the site was. However, this also compromised aspects of the test. Our users engaged in collage-‐creation tasks without necessarily knowing what ScrapWalls does. Actual users of ScrapWalls would likely have a better idea of what the site does prior to creating a collage. There was a benefit to this compromise, however. It revealed that users require some time in order to understand what ScrapWalls does. Design changes may be able to accelerate this process. Although most of our user tests ran smoothly, we did encounter an equipment problem during User Test 5. The test computer froze during the photo upload process and the computer needed to restart. This resulted in a long wait time during

16

the test to re-‐set, and the user had to start the task over again. Fortunately this did not seem to affect the results as the tester was able to eventually complete all the tasks successfully. We unfortunately lost our Camtasia recording for this test, although we did still have the camcorder footage and notes. For our photo upload task, we might have included too many photos for the the test subjects to upload. There were 47 high-‐resolutions photos in the test folder and this process always resulted in a long wait time for the users. We did try and make use of this wait time, by asking a few mid-‐point de-‐briefing questions about any difficulties in the process thus far. This allowed the participants to continue talking about their impressions of the site while they waited for the upload to complete. This wait time is also representative of what a real ScrapWalls user might experience, so it generated helpful information despite the lull in active testing. The photos that were uploaded were gathered prior to the test, and our tasks made it clear where the photos were located. This may not be representative of a customer’s collage-‐creation process, as they may not have an exact idea regarding which photos to include, or where they are located on their own computer. Because we allowed users to select their preferred browser for their test, there was a consistency issue regarding the amount of display space offered by the browser chosen. Firefox users had slightly less screen area to work with due to space taken up by toolbars and add-‐on controls. Based on the results of the tests, this did appear to cause any noticeable variation, but the potential was certainly there. We did have findings related to users preferring options available on-‐screen instead of scrolling to find potentially more appropriate options, but this occurred in every browser. There was a concern that the instructions within a task revealed too much about ScrapWalls’ features and how a task could be performed. By asking users to add text to their collage, we made it apparent that adding text was an available feature, providing cues that would not exist for real site. We could still asses the presentation of these features based on how quickly a user found the feature, and the confidence expressed as they attempted to use it.

Conclusion For this study our team conducted a usability test with five participants who were new to using ScrapWalls and who fit the target demographic. We generated tasks based on the key functions of the site that incorporated findings from our previous reports as well as input from our client. We explained the process to the participants, who then completed a pre-‐test questionnaire and performed four tasks on the site. The users then completed a post-‐test questionnaire and we de-‐briefed them on their experiences. The tests were captured with Camtasia software, a camcorder on a test monitor, as well as notes taken by the team. The results from this usability study allowed us to generate new findings as well as confirm previous

17

findings. We have generated new recommendations based on these newly discovered findings which include improving the photo upload process via a new bulk delete option and incorporating thumbnails, and improving user controls for things like Borders and picture removal. We also discovered that users often do not scroll down to access options and suggest ScrapWalls take this into consideration. Usability testing confirmed several of our previous findings and recommendations. These confirmed recommendations include improving the Facebook share option to make it easier to understand and find, clarifying shipping information on the site, streamlining the duplicate Products and Pries tab, and identifying collages in the cart with thumbnails to make them easier to differentiate.

References

Jernigan, J., Mullins, B., Palmer, C. (2012a). “An Analysis of Personas and Scenarios for ScrapWalls.com”. Jernigan, J., Mullins, B., Palmer, C. (2012b) “Comparative Analysis for ScrapWalls.com”. Jernigan, J., Mullins, B., Palmer, C. (2012c) “ScrapWalls.com Heuristic Evaluation Report”. Jernigan, J., Mullins, B., Palmer, C. (2012d). “Customer Survey Report for ScrapWalls”. User 1. (March 24, 2012). In-‐person usability testing. User 2. (March 27, 2012) In-‐person usability testing. User 3. (March 31, 2012) In-‐person usability testing. User 4. (April 3, 2012) In-‐person usability testing. User 5. (April 4, 2012) In-‐person usability testing.

Appendices (Begin on next page) Please see the attached appendices for further information which was used in the generation of this report. Appendix A -‐ User Tasks Appendix B -‐ Blank Recruitment, Pre-‐Test and Post-‐Test Quesstionnaires Appendix C -‐ Moderator Script/Preamble Appendix D -‐ Consent Form Appendix E -‐ Logging Forms Appendix F -‐ Notes from Usability Tests

18

Appendix A -‐ User Tasks

19

20

Appendix B – Blank Recruitment, Pre-‐Test, and Post-‐Test Questionnaires

21

22

23

24

Appendix C -‐ Preamble

25

Appendix D -‐ Blank Consent Form

26

Appendix E -‐ Logging Forms

27

28

29

30

Appendix F -‐Notes for Usability Tests Test Number: 1 Date March 24, 2012 Task 1 Start Time 2:46 End Time 2:58

Completed? Yes Notes on task 1 -‐>User misunderstood the question and could not find the

shipping information, but was able to recover -‐>User commented upload took a long time. 2:55pm -‐> “Oh wow! It does it for you” referring to the collage

Task 2 Start Time 3:00 End Time 3:03 Completed? Yes

Notes on task 2 -‐>Wondered where to find title (had to scroll down). -‐>Selected Border color nothing happened -‐ had to select size first. -‐>User commented there are lots of collage choices to choose from -‐>Shipping was more expensive than the user anticipated.


Notes on task 3 -‐>Could not see how to add photos to collage. -‐>When creating a new collage they added pictures to an existing collage -‐>User was able to resize the pictures


Notes on task 4 -‐>User could not find the share now button, had to scroll down…later on finds it -‐>User could not find the Facebook Like button to share, eventually saw the directions on side but took almost 5 minutes.

Debriefing Notes: -‐>User would have liked more introductory info on main page. -‐>She suggested the site offer a stock photos for inclusion in your -‐>User would have liked some window to tell your progress while uploading. -‐>Did not know she needed to scroll down to see some options. -‐>User did not know the photos would populate in the collage by itself. She wanted to drag and drop the photos manually. “If you have an idea how to do your collage, this automatic method would throw you.” -‐>“Borders were ugly.” -‐>User expected a cropping feature for the photos. -‐>Wanted more control over individual pictures. “-‐>Share Now” button too low and results after clicking were confusing

31

Test Number: 2 Date March 27, 2012 Task 1 Start Time 12:17 End Time 12:32

Completed? Yes Notes on task 1 -‐>User commented on finding shipping prices before having to

invest time into making collage -‐>12:20 Almost did not noticed the side navigation of the collage shapes -‐>The categories of the shapes were confusing -‐>Likes how you can upload pics from facebook. -‐>Commented on long photo upload. -‐>12:26 Wasn’t sure what to do after the upload -‐>Wasn’t sure how the final collage would look and thought that they had to create the shape of the collage -‐>Surprised she did not get to manually control where the photos appear. -‐>Grabbed image to move off-‐screen. -‐>Did not like how it told me changes could not be undone. -‐>“This is cool but I am not sure why I would use it” -‐>Commented on the pictures being tiny


Notes on task 2 -‐>User commented it’s not very clear you can make changes through the profile/cart…would prefer an edit button -‐>User found that the top bar of the navigation could blend in with the web browser décor -‐>12:37 “Look there is a zoom button!” -‐>User confused about adding text because the feature is called title


Notes on task 3 -‐>User did not notice the different products tab -‐>User commented there is not a high learning curve -‐>Does not like how the it defaults to the previous project that was created when you are trying to create a new one. Wondered if there was a faster way to remove unwanted photos from the project. ‘If I delete them, can I use them later?’

Task 4 Start Time 12:48 End Time 12:55 Completed? yes

32

Notes on task 4 -‐>User commented there was no sign indicating how to change the shape…decided to click on home because they felt they were no longer editing a project, but creating a new one. -‐>User was wondering what quality the photos were going to be Found the “like” button to be confusing “Am I liking scrapwalls or the collage I created?”

Debriefing Notes: -‐>Navigation needs to be improved. -‐>I need more control over what I am creating. -‐>I need better cues for what I should do. -‐>The end product looks generic.

33

Test Number: 3 (NOTE: On these notes, time starts at 0:00, and all times count up from there in min:sec format)

Date March 25, 2012


Notes on task 1 1:15 -‐> (sees shipping price info)User: “Ugh, This looks too complicated” 2:02 -‐> Missed additional canvas-‐specific shipping charge, just added ground price 2:24 -‐> Used “start now” button to begin collage 3:52 -‐> After seeing instructions to add photos, was not sure how to access that feature. Based on not seeing any better options, correctly guessed clicking the “continue” button. 3:58 -‐> Thought the “create a new album” naming option meant the collage name did not stick. U03: “I’m concerned it won’t keep what I’ve already put in [the collage] 4:22 -‐> User is not sure if this long of an upload time is normal. -‐> Remaining portion completed effortlessly

Task 2 Start Time 11:55 End Time: 16:49 Completed? yes

Notes on task 2 11: 59 -‐> Instinctively (mistakenly) goes to cart to access collage to be edited 12:30 -‐> Figured out collage could be edited by going to projects 12:50 -‐> Was not sure how to add text to collage, chose title as best possibility, but not confident in choice. 13:57 -‐> Was disappointed to see “border” meant around photos, and not the entire collage 15:03 -‐> Had trouble figuring out how to remove the proper collage from cart. Used datestamp to figure it out, but said she’d have preferred thumbnails of the collage to reference.


Notes on task 3 21:15 -‐> is skeptical that collage name will stick 22:30 -‐> figures out distinction between collage and album name 23:00 -‐> there is a lack of visible feedback to indicate facebook upload has started. User is concerned it is not working properly 24:01 -‐> browser returns to scrapwalls after a minute with no updates. States that she would have quit the process had it not been a test environment

34

25:35 -‐> is confused by photos from previous collage being active in this new collage, is not sure if the Facebook photos were actually added. 26:30 -‐> is not pleased with the possibility of having to remove each photo from earlier collage. 26:50 -‐> annoyed that she has to delete each photo individually. 30:23 -‐> concerned that changes to the collage are not saved without adding to cart or sharing.


Notes on task 4 32:50 -‐> Tries URL share first when trying to Facebook share. Tries to figure out how to Facebook share. Waits for loading to finish 44:30-‐> takes a minute of trying to figure out Facebook share, tries ‘like’ button as closest match, but is very skeptical It works, but user is critical that it seems like it is a ‘like scrapwalls’ button, not to share a collage

Debriefing Notes: -‐>Did not like borders being around photos, or the word ‘border’ describing that. -‐>Really thought Facebook share button was poorly worded -‐>Site felt professional -‐>Was hard to know what changing something in a collage might result in, but thought that might be features good for someone in a hurry -‐>Did not feel ‘lost’ in any task aside from Facebook like button

35

Test Number: 4 Date April 3, 2012 Task 1 Start Time 12:14 End Time 12:24

Completed? Yes Notes on task 1 -‐>User double clicked on shape and did not noticed the continue

button. “I thought something would pop up when I clicked the shape.” -‐>User commented that it gets all the photos to upload like on Snapfish. Snapfish however, shows you each photo that you are uploading. -‐>“It makes it for me!” referring to the collage -‐>“I am wondering what kind of image quality -‐ Snapfish lets you choose and see the quality.”


Notes on task 2 -‐>User thought that the border was for the entire collage, not just around the pictures. -‐>“This is so much fun!” -‐>User commented they wish they had a change to rename the collage because they couldn’t tell which collage was the one they wanted. User used the timestamp to select right collage.


Notes on task 3 -‐>User wanted more confirmation that they made a correct choice, they want to see the size of the collage they choose. -‐>Did not like how they had to delete each photo one by one. “I don’t like this” -‐>Commented there is no warning for low resolution photos.


Notes on task 4 -‐>“Hmm, how do I change the shape?” -‐>User wants to know if the photos are low resolution before they are loaded onto the collage.

Debriefing Notes: -‐>Well-‐done site. Sophisticated -‐>I wish I had the ability to see a thumbnail of the photos as they upload. -‐>I wish I could see if pictures are low-‐res when they are uploaded.

36

Test Number: 5 Date April 4, 2012 Task 1 Start Time 4:38 End Time 5:05 (includes

system crash/restart time)

Completed? Yes Notes on task 1 -‐>User was looking for the canvas product

-‐>User was trying to get back to the pricing “How do I get there?” -‐>User not quite sure of what the shipping prices would be -‐>“It’s odd the states are not in alphabetical order” -‐>During the photo upload portion user wanted to see the photos that were being uploaded so they could get a headstart with their collage making process -‐>***technical difficulties***


Notes on task 2 -‐>“I am not sure if I made a collage” -‐>User was not sure how to add text to the collage since the feature is called “title”…Also they were trying to see if a dynamic text box appears by clicking on the collage itself -‐>User was confused about what a border is. Border color clicked first, but nothing happened. “I am unclear if I have added a border”. -‐>User found it odd that the shopping cart defaulted the values to 1 -‐>Confused about the gift code vs the discount -‐>User having difficulty finding the remove button in the shopping cart


Notes on task 3 -‐>“I think it is a bit weird that the box is so far down to name -‐ have to scroll down for it.” -‐>User was not anticipating the fruit pictures would be in the collage -‐>“I am not sure what photos are low-‐res” -‐>There should be a way to batch delete photos -‐>In the heart shape collage, some of the photos are not visible. “Would like more control.”

37


Notes on task 4 -‐>Preferred to have shapes a ‘Collage option’ rather than having to go to back to the beginning -‐>User made a confused face when trying to use the share options. User said it was not what the expected -‐>User thought that ‘liking’ was referring to scrapwalls rather than sharing a collage on facebook.

Debriefing Notes: -‐>I expected Border to be a border around the whole collage, not around the pictures. -‐>Sharing options were confusing -‐ directions were off to the side, too far away from the buttons. Liking on Facebook is not what that usually means. -‐>You can’t manipulate the collage as one -‐ just flopping things around.

38

622 Usability Testing

Documents

users recommendation

collage recommendation

option users

new users

actual users

existing users

potential users

additional collage