Final Report for CUTGroup #24 - OpenGrid

CUTGroup #24 OpenGrid

Report by Nathalie Rayter & Sonja Marziano Smart Chicago Collaborative

CUTGroup #24: OpenGrid In-person test on Wednesday, April 20, 2016

Introduction For our 24th Civic User Testing Group (CUTGroup) test, we tested OpenGrid, an open-source interface developed by the City of Chicago that allows residents to search for, interact with and visualize City of Chicago’s datasets.

“OpenGrid has an intuitive user interface that’s highly responsive and efficient. The application is accessible from any devices (e.g., mobile, laptop, tablet or desktop). It’s unique, functional, and an easy to use tool to find city data. It extracts city data and displays it in an illustrative form which makes it easier for a user to search and understand the information.” - City of Chicago Department of Innovation & Technology staff

Target Group On April 12, we sent out an email to 1,132 CUTGroup testers who lived in Chicago. We wanted to know if they would be available for an in-person test on April 20. When segmenting our testers, we were interested in:

● Testers who live within Chicago city limits because OpenGrid specifically shows City of Chicago data

● A mix of testers with different types of mobile devices (Android/iPhone) ● Testers who had varying degrees of familiarity with what happened in their

neighborhoods ● Testers who did or did not use datasets in their personal and/or professional lives

Test Type ● In-person test component where testers would view different features on their own

device and complete tasks ● This test utilized A/B testing, where 13 of the testers (“A” Testers) began on

http://opengrid.io/ (OpenGrid’s homepage) and the other 10 testers (“B” Testers) on http://chicago.opengrid.io/opengrid/ (the app interface).

● All testers were paired with a proctor, who asked them questions, observed and recorded their interactions with OpenGrid and their responses and feedback.

Goals of Test The main goal of the OpenGrid test was to measure whether Chicago residents are able to find the information that they are looking for and whether it is easy to navigate datasets. Past usability testing of the interface had been with highly skilled users or people who work at the City of Chicago. We also wanted to learn how residents might use the information and data displayed on OpenGrid in their personal or professional lives and if they find value in OpenGrid.

2

http://opengrid.io/

http://opengrid.io/

http://chicago.opengrid.io/opengrid/


Responses 23 testers completed the OpenGrid CUTGroup test. In Group A, 8 testers used laptops, 3 used smartphones, and 2 used tablets. In Group B, 5 testers used laptops and 5 used smartphones.

Can you tell us more about your experience, if any, using data or datasets?

Group A 3 testers described proficiency in accessing data in their professional and/or personal lives. 7 testers mentioned they had some familiarity with data and/or using the internet to find more information about their neighborhoods. 3 testers gave responses that indicated they had previously had little to no exposure to using data.

Group B 2 testers described proficiency in accessing data in their professional and/or personal lives. 6 testers mentioned they had some familiarity with data and/or using the internet to find more information about their neighborhoods. 2 testers gave responses that indicated they had previously had little to no exposure to using data.

Review the homepage and tell us about the three things that you notice first.

Group A Group A testers started on OpenGrid’s homepage (http://opengrid.io/). Here are A testers pointed out when reviewing the homepage (number indicates number of times a topic of feature was mentioned):

6 - “Launch” 6 - Background image 3 - “Code” 3 - Headings 2 - Contact Us 2 - Quick Search 2 - Video 2 - Informational text 2 - Expressed confusion 1 - Managed Queries 1 - Symbol 1 - Open Source

3

http://opengrid.io/


1 - Data 1 - “Customizable” 1 - City of Chicago

Group B Group B testers began on the app interface (http://chicago.opengrid.io/opengrid/). Here are what B testers pointed out when reviewing the app, presented to them as the homepage:

4 - Map 3 - “Queries” 2 - Features on map 2 - Quick search box 2 - Colors 2 - Menu 1 - Absence of call-to-action 1 - Submit button 1 - OpenGrid logo 1 - “Commonly Used Queries” 1 - Side menu 1 - Map icons

If you had to describe what this website does, what is your best guess?

Group A After reviewing the homepage, the majority of Group A testers mentioned one of these four topics: data, map directions for getting to places, information about places on the map, and exploring neighborhoods. Here are key responses:

Charlie (#A9) says, “It's like Google Maps - put in your location, and it will give you directions, information about what's on the map.” Bronco (#A15) says, “[It’s for] Encouraging the developer community to develop application from using the data on this website. These tools would be for residents.” JG (#A14) says, “I probably wouldn't have an idea - "OpenGrid" isn't telling much, unless you start scrolling down - and even then, 'exploring neighborhood' is vague.”

Group B After reviewing the app launch page, the majority of Group B testers mentioned one of these two topics: map directions for getting to places and information about places on the map. B

4

http://chicago.opengrid.io/opengrid/


testers did not have the contextual information provided by the OpenGrid homepage, which proved to be valuable to the testers in Group A. Here are key responses:

Make Believe (#B1) says, “[It’s for] traffic or how to get around city quicker.” Paloma (#B8) says, “Traffic or how to get around city quicker.”

Who do you think this website is targeted to? Why?

Group A 7 testers said that they thought the site was either for Chicago residents. Here are some key responses:

Charlie (#A9) says, “Focus is on travelers, drivers, bicyclists, public transit riders - it may help these people know where they're going without asking other people.” Heather (#A11) says, “Chicagoans who want to know more about their neighborhood.”

3 testers indicated that they thought the OpenGrid was targeted toward developers.

Chris (#A16) says, “Anyone who has interest in viewing the data; developers that would like to integrate this into other projects.”

1 tester indicated uncertainty of who the site’s target audience is; another tester, Ms Boop (#A6), says it’s for “anyone that can get online.”

Group B There was much less consensus about who the target audience is among testers in Group B, who mentioned a general audience, people moving through the city, residents, investors, and government employees as possible audiences. Here are key responses:

Susan (#B5) says, “Targeted to the city of chicago's residents or any user. If you lived in a place like Alaska you could still pull this up. Could be for residents or people investing in property.” Zachary (#B7) says, “[It’s targeted] To individuals who are not familiar with certain parts of the city.”

5


After reviewing the homepage, what is the first thing you do or click? When exploring the website the Group A, 7 out of 13 testers in Group A (54%) clicked “Launch”, which opened the map interface; 2 of the 13 testers (15%) had already launched the map, and proceeded to use the search tools.

7 - A: Clicked “Launch” 1 - A: Clicked “Explore” 1 - A: Clicked on video 1 - Other: tried to click on a building 1 - Other: Clicked on "Manage Queries" on map page 1 - Other: Clicked on “Advanced Search” on map page 0 - A: Clicked “Contribute”

Most testers in group B did not take an overwhelmingly clear first step or action. 3 out of 10 testers (30%) used quick search and 3 more testers (30%) used chose commonly used queries, while the rest of the testers did a variety of other first actions.

3 - B: Used quick search 2 - B: Chose Commonly Used Query 2 - Other: Relocated map to a familiar area 1 - Other: Searched random addresses 1 - Other: Clicked advanced and commonly used queries 1 - Other: Clicked on the measure tool 1 - Other: Clicked on the menu 0 - B: Added datasets and searched 0 - B: Clicked on “Manage Queries”

If you searched, what was the search term you used? Testers’ search terms included “2 bedroom apartment”, addresses, “Business License 60624”, “Foodie”, buildings, and “restaurant passing initial inspection” and others from the commonly used queries.

6


Tell us more about the first thing you did, and why you did it. What were you expecting to happen? Several testers from Group A said that they didn’t expect the “Launch” button to open up a map. When looking at the map interface, a few testers some expressed confusion about why the map was centered where it was and about what they should do next in the absence of prompts. The tester who clicked on the video did so looking for explanation of the website. Testers in Group B similarly expressed a desire for prompts that would teach them how to use the interface. Here are some responses from testers about the first thing they did:

Renee54 (#A4) says, “‘Launch’ sounds technical, new-age. Upon launching, I did not expect to see a map of Chicago. I didn't ask for the map information, yet a map of the west side is here. I would expect to see links of items of interest to residents of Chicago; I don't see them here. Not sure what I'm looking at; the screen doesn't provide what I asked, so I'm not sure what I'd inquire.” Chris (#A16) knew he would see a map when he clicked “Launch” but thought he would be prompted to do something. He felt that he was just “dropped" into the map view. Make Believe (#B1) used quick search and typed ‘food’, but didn't see that the search returned an error because the keyboard was blocking information. After some difficulty clicking the “Help” button, they found that the instructions were targeted towards technical audience, and ‘a little complicated.’ They couldn’t see the entire window on help screen, and observed that the quick search function seems to be for a specific location, not data sets. Paloma (#B8) says, “I wanted to see what the computer [could] do instead of me creating search criteria on my own. [I] wanted to see what it would do for me. [I] didn't click on anything else because I wanted to see what that button would do. The result met expectations; I wanted locations and the grid returned results.”

Find 311 service requests that are currently open in your neighborhood. Tell us about your experience. 3 testers, Susan (#A5), Sigma1922 (#A13), and Gabi (#B12), were able to conduct searches that yielded results without prompting from their proctors.

7


13 testers attempted to search “311”, their neighborhood or ZIP Code, or a combination of both using the Quick Search bar, but these searches did not yield results. 2 other testers, Data User (#A23) and Paloma (#B8), attempted to use the “Near Me” location filter to search for open 311 requests, but did not have initial success. 10 testers (only 3 of whom did not initially attempt a search - Tact (#B17), Community Interest (#B20), and Family Safety Interest (#B21)) were prompted by their proctors to use the Commonly Used Queries and/or Advanced Search options, which yielded results. Here are some key observations of testers:

Cleveland54 (#B3) says, “It should [say] enter zip code here or enter here… [There’s] no arrow no graphical element to tell me to enter data. It doesn’t stand out. Expected a drop down. The up and down arrows expect to a droplist with a list of zipcodes. The submit button is not noticeable.” Renee54 (#A4) would have tried “advanced search” out of desperation, but would want a quick search to get everything needed. The tester felt that “advanced' implies a specific person or place, not general information about a neighborhood. Susan (#B5) says she thought there should be a clearer place to enter addresses. She wouldn’t put it where there is “Quick Search.”

Did this search meet your expectations?

Group A Yes 23% (3) No 77% (10)

Group B Yes 60% (6) No 40% (4)

Why or why not? Here are some responses from testers who said “No” this search did not meet their expectations:

Make Believe (#B1) says, “[It] is not intuitive. [It’s] for a technical audience [and it] needs different UX to engage a non-technical audience”

8


Ms Boop (#A6) says, “It is hard to interpret the data. There are no keys.” JG (#A14) says, “The options that are given to work with aren't clear. [It] needs to be more specific yet easy to understand - [I] played with a bunch of options, yet nothing showed up.”

Here are some responses from testers who said “Yes” this search did meet their expectations:

Paloma (#B8) says, “It produced something but [I’m] trying to understand the info in the data points. Trying to guess what it is.” Gabi (#B12) says, “Often when you try to map stuff it doesn't work the first time. I am used to instructions but I know that isn't common with this stuff. My expectation wasn't high. I will try this again on a computer.” Sigma1922 (#A13) says, “Because when [I] was able to find the 311 request I was able to find the information without doing additional search. It was right in front of me.”

How easy was it to find open 311 service requests in your neighborhood?

Group A 5 - Very easy 0% 4 - Easy 15% (2) 3 - Neutral 31% (4) 2 - Difficult 31% (4) 1 - Very difficult 23% (3)

Group B 5 - Very easy 10% (1) 4 - Easy 0% 3 - Neutral 10% (1) 2 - Difficult 40% (4) 1 - Very difficult 40% (4)

How would you improve this search experience?

Search Functionality 13 testers recommended that changes should be made to the OpenGrid search tools to make it easier for users to find the information they are looking for. 7 of these testers thought that the search bar should respond to addresses/ZIP codes or tags like “311”. 3 testers commented that

9


the existing search filters are too complicated, and that they should be simplified. Paloma (#B8) says, “Commonly used queries was easy; all other filters made it difficult. Dataset and Standard filters should be first; it would be easier.”

User Orientation 7 testers mentioned that they would improve the descriptions and instructions that orient the user to the OpenGrid interface. 2 of these testers recommended adding instructions that suggest how a user would interact with the page, such as suggesting a search or having sample questions. For example: Cleveland54 (#B3) says, “The advanced search panel should be pushed down with a note at top saying ‘welcome, you can search blah blah here.’ The landing page should have description of the website and then launch to the site.” 2 other testers recommended improving the language used to label data points and information on the map.

Design Elements 7 testers said that they would make changes to design elements including color, readability, and map appearance to improve the search experience. 2 testers commented that the text and symbols on the interface were small and might be difficult for some people to read. 2 other testers (Zachary (#B7) on a laptop and Gabi (#B12) on an iPhone 5, mentioned that it was difficult to see all of the search text on their screens. 2 other testers mentioned data point color as an area that could be improved. JG (#A14) indicated that “Information should be more easy to find. For starters, the blue and white [on the map] is not the best color option, too. As an additional walkthrough, tester chose street closures within zip code value, and found 13 records. The records that are showing up, however, are not reflected on the map -- tester requests highlighting streets or including more easily visible what the closures are.”

Pretend that you are in the process of moving to a new part of Chicago; use OpenGrid to find information about this area and let us know what you find.

What did you learn using OpenGrid? When seeking information about a new part of Chicago, 7 of the 23 testers initially entered in neighborhoods or ZIP codes into the search fields without indicating the datasets, and their searches yielded no results. JG (#A14) says, “Honestly - the site is not giving what I want, need; information is confusing and not clearly presented. Not much has been learned.”

10


10 testers (6 from Group A and 4 from Group B) were able to successfully obtain information by using the search function, but had varying degrees of difficulty. Here are a couple of experiences we observed:

Paloma (#B8) searched 60638 with several different filters, including crimes. She says, “This is telling me a lot about the neighborhood. [I] would be worried about seeing all these crimes and thefts… for everyday citizens, try translating the style of writing more user-friendly. It looks like the data was copied and pasted out of the inspector’s user book and it will take time to understand the data.”

Tact (#B17) first went to add a ZIP code, then found data sets and started to scroll to see options. The tester ended up looking at landmarks before settling on crime. Tact was a little slow to press on result dots; they were looking for historical data but were only getting data from the most recent year (although they did not enter a data range). Tact recommended that the labels for filters and search criteria could be improved, for example: ‘geospatial’ is not commonly used phrase. Yoonie (#A18) could not search by community area "Pilsen" as expected, so the tester decided to move the map to that general location and look up crime data from 2001-present. The results do not prepopulate as the tester expected, but they eventually change the geospatial filters to “map extent.”

Some testers moved beyond the search bar. Charlie (#A9) tried multiple addresses through quick search, but none provided a result; she found that moving the map around manually was more effective than using the search bar. Tester Chris (#A16) did the same thing to find their results.

Search for crime data from the last year in the neighborhood you live in.

Group A 8 of 13 A testers (62%) encountered difficulties in searching for crime data from their neighborhoods. 2 of these testers initially attempted to search using the names of their neighborhoods, which did not return results; using ZIP codes or ward numbers yielded data points. 2 others were not able to return results. Here are some of the key observations:

Doc (#A2) tested OpenGrid on a smartphone and had a hard time finding crime and filtering for a time period. He also lost his search criteria when he navigated back to the map from the search panel. When he submitted his search query, the app returned with an error, “Search is invalid,” without giving any explanation. ChicaoJAE (#A10) started her search with location, then selected crime data. With a little instruction, she found search filters, but had to manually enter the dataset. The

11


tester hit submit, and the app produced results on the grid. As she scrolled through, the tester saw the icons but was more interested to read grid data.

Group B 9 out of 10 B testers (90%) had a difficult experience conducting this search. 4 of these testers found it hard to select the correct data set from the listed options. 3 were not able to return results. Here are some of the key observations:

Tact (#A17) used advanced search to enter a ZIP code into the boundary filter and select crime data from last 15 years, with some help from the test proctor. On the first attempt, the search produced error; then, the tester removed some empty filters and it produced results. Family Safety Interest (#B21) did not recognize the menus that they needed to pull up the crimes. The tester would have liked to select a neighborhood from a drop-down menu instead of a ZIP code. They found rectangle selection tool a little difficult to use, but the polygon tool was friendlier.

Did this search meet your expectations?

Group A Yes 46% (6) No 54% (7)

Group B Yes 70% (7) No 30% (3)

Why or why not? Here are some responses from testers who said “No” this search did not meet their expectations:

Susan (#B5) says, “The language is not at a 6th grade level and is not at a level that anyone can use.” Ms Boop (#A6) says, “It was not easy to determine how to get the data and once I did it was much more information that I needed. I just needed crime and address. I don't need ward or district.”

12


Charlie (#A9) observed that it's not as helpful as it should be to find anything -- you should only have to click once or twice to get the desired results. Charlie thinks that OpenGrid is not designed for novices, and would require training to know how to use it.

Here are some responses from testers who said “Yes” this search did not meet their expectations:

Paloma (#B8) says, “It got easier because I got use to practicing.” Heather (#A11) says, “The data is good, but having the address more easier to find would be nice.” Chris (#A16) says, “Adding the rule took a second to figure out, and then it made sense.”

How easy was it to find crime data from last year?


Group B 5 - Very easy 0% 4 - Easy 40% (4) 3 - Neutral 20% (2) 2 - Difficult 10% (1) 1 - Very difficult 30% (3)

How would you improve this experience?

Search Functionality 9 testers recommended making changes to the OpenGrid search experience. 2 of these testers (Doc (#A2) and Charlie (#A9)) described the search experience as not too difficult, but only after learning how to navigate it, indicating a need for more instruction around searching. The other 7 testers had comments on how to improve the search selector fields. 2 testers mentioned improvements to the way the search function handles location. Chris (#A9) feels that the OpenGrid interface emphasizes the data set and then the location; he would swap that

13


around and make it more location-driven, such that users would first select location, then select their desired data set. Cleveland54 (#B3) would prefer a drop-down menu of community areas to having to manually input locations. 2 testers (Susan (#B5) and Gabi (#B12)) thought that it was difficult to search crime. Gabi (#B12) recommends better defining the suggested queries and giving more options to refine the results. Similarly, 1 tester, Tact (#B17), hope for the ability to search for specific events or incidents within the crime data set. 2 testers (Paloma (#B8) and Downtown Closures (#B21)) both recommended reducing the initial number of search fields to make it easier to search.

User Comfortability 5 testers said that OpenGrid should be made more user-friendly. 2 of these testers (Renee54 (#A4) and Make Believe (#B1)) mentioned that the language currently used is too technical Renee54 (#A4) says, “It needs to be user-friendly for all Chicago citizens - terms like ‘geo-spatial filters' don't mean anything to most users. It sounds deep, but gets people nowhere. This isn't Star Trek over here.” 1 tester (JG (#A14)) said that clearly labeling tabs would be helpful to orienting the user.

Design Elements 1 tester Community Interest (#B20) encountered challenges related to the design; when searching for crime in their neighborhood on an Android smartphone, the gridview was up and blocking the advanced search pane. The tester had to minimize the grid to pull-up the advanced search pane.

Relevance of Data 5 testers recommended improvements related to the data returned by searches. 2 testers (Zachary (#B7) and Family Safety Interest (#B21)) wanted more descriptive information on the map data points, while 1 tester (Ms Boop (#A6)) would prefer a shortened description of the crime in the data point. Tester ChicagoJAE (#A10) recommends displaying both the grid and map results at the same time. Tester Bronco (#A15) had concerns about the quality of data returned, noting that they were not able to obtain information from within the last year.

14


Next, find a dataset that you are interested in learning more about that is relevant to you. What did you look for?

3 - Food inspections 2 - Filming locations 2 - Landmarks 2 - Potholes 2 - Building violations 1 - Schools 1 - Red light cameras 1 - Speed cameras 1 - Street closures 1 - 311 service requests 1 - Environmental complaints 1 - 911 reports 1 - Transportation 1 - Common Queries

Tell us about your experience. How did you search? Can you describe the information you found? 9 testers were successful when searching for datasets they were interested in learning more about.

Charlie (#A9) tried to search for “speed cameras” and “locations,” but that didn’t work; the tester then tried “citywide,” which yielded 50-60 results. After using the website for a while, it became slightly easier, and the tester saw the process as more useful over time. Charlies says, “You need to know specifically what you want in order to get any value out of [OpenGrid].”

Community Interest (#B20) was able to look at potholes to see where they were reported. The tester mentioned it would be helpful to report 311 calls through the app.

5 testers were not able to successfully return results through their searches. Here are some of the things we observed:

Renee54 (#A4) was not able to find information they were looking for, and would instead just go to another website.They recommended that OpenGrid developers look at better models out there for public information and improve it; the tester would not use this site because it was unclear what kind of data is available in the first place. Without knowing what information is there, the site is useless for now -- but it can be improved.

15


JG (#A14) tried to search “schools” using “quick search” at first, which yielded no results. The tester then went to “advanced search,” but didn’t see their query under “commonly used queries”; the tester then clicked on “datasets -> add dataset" and did not see it there either. The tester concluded that schools are not included in the OpenGrid system.

The remaining 9 testers’ search outcome is unclear based on the test notes, but 5 of these testers described encountering obstacles in their search processes. 13 testers (4 testers who had successful searches) described some difficulties during the search. Here are some of their experiences:

Gabi (#B12) searched for “film locations,” thought that the filters were nice. Her search yielded results from the whole city since she did not specify boundaries. She zoomed in and found that the data points were from three years ago. “They only paid $25?! [It’s] exciting there are so many of them.” She could not figure out how to limit the search by date, and accidentally hit an arrow that opened the bottom table list. It was hard to close the popup when it takes up most of the screen on a mobile device. Family Safety Interest (#B21) found that once she got the hang of the query, it was easier, but took a bit of effort to find the query in the menus. The custom location queries took a number of taps on her smartphone to find.

Overall, how easy do you think it is to use the OpenGrid website?


Group B 5 - Very easy 0% 4 - Easy 20% (2) 3 - Neutral 40% (4) 2 - Difficult 10% (1) 1 - Very difficult 30% (3)

16


What improvements might you make to make it easier to use?

Search Functionality 4 testers recommended that changes should be made to the OpenGrid search tools to make it easier for users to find the information they are looking for. Make believe (#B1) says, “Search by natural feedback. More functionality in primary search.” Doc (#A2) says, “Saving information so you don't have to reenter everything. Less filters. They are too broad.” 4 other testers mention improving the search-by-location mechanism, which tripped many of them up during the test exercises. Charlie (#A9) says, “Get it to recognize addresses in quick search query quicker.” Zachary (#B7) says that “Quick search should allow address.”

Linguistic Accessibility 4 testers mentioned that language used on the OpenGrid interface was unclear and/or hard to understand. Chris (#A16) feels that there is “too much technical language that could be more user-friendly.”

User Orientation 4 testers recommend adding more explanation about elements of the app. Gabi (#B12) wants “More explanation of the buttons and icons,” while Renee54 (#A4) suggested including “More links, lists of what data is available, and more user-friendly design.” 2 testers mention the desire for instructions on how to use the app. Paloma (#B8) says that a “Pop-up about the site would be useful on the landing page or quick tutorial about how to use the site.” 2 testers suggest making changes to the look and feel of the app. ChicagoJAE (#A10) says, “Larger font. Make the grey bar larger.” Cleveland54 (#B3) says that “adding graphical elements” would make it better.

Overall, how easy do you think it is to understand the content and information on OpenGrid?

Group A 5 - Very easy 15% (2) 4 - Easy 38% (5) 3 - Neutral 8% (1) 2 - Difficult 15% (2)

17


1 - Very difficult 23% (3)

Group B 5 - Very easy 10% (1) 4 - Easy 40% (4) 3 - Neutral 0% 2 - Difficult 30% (3) 1 - Very difficult 20% (2)

What improvements might you make for the information to be easier to understand? Language Accessibility 3 testers mentioned difficulty understanding the language used on the OpenGrid interface. Paloma (#B8) didn’t understand the vocabulary used, and found that it had alot of technical wording, while Bronco (#A15) found that searching was intuitive "once I got the language." Doc (#A2) recommended use of “more plain/laymen terms.” User Orientation 5 of 23 testers expressed a desire for more information about how to use OpenGrid when the interface is first launched. 1 tester, Sigma1922 (#A13) said understanding would be improved by including more detailed information and a tutorial on the main page on how to navigate. The other 4 testers recommended giving the user more instruction about what kinds of information are accessible through OpenGrid; Charlie (#A9) says, “Once you get inside and get a better feel for it, the information is easy to understand - key is [the] user needs to understand the information that they're looking for.” Information Displayed 5 testers recommended various improvements to the format and content of the returned search results. Here is what they said:

● “Each result on the map would have a link to click on for more information.” - Zachary (#B7)

● “Have the address more predominant.” - Heather (#A11) ● “When I was scrolling through the text information about 311 calls, there are a lot of

fields there which is okay. Maybe there could be more space between fields so they don’t all run together. I don't think I want the fields to go away but it is not easy [to understand].” - Gabi (#B12)

● “[I] was more interested in the grid instead of the map.” - Community Interest (#B20) ● “[I] would like to see some more data sets. The grid search was also difficult to find.” -

Family Safety Interest (#B21)

18


What other types of information would you like to see in OpenGrid that would be helpful to you as a user?

6 - Businesses 3 - Crime and safety 3 - Events 2 - Neighborhoods 2 - Parking and traffic 2 - Transportation 1 - 311 reporting function 1 - hospitals 1 - houses 1 - parks 1 - schools

After reviewing parts of the website, do you think the OpenGrid website is targeted to you?

Group A Yes 69% (9) No 31% (4)

Group B Yes 40% (4) No 60% (6)

Why or why not? If not, who is the target audience? Here are some responses from testers who said “Yes” they are the target audience:

Bronco (#A15) says, “More than I thought! If I just looked on the opening page and saw references to open source and APIs and SQL like at the beginning, it would be targeted to the developer group… but after exploring, maybe it is for end users!" Community Interest (#B20) says, “Would be useful, but still too difficult to use.”

Here are some responses from testers who said “No” they are not the target audience:

19


Renee54 (#A4) says, “It's currently not targeted to anybody - not the citizens of Chicago; it’s not even targeted to the writer of this program. It’s not easy to use or useful in its current state.” Gabi (#B12) says, “Not specifically, but with some practice I could learn how to use it and it has some information that I am interested in.”

Do you like the OpenGrid website?

Group A Yes 38% (5) No 31% (4) Not sure 31% (4)

Group B Yes 60% (6) No 30% (3) Not sure 10% (1)

Why or why not? Here are some responses from testers who said “Yes”:

Heather (#A11) initially didn’t like OpenGrid, but when she was able to find the data that she wanted she was pleased with the information. Gabi (#B12) says, “There is potential to access all that data and mash it up with the spatial representation. That is very exciting and worthwhile.”

Here are some responses from testers who said “No” they do not like OpenGrid:

Renee54 (#A4) says, “Thumbs down - the page itself was beautiful, but beyond that, that's all it was - it was not clear how I would use it or what I could get from it. It's very disappointing.” Susan (#B8) says that OpenGrid is “Too frustrating to use.”

Here are some responses from testers who said they are “Not sure” whether they like OpenGrid:

Ms Boop (#A6) says, “I am not sure how easy it will be to start using it as a brand new person.”

20


Bronco (#A15) says, “I like the idea of it... I understand it is a work in progress. [But] It needs a lot of work to plot the results.”

How likely are you to use the OpenGrid website in your personal life?

Group A 5 - Very likely 31% (4) 4 - Likely 8% (1) 3 - Neutral 31% (4) 2 - Not very likely 8% (1) 1 - Not at all likely 23% (3)

Group B 5 - Very likely 1% (1) 4 - Likely 40% (4) 3 - Neutral 10% (1) 2 - Not very likely 20% (2) 1 - Not at all likely 20% (2)

What makes you say this? Testers who said they would be “not very likely” or “not at all likely” to use the OpenGrid website in their personal lives mention finding OpenGrid too difficult to use, or not having reason to use it. Bronco (#A15) says, “In it's a current state ... it might be easier on the City's data portal. I can get the data and know I'll get consistency of the results. I know what I am getting on the data portal.” Another tester, Downtown Closures (#B22) said that they don’t access computers very often. Testers who responded “neutral” mention some difficulties in using the site, but also uncertainty about how they would use it. Heather (#A11) says she would find OpenGrid useful in figuring out what neighborhood to live, work, and be around during certain times in her life. JG (#A14) says, “Based on the data choices available, things like neighborhood crime, 311 calls could be useful for me from time to time.” Testers who said they would be “likely” or “very likely” mention the appeal of relevant information. Ms Boop (#A6) says, “Several of the common queries would be helpful as a contributing neighbor to my neighborhood.” Gabi (#B12) says, “I am a curious person so looking up crime or zoning code violations or whatever is in there. It might be amusing/interesting. It might become vital if we decide to buy a house. Curious about future information that may appear.”

21


How likely are you to use the OpenGrid website in your professional life?

Group A 5 - Very likely 8% (1) 4 - Likely 15% (2) 3 - Neutral 15% (2) 2 - Not very likely 23% (3) 1 - Not at all likely 46% (6)

Group B 5 - Very likely 0% 4 - Likely 30% (3) 3 - Neutral 30% (3) 2 - Not very likely 10% (1) 1 - Not at all likely 30% (3)

What makes you say this? Testers who said they would be “not very likely” or “not at all likely” to use the OpenGrid website in their professional lives gave either difficulty of use or lack of relevance as their reasons. Charlie (#A9) says, “I'm not working in transportation, sanitation, other areas that apply to city governance - might take a glance for traffic information related to work, but it doesn't do that anyway.” Bronco (#A15) says that as a sociologist, they really see the importance in consistent results. Testers who responded “neutral” describe OpenGrid as out of line with their work. Doc (#A2) says he might use it “only if it hand anything pertaining to restaurants or cooking jobs.” Testers who said they would be “likely” or “very likely” mention the relevance of the tool. Gabi (#B12) says that she is an “academic librarian so there are always students looking for data & information (public health, social justice, geography, GIS program).”

How likely are you to share the OpenGrid website or its information with your neighbors or community members?

Group A 5 - Very likely 15% (2) 4 - Likely 23% (3)

22


3 - Neutral 15% (2) 2 - Not very likely 15% (2) 1 - Not at all likely 31% (4)

Group B 5 - Very likely 20% (2) 4 - Likely 30% (3) 3 - Neutral 0% 2 - Not very likely 20% (2) 1 - Not at all likely 30% (3)

What makes you say this? Testers who said they would be “not very likely” or “not at all likely” to use share the OpenGrid website gave either difficulty of explanation and use or lack of relevance as their reasons. Zachary (#B7) says his neighbors may be frustrated by OpenGrid and if he were to recommend it, it would make him look bad. Make believe (#B1) was afraid they’d have to do too much explaining. Testers who responded “neutral” were split between the value of the information and the challenges of using the tool. Yoonie (#A18) says, “Data seems really interesting - that would be a 4. Accessibility is challenging is 2. There's potential to have this tool beat out all of the other tools (crime map, food inspections stuff, public safety concerns).” Testers who said they would be “likely” or “very likely” mention the relevance of the information to their communities. Gabi (#B12) says, “Most neighbors are elderly or don't speak English. Would share with son's friends who could share with parents.” Chris (#A16) says that his community has a bunch of other data geeks that are interested in checking this out.

In what other ways would you improve the OpenGrid website? Recommendations for further improvements were largely in the same categories as previous suggestions. Here are some of the improvements that other testers suggested:

Search Functionality 9 testers suggested making improvements to the OpenGrid search function, finding it difficult to use quick search, especially when searching by location. Neighborhood Watch (#A20) says, “[It] needs to improve navigation. I would also like to see neighborhood as a drop-down option… [and] to be able to type in the ZIP code to see everything that's happening.” Tact (#B17) says, “Improve the menu items by including dataset info.”

23


Design Elements 8 testers recommended making changes to the design elements of the OpenGrid app interface, pinpointing several current features as obstacles to easy use. Cleveland54 (#B3) says, “Graphic design information [is] needed to explain. There should be a floating help; the nav tools are completely useless until I know what I’m actually doing.” Renee54 (#A4) says, “The user should be able to figure out easily what they want to do. In its current state, it needs an overhaul. It seems like there are great ideas embedded here, but the design did not let me get to them.” 2 testers commented that the legibility of the app was poor. Yoonie (#A18) says, “Legibility is difficult to read, not enough contrast for this search area… Make it a little bit easier to read. Launch was terrific simple action. White on blue text.”

Linguistic Accessibility 2 users recommended making improvements to the lexicon used in OpenGrid, commenting that the language is too technical for non-developer audiences. Susan (#B5) says, “The language was too difficult to understand.”

Was this your first CUTGroup test? Yes 39% (9) No 61% (14)

Do you like the CUTGroup? Yes 100% (23) No 0%

Did you like this CUTGroup test? Yes 91% (21) No 9% (2)

Anything else to add?

Susan (#B5) says, “No comment. Awful.”

Gabi (#B12) says, “This was the hardest test.” Bronco (#A15) says, “This was fun! I like breaking databases!”

24


Yoonie (#A18) says, “This tool feels a lot less refined or finalized than other tools tested. Seems like an earlier version.”

25

Make $20 for a test of a Chicago data website.

Hi <<First Name>>,

We've got a new opportunity for you to make money in the Civic User Testing Group(CUTGroup) by testing a website where you can navigate Chicago data on yourcomputer or mobile device.

Are you available for a 30 - 45 minute in-person test on Wednesday, April20 between 4:00 - 7:30 PM? If so, please complete this form.

We are in the process of gathering responses, so we will be in touch in a couple ofdays to check availability, confirm the location, and confirm that you are right for thistest. We are looking for about 15-20 testers.

For your participation, you will receive a $20 VISA gift card. You'll also help makebetter software for Cook County. Thanks for being a member of the CUTGroup! Asusual, call or write with questions.

Remember: if you want to be a part of this test, please complete this form.

--

Sonja Marziano(312) 565-3274 [email protected]

Facebook Twitter Website

Subscribe Share Past Issues Translate

http://www.cutgroup.org/

https://smartchicago2012.wufoo.com/forms/cutgroup-test-chicago-data-website/def/Field113=%3C%3CEmail%20Address%3E%3E

https://smartchicago2012.wufoo.com/forms/cutgroup-test-chicago-data-website/def/Field113=%3C%3CEmail%20Address%3E%3E

mailto:[email protected]

https://www.facebook.com/smartchicago


https://www.twitter.com/smartchicago/


http://www.smartchicagocollaborative.org/


http://eepurl.com/y5gNX

http://us5.campaign-archive1.com/home/?u=085247dea37361c266002462c&id=74faafe842

http://us5.campaign-archive1.com/feed?u=085247dea37361c266002462c&id=74faafe842

CUTGroup Test: Chicago Data WebsiteIf you would like to participate in the next CUTGroup test, please complete this form. We will follow‑up to check youravailability and let you know if you qualify for this test!

Smart Chicago values your online privacy and is committed to safeguarding the data you share with us and youronline privacy. Our On‑Line Privacy Policy applies to data collected at www.smartchicago2012.wufoo.com domain.

Email *

Yes

No

Are you available for an in‑person test on Wednesday, April 20 for 30‑45 minutes between 4:00 and 7:30 PM? *

Yes

No

Are you 18 years old or older? *

5 ‑ Very familiar

4 ‑ Somewhat familiar

3 ‑ Neutral

2 ‑ Not very familiar

1 ‑ Not at all familiar

How familiar are you with what is happening in your neighborhood? *

Tell us more.

If anything, what are you interested in learning about in yourneighborhood that you do not already know? *

Yes

No

Have you looked up information online about the neighborhood you livein now? *

Yes

No

Do you use maps to explore your neighborhood? *

What characteristics would you look for when moving to a newneighborhood? *

How would you search or find that information? *

Yes

No

Have you ever used the City of Chicago’s data portal before? *

Yes

No

Do you work with or analyze datasets in your personal or professionallife? *

Let us know when you could join us on Wednesday for this test.

Hi <<First Name>>,

Thanks for your interest in our CUTGroup test of a website where you can navigateChicago data on your computer or mobile device. We'd like to know if you can stillmake it next Wednesday and, if so, what time you want. Here are the details:

Wednesday, April 20, 2016Between 4:00 - 7:30 PMChicago Public Library Legler Branch115 S. Pulaski Road, Chicago IL 60624

If you can still make it on this night, please let us know what time slot you wantby completing this form. For your participation, you will receive a $20 VISA gift card.

We will get back to you with a confirmation of your time.

If this location doesn't work for you, or if you just can't make it on this night, after all,no big deal. If you have any questions or comments, just hit "reply" and let me knowwhat you think.

Remember: use this form to tell us about your availability.

Thanks for being a member of the CUTGroup!

--

Sonja Marziano(312) [email protected]


http://www.chipublib.org/locations/47/

https://www.google.com/maps/place/115+S+Pulaski+Rd,+Chicago,+IL+60624/@41.8793121,-87.7296802,16z/data=!4m2!3m1!1s0x880e32c2fac680ab:0x42cbbca0262159e9

https://smartchicago2012.wufoo.com/forms/cutgroup-test-chicago-data-website-availability/def/Field113=%3C%3CEmail%20Address%3E%3E

https://smartchicago2012.wufoo.com/forms/cutgroup-test-chicago-data-website-availability/def/Field113=%3C%3CEmail%20Address%3E%3E





CUTGroup Test: Chicago Data Website AvailabilityIf you would like to participate in our CUTGroup test, please complete this form. The test is Wednesday, April 20between 4:00 ‑ 7:30 PM at the Chicago Public Library Legler Branch, 115 S Pulaski Rd, Chicago, IL 60624.

Email *

Yes

No

Are you still available on Wednesday, April 20 to attend this test inperson? *

4:00 ‑ 4:30 PM ‑‑ This slot is now full! Please choose another.



6:15 ‑ 6:45 PM

7:00 ‑ 7:30 PM

Sorry, this day does not work for me!

Test time slot 7 remaining.

Which of these time slots would work for you? Please select all that mayapply, and we will get back to you with your confirmed time. *

Yes

No

Would you be willing to bring your mobile device to this test to testwith? *

Thanks for being willing to help out.

Hi <<First Name>>,

Thanks for your interest in our CUTGroup test of a website that helps you navigateChicago data on your computer or mobile device. We really appreciate yourresponse to our call-out. We got a lot of responses for this test, and unfortunately,you were not selected. We took a look at all of the responses we received especiallyaround the data analysis tools you use, how you've used the Chicago data portal,how you get neighborhood information and more.

To keep you in the loop, the website we are testing is OpenGrid. If you are interestedin contributing this project or have feedback you can email the developer teamdirectly or contribute to their GitHub.

Even though you were not selected for this test, we have a lot more tests coming up.We will send you info on our next test, and we'll definitely try to spread the testsaround so that everyone gets what they want out of the CUTGroup experience.

Thanks for being a member of the CUTGroup. As usual, call or write with questions.

--

Sonja Marziano(312) [email protected]

Facebook Twitter Website


http://opengrid.io/


https://github.com/Chicago/opengrid/issues











CUTGroup 24: OpenGridThanks for participating in our test of OpenGrid, a website that lets you navigate Chicago data.

About youFirst, some preliminaries. Keep in mind that we are going to publish the results of this test, but will not associate your name oremail address with the results.

Background review

First Last

Name *

Email or Phone Number *

"A" tester

"B" tester

Is this an "A" or "B" tester? *

You can check this on the tester sheet!

Give yourself a tester profile name. *

Proctored by:

What device are you using for this test? *

You can check this on the tester sheet whether they should be using a laptop or theirmobile device.

Safari

Internet Explorer

Chrome

Firefox

Other

What browser are you using right now? *

Can you tell us more about your experience, if any, using data ordatasets? *

Homepage reviewA testers should start on http://opengrid.io/

B testers should begin on http://chicago.opengrid.io/opengrid/

Testers have answered screening questions which should be a starting point to thisconversation.

Take a look at the tester sheet for more information!

Review the homepage and tell us about the three things that you noticefirst.

While you review, try not to click on the next page just yet, but feel freeto scroll up and down the page. *

If you had to describe what this website does, what is your best guess? *

Who do you think this website is targeted to? Why? *

A: Clicked "Launch"

A: Clicked "Explore"

A: Clicked "Contribute"

A: Clicked on video

B: Used quick search

B: Chose Commonly Used Query

B: Added datasets and searched

B: Clicked on "Manage Queries"

Other

After reviewing the homepage, what is the first thing you do or click? *

Website review

If you searched, what was the search term you used?

Tell us more about the first thing you did, and why you did it.

What were you expecting to happen? *

Find 311 service requests that are currently open in your neighborhood.Tell us about your experience *

Yes

No

Did this search meet your expectations? *

Why or why not?

5 ‑ Very easy

4 ‑ Easy

3 ‑ Neutral

2 ‑ Difficult

1 ‑ Very difficult

How easy was it to find open 311 service requests in yourneighborhood? *

How would you improve this search experience? *

Pretend that you are in the process of moving to a new part of Chicago,use OpenGrid to find information about this area and let us know whatyou find.

What did you learn using OpenGrid? *

Search for crime data from the last year in the neighborhood you live in.*

Yes

No

Did this search meet your expectations? *

Why or why not?

5 ‑ Very easy

4 ‑ Easy

3 ‑ Neutral

2 ‑ Difficult

1 ‑ Very difficult

How easy was it to find crime data from last year? *

How would you improve this experience? *

Overall Review

Next find a dataset that you are interested in learning more about that isrelevant to you. What did you look for? *

Tell us about your experience. How did you search? Can you describethe information you found? *

5 ‑ Very Easy

4 ‑ Easy

3 ‑ Neutral

2 ‑ Difficult

1 ‑ Very Difficult

Overall, how easy do you think it is to use the OpenGrid website? *

What improvements might you make to make it easier to use?

5 ‑ Very Easy

4 ‑ Easy

3 ‑ Neutral

2 ‑ Difficult

1 ‑ Very Difficult

Overall, how easy do you think it is to understand the content andinformation on OpenGrid? *

What improvements might you make for the information to be easier tounderstand?

What other types of information would you like to see in OpenGrid thatwould be helpful to you as a user?

Yes

No

After reviewing parts of the website, do you think the OpenGrid websiteis targeted to you? *

Why or why not? If not, who is the target audience?

Yes

No

Not sure

Do you like the OpenGrid website? *

Why or why not?

5 ‑ Very likely

4 ‑ Likely

3 ‑ Neutral

2 ‑ Not very likely

1 ‑ Not at all likely

How likely are you to use the OpenGrid website in your personal life? *

What makes you say this?

5 ‑ Very likely

4 ‑ Likely

3 ‑ Neutral



How likely are you to use the OpenGrid website in your professionallife? *


5 ‑ Very likely

4 ‑ Likely

3 ‑ Neutral


How likely are you to share the OpenGrid website or its information withyour neighbors or community members? *

Overall review of the CUTGroup experience



In what other ways would you improve the OpenGrid website? *

Yes

No

Was this your first CUTGroup test? *

Yes

No

Do you like the CUTGroup? *

Yes

No

Did you like this CUTGroup test? *

Anything else to add?