Census Data Visualization Gallery As Data For the Digital Government Strategy

Dr. Brand NiemannDirector and Senior Data Scientist

Semantic Communityhttp://semanticommunity.info/

AOL Government Bloggerhttp://gov.aol.com/bloggers/brand-niemann/

March 28, 2013http://semanticommunity.info/Census_Data_Visualization

Background• Received Email about Data Visualization: Images That Tell a

Story Webinar:– http://www.howto.gov/training/classes/creating-data-visualizatio

ns• Email Referenced Data Visualization of the Week:

– http://www.census.gov/dataviz/• I Started to Build a Census Data Visualization Gallery as Data

(Accessible to All) For the Digital Government Strategy:– http://semanticommunity.info/Census_Data_Visualization

• I Heard the Webinar and Was Further Motivated to Respond to Their Invitation to Provide Comments:– http://www.howto.gov/training/classes/creating-data-visualizatio

The 7 Habits of Highly Effective People Habit 2: Begin With The End In Mind

• Habit 2 is based on imagination--the ability to envision in your mind what you cannot at present see with your eyes. It is based on the principle that all things are created twice. There is a mental (first) creation, and a physical (second) creation. The physical creation follows the mental, just as a building follows a blueprint. If you don't make a conscious effort to visualize who you are and what you want in life, then you empower other people and circumstances to shape you and your life by default. It's about connecting again with your own uniqueness and then defining the personal, moral, and ethical guidelines within which you can most happily express and fulfill yourself. Begin with the End in Mind means to begin each day, task, or project with a clear vision of your desired direction and destination, and then continue by flexing your proactive muscles to make things happen.

• My Note: My End In Mind Is The Census Data Visualization Gallery As Data For the Digital Government Strategy.

Source: https://www.stephencovey.com/7habits/7habits-habit2.php

Census Data Visualization Gallery

http://www.census.gov/dataviz/

My Note: Added Census Flows Mapper Since I Started That I Will Do Next Separately.I Found the GreatMigrations Says NotAuthorized to Access.

Census Data Visualization:Knowledge Base

http://semanticommunity.info/Census_Data_Visualization

My Note: Well-Defined URLTo Each Piece and Mashupof Related Pieces.

Census Data Visualization:Spreadsheet

http://semanticommunity.info/@api/deki/files/23626/CensusVisualizations.xlsx

My Note: Makes Web Linked Dataand Identifies Boundary Files Needed.

Booming Cities Decade to Decade

http://www.census.gov/dataviz/visualizations/017/

My Comment: Visualization, Story, and MetadataAre Together, But not the Data Table.See Next Slide That Does That WithInteractivity Between Them and Makes DataAccessible to All!

Booming Cities: Spotfire

Story and Metadata

Visualization

My Note: See Data Sorted and Linked to Visualization!

Data Visualization Gallery• Population Bracketology• Migration Between California & Other States• U.S. Territory and Statehood Status• Spoken Languages Other than English• Center of Population, 1790-2010• Population Change by Decade• Without A High School Education• A Decade of State Population Change• State-to-State Migration• Population Under Age 5 by Congressional

District• Components of Metro Area Change• Blooming States• Largest Urbanized Areas• Coastline County Population• I-90 Population Density Profile, 2010• Keeping Pace with New York

• Population Shift to the West and South• I-10 Population Density Profile, 2010• Booming Cities, 1830-2010 (Previous

Example)• I-5 Population Density Profile, 2010• Islands of High Income• The Great Migration, 1910 to 1970• Following the Frontier Line, 1790 to 1890• Changing Ranks of States• State Populations: 1890, 1950, and 2010• Before and After 1940• Physical to Political Geography• Differential City Growth Patterns• I-95 Population Density Profile• Increasing Urbanization• Gaining and Losing Shares• Top 20 Cities

Population Bracketology

My Comment: Why Not Show Data Sorted and Linked To Visualization?See Next Slide.

Population Ranking: Spotfire

Story and Metadata

Visualization

My Story

Net Migration Between California and Other States

My Comment: Why Not Just Show the Data and A Summary?See Next Slide.

Net Migration Between California and Other States: Spotfire

Data Summary

Data Table

My Story

Story and Metadata

My Note: Data Sorted in Rank Order

U.S. Territory and Statehood Status by Decade

My Comment: Why Not Show Data Sorted and Linked To Visualization?See Next Slide.

Statehood Status: Spotfire

My Story

Story and MetadataData

Visualization

Spoken Languages Other than English

My Comment: Why Not A Simple Bar Chart To Tell The Story?See Next Slide.

Spoken Languages: Spotfire

My Note: See Visualization Labeled andLinked to Filtered and Sorted Data!

Visualization

My Story

Story and Metadata

Center of Population, 1790-2010

My Comment: The Territorial Expansion Boundaries Were Not Provided.This is Not Providing Access to All the Data!

Center of Population: Spotfire

My Note: See Visualization Labeled and Linked to Data!

Visualization

My Story

Story and Metadata

Population Change by Decade

My Comment: Small-multiples of County-level Maps are Difficult to See So a Few Maps with Say the Top 10 Population Changes and the Ability to Sort the Data is More Useful. Also Note the Large Number of Counties with No Data (Empty). See Next Slide.

Population Change by Decade: Spotfire

My Story

Story and Metadata

Visualizations

My Note: See Visualization Labeled and Linked to Sorted Data!

My Note: This Data Set Had to BeAugmented with County Namesto Work with the Shape File.

Without A High School Education

My Comment: These Comparisons Require Lots of Eye Movement.See Next Slide.

Without A High School Education: Spotfire

My Story: This Trellis Chart Shows the Variation For Each of the Four Categories (Male and Female and Native and Foreign born) Across each Region by Race or Hispanic Origin and the Columns Can be Sorted in the Table Below to See Their Rank Order.

Visualization

My Story

Story and Metadata

A Decade of State Population Change

My Comment: These Comparisons Require Lots of Eye Movement and An Extra Click toSee the Details for Each State. See Next Slide.

A Decade of State Population Change: Spotfire

Visualization

Filter

Visualization

My Comment: See Visualization Linked to Map and To Sorted Data Table.

State-to-State Migrations

My Comment: This Is Difficult To Understand and The Data Needs Summarization.See Next Slide.

State-to-State Migrations: Spotfire

My Story

Story and Metadata

Visualization

Filter

My Note: Use Filters and Sorting To See Links Between Visualization and Data.

Population by Congressional District

My Comment: I Would Use a Congressional District Map and a Statistical Distribution Function to Show These Data. The Data Set Was Augmented For That. See Next Slide.

Population by Congressional District: Spotfire

My Story

Story and Metadata

Visualization

My Note: Use Filters and Sorting To See Links Between Visualization and Data.

Filter

Components of Metro Area Change

My Comment: I Would Use a State Map and a Statistical Distribution Function to Show These Data. The Data Set Was Augmented For That. See Next Slide.

Components of Metro Area Change: Spotfire

Visualization

Scatter Plot WithStraight Line Fit

VisualizationMy Story

Story and Metadata

My Note: See Links Between Visualization and Data.

Blooming States

My Comment: I Would Use a Trellis Chart To Show These Data.The – in the Data Set Were Eliminated For Spotfire. See Next Slide.

Blooming States: Spotfire

My Story

Story and Metadata

VisualizationMy Note: See Links Between Visualization and Data.

Largest Urbanized Areas

My Note: This Data Set Needed To Be Separated Into Two Data Sets, Latitude andLongitude Added and Visualized In a Different Way. See Next Slide.

Largest Urbanized Areas: Spotfire

My Story

Story and Metadata

Scatter PlotsWith Lines Visualization

Coastline County Population

My Comment: This Map Shows Overplotting and Not the Data, WhichHad To Be Calculated By Spotfire and Displayed. See Next Slide.

Coastline County Population: Spotfire

My Story

Story and MetadataVisualizations

VisualizationData

Summary DataFilters

My Note: Each shape maps to more than one value in the column used to color it.

My Note: Data Calculated by Summary Table

I-90 Population Density Profile

My Comment: It Is Helpful to See All Four of These Major HighwayPopulation Density Profiles in the Same Place and Scale.See Slide 65.

I-90 Population Density Profile: Spotfire

My Story

Story and Metadata

Screen Capture

Visualization

My Note: Number of Bins Corresponds to Resolution of the Data.

Keeping Pace With New York

My Comment: These Data Needed To Be Reformatted and Visualized Separately.See Next Slide.

Keeping Pace With New York: Spotfire

My StoryStory and Metadata

Visualizations Data

My Comment: Adjacent Small Multiple Data Sets and Visualizations Make Comparisons Easier.

Population Shift

My Comment: Like To See All the Data in a Trellis Grid as a Times Series For Every Grid Square. See Next Slide.

Population Shift: Spotfire

My Story

Story and Metadata

Trellis Grid Bar Chart

My Note: Compare Above and Below the 100W Longitude.

My Story

Story and Metadata

Visualization

Screen Capture

My Note: Number of Bins Corresponds to Resolution of the Data.

My Story

Story and Metadata

VisualizationImage

My Note: See Links Between Visualization and Sorted Data.

My Note: Bins Not Available in Vertical Display of Data.

Islands of High Income

My Comment: One Map With Three Ranges of Income Would Be Useful.See Next Slide.

Islands of High Income: Spotfire

My Story

Story and Metadata Data

Visualization

The Great Migration

My Comment: Unauthorized to Access. See Next Slide.

The Great Migration

Following the Frontier Line

My Comment: The Data Was Not Accessible So Only The Text and ImagesCould Be Integrated. See Next Slide.

Following the Frontier Line: Spotfire

My Story

Story and MetadataText as Data

Text as Data

My Note: See the Text as Data That Goes With the Images.

Changing Ranks of States

My Comment: This is One of the “Rough Graphics” That Should Have BeenDiscarded. See Next Slide.

Changing Ranks of States: Spotfire

My Story

Story and Metadata

Visualization

Filters

Visualization

Cartograms of State Populations

My Comment: This is Another One of the “Rough Graphics” ThatShould Have Been Discarded. See Next Slide.

Cartograms of State Populations: Spotfire

My Story

Story and Metadata

Visualization

Filters

Before and After 1940

My Comment: The County Data Is Plotted As Colored Dots. It CouldAlso Be Plotted As Colored Areas With Statistical Measures. See Next Slide.

Before and After 1940: Spotfire

Visualization

DataStatistical Summary

My Story

Story and Metadata

My Note: See Changes in US Summary Statistics.

From Physical to Political Geography

My Comment: This Word Cloud Is Possibly Misleading and Should BeExpressed in Numbers of Terms in Each of the Four Categories by Year.See Next Slide.

From Physical to Political Geography: Spotfire

My Story

Story and Metadata

Visualization

Differential City Growth Patterns

My Comment: Even with the Zoom-In Slider, These are Difficult to See and Compare.See Next Slide.

Differential City Growth Patterns: Spotfire

My Story

Story and Metadata

Visualization

Filters

Trend in PopulationTrend in Density

My Comment: It Is Helpful to See All Four of These Major HighwayPopulation Density Profiles in the Same Place and Scale.See Next Slide.

My Story

Story and Metadata

Visualizations

Increasing Urbanization

My Comment: The Geo-Referencing Data and Boundaries Were Not Provided.The Time Series Does Not Shown the Correct Value For 1890. See Next Slide.

Increasing Urbanization: Spotfire

My Story

Story and Metadata

Filters

Visualization

My Note: These Are Not Real Data.

Gaining and Losing Shares

My Comment: These Data Could All Be Displayed At Once Instead of Using a Slider.See Next Slide.

Gaining and Losing Shares: Spotfire

My Story

Story and MetadataData

Visualization

My Note: See Visualization Linked to Data.

Top 20 Cities

My Comment: Latitude and Longitude of Cities Are Not Provided.I Had to Find a Source and Add Them. Size of Names Obscures Locations. See Next Slides.

TIGER Shapefiles: Census Bureau

http://www.census.gov/geo/maps-data/data/tiger-data.html

My Comment: Looked for Longitude and Latitude of Cities Here.

List of United States Cities: Wikipedia

http://en.wikipedia.org/wiki/List_of_United_States_cities_by_population

My Comment: Found Longitude and Latitude of Cities Here.

Albany NY Latitude and Longitude

Web Link

My Comment: Found Longitude and Latitude of Cities Here.

Top 20 Cities: Spotfire

Story and NotesMy Story

Visualization

Conclusions and Recommendations

• The Census Data Visualization Gallery Was Used As An Example of the Digital Government Strategy– All Content = Big Data.

• Spotfire Was Used to Make the Census Visualizations Statistical and Interactive.– Spotfire Was Originally Co-developed by Professor Ben Schneiderman,

University of Maryland, Who Developed the Treemap, Who Was a Consultant to the Census Bureau.

• I Responded to Their Invitation to Provide Comments.– Specifically, the Concept to “Keep focus on good visual ideas not cutting edge

interactivity” Should Be to Focus on Both and To Make the Data More Accessible and Complete. See Next Slide.

• The Census Flow Mapper-Beta is Next.– See http://flowsmapper.geo.census.gov/flowsmapper/flowsmapper.html

Conclusions and Recommendations: Spotfire

https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?CensusVisualizations-Spotfire

Census Data Visualization Gallery As Data For the Digital Government Strategy

data accessible

data table

decade7 http

digital government strategydr

census flows mapper

story webinar

physical creation

booming cities decade

Documents

Data Visualization - dbsloan.github.io · Data...

Data Exploration and Visualization - Big Data | Big Data ·...

Visualization (44413) Data Visualization (44613)

INI235 Library Workshop on Finding and Visualizing Census...

Data Visualization Data Visualization. Business Analytics...

Data Visualization & R Shiny · Data Visualization & R...

Data Visualization Summit - The Innovation...

The Census Bureau’s Data Visualization Mission: To...

census Gateway to Census Data

Introduction to Data Visualization - Virginia Tech ·...

Census 2010: Accessing Census Data

Reaching Broader Audiences with Data Visualization ·...

Coupling visualization and data analysis for knowledge...

DATA VISUALIZATION 1 - University of...

Data Visualization Effectiveness Profile - Perceptual ·...

Introduction to Data Visualization Definition of Data...