Building The Spatial Enterprise for Denver Public Schools Matthew Baker Department of Planning and Analysis
Dec 01, 2014
Building The Spatial Enterprisefor Denver Public Schools
Matthew Baker Department of Planning and Analysis
...Or: How I learned to stop
worrying and love SQL Spatial.
DPS Overview
● Fastest Growing District in USA● 84,424 Students
– Largest district in CO
● 120 Languages● 172 Schools
DPS Student Ethnicity
DPS Student Performance
Overview Planning & Analysis
● DPS Administration Building (900 Grant)– 1870 Lincoln (Summer 2014)
● Demographic Analysis and Projections– Student demand and performance
● Capacity and Resource Planning– Strategic Regional Analysis (SRA)
Spatial Data and Client Applications
Infinite Campus (IC)
StudentsTeachersPrincipals
SuperintendentAdministration
Planning&
Analysis
All DPS database tables
Spatial Data Primary
● School Points● Boundaries● Address Points● Student Households (addresses)● Planning and Forecast Regions● October Count (Student count on Oct. 1)
Address Point File
● Currently ~280,000 geocoded address points● Central Database: Infinite Campus (IC)
– Origin and source of truth for student data
● Weekly sync to IC addresses and GIS addresses– IC doesn't (yet) store spatial data!
Spatial Data Other
City & County of Denver
● Parcels
● Neighborhoods
● Street Centerlines
● Election Precincts
● Police Districts
● Historic Landmark Districts
● Zoning
● Land Use
● Council Districts
Census
● Block Groups
● Blocks
● Tracts
● Birth data
Clients DPS Enterprise
● IC - School Matrix● School by Grade for each address point
● DPS Business Intelligence (BI)● DPS Transportation Department● Operational Data Store (ODS)● Colorado Department of Education
– CDE Pipeline
DPS School Locator
● Enter Address● Get:
– Boundary Schools● Elem● Mid● High
– Transportation Info
● ArcGIS Server● Google Maps API
http://maps.dpsk12.org/
Building the Spatial Enterprise
Design Tasks Initial
● Identify and interview users and stakeholders● Design GIS database
– ArcGIS 9.3.1 to 10.1
● Dev / QA database design● Implement production● Applications
– Re-build ArcGIS Server– Re-build web applications (school locator, etc.)
Workflow Challenges
IC
SQL Data Pull
What's broken?
● Links to Data– Enterprise to GIS
– SQL to Map
● ArcGIS – Access and create spatial data, no link to Enterprise
● SQL Server – Access tabular data
– Enables enterprise-wide analysis
– No access to spatial data
Solution SQL Server Spatial
Improvements
● SQL Server “workspace”– MS SQL Server 2012
– Analysts build and use spatial functions
– Create and modify database views● Use in Excel● Use in ArcMap
● ETL Automation– To / From SDE
– Analysis to DPS Enterprise, etc.
Enabling SQL Spatial with FME
● FME Workbench– From SDE to SQL Spatial Native format
– Create spatial index
– Create Unique Object ID
● “Take What We Need”– Some fields for GIS
– Some for SQL Analysis
Creating Spatial Data with FME
● Assemble query in SQL● Join Spatial data (shape field)
– Using AddressID
– Must also use OBJECTID for use in ArcMap
● Pull data from SQL to ArcSDE for use in GIS
FME Workbench Example
● ~280,000 points x3 feature classes ● Reproject while loading● Create SQL Spatial Index
Other Benefits ETL with FME
● Promoting SDE Tables– Dev to QA
– QA to Prod● Simple tool for DBA to run● Enter User/Pass● ETL runs transparently
SQL Spatial Example
● Need: – Grade 9 students
– Projected School Assignment for 2015
– Within Specific Boundary
● Pulls from:– Projected Students
– Address Master
SELECT count( [Student_Number]) as StudentCount FROM [GISWork].[dpsdata].[ProjectedStudents]join[GISWork].dpsdata.[Address_Master]on (ProjectedStudents.addressID = Address_Master.AddressID)
join GISWork.dpsdata.SchoolBoundaries_Allon (Address_Master.shape.STIntersects(SchoolBoundaries_All.shape) = 1)
where ProjectedGrade = '9'and SchoolBoundaries_All.HIGH_NUM = '464'
group by ProjectedSchoolorder by StudentCount desc
SQL Spatial Example
● Assign special needs transition-grade students to feeder or closest center program – Give top 3 closest schools by program
– Distance to each school
SELECT BT.[StudentID],BT.[LastName],BT.[FirstName],BT.[CurrentGrade],BT.[CurrentSchoolNum],BT.[CurrentSchoolName],BT.[ProjResSch],BT.[ProjectedEdLevel],BT.[CenterProgram] ,BT.[CenterProgramGroup],BT.[shape] ,ROW_NUMBER() OVER(PARTITION BY BT.[StudentID] ORDER BY ROUND((SC.Shape.STDistance(BT.Shape)/5280),2) ASC) AS [OrderOfFeeders],CASE WHEN BT.[CenterProgramGroup] in ('AN','MI') and BT.[CurrentSchoolNum] = '447' THEN '466'
WHEN BT.[CenterProgramGroup] in ('AUT', 'MI-Severe') and BT.[CurrentSchoolNum] = '419' THEN '469'ELSE SC.[CenterSchNum] END as [ProjFeederCenterSchNum]
,CASE WHEN BT.[CenterProgramGroup] in ('AN','MI') and BT.[CurrentSchoolNum] = '447' THEN 'DCIS/NCAS'WHEN BT.[CenterProgramGroup] in ('AUT', 'MI-Severe') and BT.[CurrentSchoolNum] = '419' THEN 'MLK'ELSE SC.[CenterSchName] END as [ProjFeederCenterSchName]
--,CASE WHEN BT.[CenterProgramGroup] in ('AN','MI') and BT.[CurrentSchoolNum] = '447' THEN
,ROUND((SC.Shape.STDistance(BT.Shape)/5280),2) as [DistancetoFeeder]
FROM(SELECT ST.[StudentID]
,ST.[LastName] ,ST.[FirstName] ,ST.[Grade] as [CurrentGrade] ,ST.[SchoolNum] as [CurrentSchoolNum] ,ST.[SchoolName] as [CurrentSchoolName] ,CASE WHEN ST.[grade] = '5th' THEN ST.[SCHOOL_6] WHEN ST.[Grade] = '8th' THEN ST.[SCHOOL_9] WHEN ST.[grade] in ('Pre-K', '2nd') THEN ST.
[resschnum] END as [ProjResSch] --,ST.[ResSchNum] --,ST.[ResSchName] ,CASE WHEN ST.[grade] = '2nd' THEN 'ElemInt' WHEN ST.[grade] = 'Pre-K' THEN 'ElemPri' ELSE ST.[ProjectedEdLevel] END as [ProjectedEdLevel] --,ST.[DOB] --,ST.[Disability] ,ST.[CenterProgram] ,CASE WHEN ST.[CenterProgramGroup] = 'ECE' THEN 'MI' ELSE ST.[CenterProgramGroup] END as [CenterProgramGroup] --,ST.[CenterDescription] ,ST.[addressLine1] ,ST.[addressLine2] ,ST.[addressID] ,ST.[shape]
FROM [GISWork].[dpsdata].[CenterProgramStudents_1114_Geo_WithResSch] as ST
WHERE [Grade] in ('5th','8th','Pre-K') or ([Schoolnum] = '230' and [CenterProgramGroup] = 'AUT') or ([Schoolnum] in ('196','282') and [CenterProgramGroup] = 'MI') or ([Schoolnum] in ('290','275','218','220','284') and [CenterProgramGroup] = 'AN')
)as BT -- Base Table
SQL Spatial Functions
● MSSQL Spatial– STIntersects
– STContains
– STDistance
– STCentroid
Tools Used
● SQL Server 2012– Native Spatial Data / Geometry
● ArcGIS 10.1– ArcGIS Server 10.2
● FME Workbench● PGModeler
–
Thank you!
Matthew Baker
Department of Planning and Analysis
@MapBaker