Process Automation for Hydrological Data Mapping over GIS Software By Rohan Jain (08AG1016)
Process Automation for Hydrological Data Mapping over GIS Software
By Rohan Jain (08AG1016)
Introduction● Weather data is available from various
organisations like IMD, CWC through their stations spanning all over the country, periodically.
● The data available from these places can be used for further processing.
● Processing is done via various GIS Software available.
● ArcGIS is one such popular software. It is used for this project
Introduction: Problem● Data is not available in format ArcGIS support● So it cannot be directly imported● Manually importing 10s of thousands of data
is not possible.● Hence data needs to be automatically
converted into an ArcGIS format.● But again data from all the sources is not in a
standardised format.● So each data source needs special attention
Objectives● Automatic conversion of existing
hydrological data of Mahanadi river basin into a universal time-series format
● Mapping of the data into ArcHydro model of the ArcGIS software
Study Area: Description● Mahandi river basin, located between
longitudes 800 30' and 870 E, and latitudes 190 21' and 230 35' N
● 4.3% of the total geographical area of India● Mahanadi was notorious for its devastating
floods.● Hirakud Dam, one of the longest dams
improved the situation greatly.
Mahanadi river basin
Study Area: Data Available● Data from India Meteorological Department
and Central Water Commission (CWC)● Rainfall data● Escape Discharge data● Water Level Data● Data from remote sensing
Methodology: Requirements● ArcGIS (Version 9.3)● ArcHydro tools (Version 1.4) and ArcHydro
data model● Python Programming Language (Version >
2.6)● External Python Libraries
○ xlrd (for reading spreadsheets)○ dbfpy (for writing dBase files)
Methodology: Study Material● Book: ArcHydro - GIS for Water Resources
by David R. Maidment[7]● Book: Arc Hydro Tools - Tutorials● GIS Course Content - University of Texas● Web Resources, Lectures made available by
ESRI[8] (ArcGIS Developer organisation)
Methodology● For interfacing with ArcGIS dBase (*.dbf)
database file format used● dBase is a popular database and ArcGIS
relies on it itself for storing data, so a good choice for using it for our task
● Python libraries available (dbfpy)● For data model to store the time series, used
the TimeSeries model from ArcHydro data models.
Methodology: Data Model● FeatureID: ID of the feature for which this
time series data exists. IMD Stations, CWC Gauges etc.
● TSTypeID: ID of the time series type. We have Precipitaion, Discharge, Water Level etc defined
● TSDateTime: The date and time of individual data
● TSValue: Individual data value
Methodology: Automation1. The data obtained from various organisations
is converted into a format which follows python data structures.
2. Separate (dBase) files contain information about HydroIDs (which will help find FeatureID). The information is extracted and used to find FeatureIDs for station names
3. Time Series is generated and then further published as dBase files for use with ArcGIS software.
The Data Conversion Process
Methodology: Code Written● Modules
○ These are for generic tasks which are applicable to all data sources
○ timeseries.py■ Takes care of timeseries related internal tasks■ Also generates the dBase files
○ stations.py: ■ Process the HydroIDs (FeatureIDs in Time
Series database)■ Fetches ID - Name info about the stations
Methodology: Code Written● Individual Data Source Scripts
○ Since each data source provides information in a different format, they all need a separate script.
○ These scripts process the raw data to pythonic format and then generate time series database
● Written in Python Programming Language● Total roughly 450 lines of python code● A C/Java equivalent will easily measure 2-3
times
Results● Set up an initial project with correct directory
hierarchy and install python + the required libraries
● Then, on execution of the scripts the time series files are generated automatically
● The time series files can then be imported into ArcGIS table
Results: Loading Data
Loading data into a Time Series table in ArcCatalog
Result: Loading Data
ArcCatalog data loading dialogs
Result: Loading DataDisplaying data after being imported.
Result: Processing DataProcessing the data in ArcMap using ArcHydro tools
Result: Processing Data
ArcMap Processing the Discharge Time Series
Future Work● Rewrite the modules using Object Oriented
Approach to improve the code quality and future additions of code easier
● Apart from this Rainfall, Discharge, Water Level series more data can be obtained and added
Thank You