Automatical monitoring of marine traffic fluxes

Context: the SemaphoresTask 1 - Clean the DataTask 2 - Extract Routes

Ongoing and Future work

A Python-GIS procedure to clean up semaphoredata and automatically build boat synthetic

routes- Ongoing work and future developments -

Annalisa MinelliIUEM-LETG, Brest

April 14, 2014Seminar “Observation et modelisation des activites humaines en

mer cotiere”

Annalisa Minelli Semaphore Data Treatment



Outline

Context: the SemaphoresThe originsTodayThe data obtained

Task 1 - Clean the DataStandardisationCoding: Clean Data By Dictionaries

Task 2 - Extract RoutesLet’s spatialize!Coding: Automatical Extraction of the Routes

Ongoing and Future workOngoing WorkImprovingsFuture work




The originsTodayThe data obtained

The history

I Inspired to the ancient ClaudeChappe’s telegraph, late 1700

I Semaphores were a system tosend news from one place toanother

I The first semaphore’s line wasfrom Paris to Lille

I Located along the coasts and inthe inland

The Louvre Semaphore and the Chappe’s semaphore net, 1792;source: commons.wikimedia.org





The modern Semaphores

I Semaphores constitute a systemof sourveillance, active most ofthe time 24/24 h

I Ideated by Louis Jacob underNapoleon 1st, in the 1806,taking inspiration from Chappe’stelegraph

I All along the French coasts

I 59 semaphores in the net

Schematic map of “modern” semaphores distribution.





Functions

I Since the beginning of 1900 thesemaphores are under militarysupervision

I Growing of matiritime traffic(around 1960) implied moresourveillance marine, militaryand civil

I Cooperation with CROSS(Centre Regional Operationnelde Surveillance et de Sauvetage)

The Semaphore of Saint Mathieu, Ouest Bretagne; source:www.defense.gouv.fr





Data collected

I Each officer records as muchboats as he is able to identify

I These data are stored in .xlsfiles, one for each day

I The informations recorded are:

I date/timeI name of the boatI matricule of the boatI type of boatI routeI azimuth/distance

Example of the Semaphore’s raw data.





Data collected










Data collected










Data collected










Data collected










Data collected










Data collected










Data collected










Data collected










Data exploitation

Data is recorded for:

1. national security reasons

2. control the trafic fluxes at national and international level

3. public service

...many different possibilities:

I Activities of coastal zone management

I Individuation of potential conflicts between different marineactivities (fishing, commerce, leisure etc)

I Evaluation of pressure level on the Marin Protected Areas,more specifically, Iroise sea

I And many different research issues..




StandardisationCoding: Clean Data By Dictionaries

Data acquired

Since now we collected data from 3 different semaphores along theBritanny coast, and the data yelds different years for each one:

I Saint Mathieu: years 2011, 2012, 2013I Toulinguet: year 2011I La Chevre: year 2011





Clean the Data: Lack of shared language

The support of recording is an empty spreadsheet, there are norules in the recording process:

I different encoding for different officers (hours of the day):

I routesI types

I no shared rules for handling missing informations

I eventual errors cannot be prevented

All these things affect negatively an objective data treatment





IUEM-LETG standardisation

An initial standardisation has beencreated by the IUEM-LETG,grouping boats in order to have:

I 16 type of boats

Stage Report; C.Gohn, 2013







I 16 type of boats

I 12 usage for the boats








I 16 type of boats

I 12 usage for the boats

I 106 routes for Saint Mathieusemaphore







I This standardisation, executedfor Saint Mathieu semaphore,for the year 2011, lasted ˜80hours.. a lot of time andenergies

I a necessary beginningI a trace for the future work:

automation









automation









automation






Why Python?

I Open source

I Widely used and growing

I Active scientific community

I Clean language design, highlevel and strong structuralcontrol

I Object oriented

I Efficient (can be compiled)

I PortablePython’s philosophy





Create Dictionaries

The first tool created has the aim to build a primary collection ofoccurrences in order to crate a database (dictionaries) for:

I type of boats in reason of the type

I usage of boats in reason of the type/name

I routes synthesis

createDictionary.py flowchart





Create Dictionaries

I Work based on the firststandardisation for SaintMathieu semaphore, year 2011

I ˜150 Python code rows

I Is a command line tool

I The dictionaries will be recalledin order to clean data from othersempahores and other years

An extract from each dictionary: Route, Type, UsageAnnalisa Minelli Semaphore Data Treatment




Clean Data By Dictionaries

Once the dictionaries (or a core of) are created, let’s use them toclean all the raw data.cleanDataByDicts.py is:


I Is usable by command line, but has also a graphic userinterface written in PyQt

I Takes as input the raw data and the dictionaries and gives inoutput the clean .xls file

I Implements a routine to enlarge dictionaries, if necessary



































The FlowchartRead/extract data

cleanDataByDicts.py flowchart





The FlowchartEnlarge dictionaries






The FlowchartClean data






Clean Data By DictionariesThe GUI

cleanDataByDicts.py GUI




Let’s spatialize!Coding: Automatical Extraction of the Routes

Synthetic Routes

Aim of the analysis: quantify and possibly group the traffic fluxesI using synthetic routes

I using a geometrical grid: vector file created by superposition oftwo regular vector grids

The vector regular grid and gates Annalisa Minelli Semaphore Data Treatment




The Gates approach

Gates approach: allow the software to autonomously find theshortest path between two points

I The output vector file of the path has a direction

I Each path has the same probability to be chosen

Automatical extraction of the shortest path routes





Why GRASS GIS?

I Since it’s Open Source it is possible to verify the fitting of thecode to initial purpose, because everything of it can be read..and deeply understood

I Is a 33rd year old project, sustained by some internationalresearch institutes and continuously in evolution thanks tohundreds of developers and users all over the world

I extremely stableI Is written in many different programming languages (C, shell,

fortran, python, tcltk..), all grouped and compiled togetherI Is powerful in analyzing, modifying and creating raster, vector

and database geographical dataI Composed by more than 300 tools, fruit of the worldwide

cooperation (many different purposes leads to many differentutilities implemented)





v.createRoutes.py

v.createRoutes.py...

I Is a GRASS GIS tool ˜300 Python code rows

I It takes as input a clean semaphore recordings file and a textfile containing the gates’ coordinates

I Gives in Output two vector maps of routes and gates,quantifying the traffic for the given semaphore

An extract from createRoutes.py





v.createRoutes.py










v.createRoutes.py










Flowchart

v.createRoutes.py flowchart





FlowchartPreparing data






FlowchartCreating Grid and Gates






FlowchartComputing Traffic






Create RoutesThe GUI

v.createRoutes.py GUI





The OutputQuantify the traffic

I The passage of each single boatis recorded in each singlesegment of the net (vector linefile) and quantified by aparameter “npass”

I The stationement of each singleboat is recorded in each gate(vector point file) and quantifiedby the same parameter, “npass”





The OutputVector Maps

An example of v.createRoutes.py output for the Toulinguet semaphore, 2011 data




Ongoing WorkImprovingsFuture work

Current Situation

Simulations for semaphores of Saint Mathieu, La Chevre andToulinguet, using 2011 data, have been done





Current Situation





The splitting path issue

I Since all the routes have the same probability and the grid isregular, sometimes for the same gates, the path changes

I to efficiently evaluate the flux of the traffic could be good toavoid splitting fluxes on the same route

I considering the parameter “npass” as speed of the boatsI or computing, before cycling, the unique path through the

gates - this also reduces computational time!
































Grids and Time issues

I Currently each elaboration provides a Grid (one for eachsemaphore)

I If the semaphores are seen like an unique system could begood to have an unique grid for all of them

I Computational time is quite long: 1 hour for 500 records ofthe input file (boats)

I find alternative and more rapid solutions





Grids and Time issues

I Currently each elaboration provides a Grid (one for eachsemaphore)

I If the semaphores are seen like an unique system could begood to have an unique grid for all of them

I Computational time is quite long: 1 hour for 500 records ofthe input file (boats)

I find alternative and more rapid solutions





The same boat seen by different semaphores

I Recognizing the same boat passing through differentsemaphore recordings is also an interesting task





Introduction of Timestamps: TGRASS

One possibility to treat this data in order to recognize the boats isto add the 3rd dimension to the model: the Time.GRASS GIS has a specific branch doing this: TGRASS (TemporalGRASS), recently developed (2012)






I TGRASS treats temporal data assigning time stamps to eachdata (in this case the single records of each spreadsheet)

I Timestamps have not a fixed unity of measure, can beirregular too

I Using the parameters “name of the boat” and such timestampscould be possible to identify the boats from one semaphore tothe other

I In this way it is possible to evaluate traffic in the net ofsemaphores and study the golbal system temporal evolution
































Thank you allMerci pour l’attention

Any suggestion/hint is always more than appreciated

[email protected]


Automatical monitoring of marine traffic fluxes

Science

clean data

semaphores raw data

extract routesongoing

boati matricule

future workoutlinecontext

extract routeslets

ocer records

chappes semaphore net