FME and FMEServer in SDFE best practice...About me Working with FME since 1997 Working with FMEServer since 2001 Certifiede professionel 2007 Works at SDFE (KMS, GST) SQLExecuter and

FME and FMEServer in SDFEbest practice

FME and FMEServer in SDFEbest practice

Peter LaulundPeter Laulund

About me

● Working with FME since 1997● Working with FMEServer since 2001● Certifiede professionel 2007● Works at SDFE (KMS, GST)● SQLExecuter and TCLCaller

About SDFE and FME

● FME and FMEServer● Kortforsyningen.download● Transportbåndet● Datamodtagelsen

Databank

Services

Validering

Styrelsen for Dataforsyning og Effektivisering

klienterGIS

klienter AppsGIS

AppsAtomFeed Download

Switchboard

WMS WFSFTP HTTP

Rest

AppsWebApps

Data on demand

FMEServer

Filer

Data Relationelle Raster CSV Proprietærer formater

DownloadWeb services

…..

…..

Databank - Oracle

WorkflowWorkflowToday FME is more than a translator as it has a big number of toolshandling the workflow, like zipping and unzipping, copying files,moving to ftp, running workspaces, etc.

Today FME is more than a translator as it has a big number of toolshandling the workflow, like zipping and unzipping, copying files,moving to ftp, running workspaces, etc.

FME and Workflow

Reading and writing zip files

File Copy writer – including zipping

Directory and File Pathnames Reader

HttpCaller, FtpCaller

AttributeFileReader/Writer

SystemCaller

………

SQLCreator, SQLExecutere

XMLAppender

FeatureReader

FeatureWriter

WorkspaceRunner

FMEServer…

Tools Structural

SDFE INSPIRE project

INSPIRE.fmw

INSPIRE.fmw - Gml_Writer

INSPIRE.fmw - Move2server

Atom feed

Atom feed

● An Atom feed is a xml file with ‘news’● In this context it is an index to data files● The contend is metadata about the dataset - Time, area, name, bounding

box, link to data● We can view it in a browser● We can use it in a machine to machine scenario● FME reads and write the feed as well as the data● Feeds are mandatory in relation to INSPIRE

View in a browser

AtomFeed.fmw

SQLExecutor

● Run any sql – Select, Insert, Delete● Returns the result of the command● Run pl/sql code● Run more than one command

AtomFeed – SQLExecuter sql

FME_SQL_DELIMITER ;

UPDATE LOG_ATOMFEED SET STATUS = 1, SKREVET = @Count(feature_count),TID = ( @Timestamp(^s) - $(t0) +1 ) where JOBID = @Value(jobId);

COMMIT ;

SELECT JOBID, FME_JOB_ID, MASKINE, FILNAVN AS "filecopy_source_dataset", DESTPATH AS "dest_dir"

FROM LOG_ATOMFEEDJOIN ( SELECT UNIQUE DATASET, DESTPATH FROM LDS_OPSLAG.LDS_ATOM_INIT ) ONDATASET = TEMAWHERE JOBID = @ Value(jobId)

AutoAjour

AutoAjour.fmw

Part of INSPIRE job view

AutoAjour jobs this month

Database and system design

FMEServer

AutoAjour.fmw

Oracle views

INS_<FeatureType><user attributes>sdfe_feature_type

sdfe_sortsdfe_sort_key

JOB_INSPIRE

INS_ATOMFEED

LOG_JOB_INSPIRE

INSPIRE.fmw

AtomFeed.fmw

Log2html.fmw

Oracle tables

LOG_INSPIRE

METADATA

GeoDatabank<FeatureType>

LOG_AUTOAJOUR

SEQUENCE.BatchIdSEQUENCE.JobId

Summary

Best practice summary

- Let the database do the job- Use the database for configuration- Log all jobs to the database- Use FME for the workflow- Generic and dynamic workspace- One way trough the workspace- Split job on more than one workspace

RESOURCESYou can finde the INSPIRE data and Atom feed here

● https://download.kortforsyningen.dk/● https://download.kortforsyningen.dk/feeds/

THANK YOU!THANK YOU!

Peter Laulundpelau@sdfe.dkPeter Laulundpelau@sdfe.dk