FME and FMEServer in SDFE best practice FME and FMEServer in SDFE best practice Peter Laulund Peter Laulund
FME and FMEServer in SDFEbest practice
FME and FMEServer in SDFEbest practice
Peter LaulundPeter Laulund
About me
● Working with FME since 1997● Working with FMEServer since 2001● Certifiede professionel 2007● Works at SDFE (KMS, GST)● SQLExecuter and TCLCaller
About SDFE and FME
● FME and FMEServer● Kortforsyningen.download● Transportbåndet● Datamodtagelsen
Databank
Services
Validering
Styrelsen for Dataforsyning og Effektivisering
klienterGIS
klienter AppsGIS
AppsAtomFeed Download
Switchboard
WMS WFSFTP HTTP
Rest
AppsWebApps
Data on demand
FMEServer
Filer
Data Relationelle Raster CSV Proprietærer formater
DownloadWeb services
…..
…..
Databank - Oracle
WorkflowWorkflowToday FME is more than a translator as it has a big number of toolshandling the workflow, like zipping and unzipping, copying files,moving to ftp, running workspaces, etc.
Today FME is more than a translator as it has a big number of toolshandling the workflow, like zipping and unzipping, copying files,moving to ftp, running workspaces, etc.
FME and Workflow
Reading and writing zip files
File Copy writer – including zipping
Directory and File Pathnames Reader
HttpCaller, FtpCaller
AttributeFileReader/Writer
SystemCaller
………
SQLCreator, SQLExecutere
XMLAppender
FeatureReader
FeatureWriter
WorkspaceRunner
FMEServer…
Tools Structural
SDFE INSPIRE project
INSPIRE.fmw
INSPIRE.fmw - Gml_Writer
INSPIRE.fmw - Move2server
Atom feed
Atom feed
● An Atom feed is a xml file with ‘news’● In this context it is an index to data files● The contend is metadata about the dataset - Time, area, name, bounding
box, link to data● We can view it in a browser● We can use it in a machine to machine scenario● FME reads and write the feed as well as the data● Feeds are mandatory in relation to INSPIRE
View in a browser
AtomFeed.fmw
SQLExecutor
● Run any sql – Select, Insert, Delete● Returns the result of the command● Run pl/sql code● Run more than one command
AtomFeed – SQLExecuter sql
FME_SQL_DELIMITER ;
UPDATE LOG_ATOMFEED SET STATUS = 1, SKREVET = @Count(feature_count),TID = ( @Timestamp(^s) - $(t0) +1 ) where JOBID = @Value(jobId);
COMMIT ;
SELECT JOBID, FME_JOB_ID, MASKINE, FILNAVN AS "filecopy_source_dataset", DESTPATH AS "dest_dir"
FROM LOG_ATOMFEEDJOIN ( SELECT UNIQUE DATASET, DESTPATH FROM LDS_OPSLAG.LDS_ATOM_INIT ) ONDATASET = TEMAWHERE JOBID = @ Value(jobId)
AutoAjour
AutoAjour.fmw
Part of INSPIRE job view
AutoAjour jobs this month
Database and system design
FMEServer
AutoAjour.fmw
Oracle views
INS_<FeatureType><user attributes>sdfe_feature_type
sdfe_sortsdfe_sort_key
JOB_INSPIRE
INS_ATOMFEED
LOG_JOB_INSPIRE
INSPIRE.fmw
AtomFeed.fmw
Log2html.fmw
Oracle tables
LOG_INSPIRE
METADATA
GeoDatabank<FeatureType>
LOG_AUTOAJOUR
SEQUENCE.BatchIdSEQUENCE.JobId
Summary
Best practice summary
- Let the database do the job- Use the database for configuration- Log all jobs to the database- Use FME for the workflow- Generic and dynamic workspace- One way trough the workspace- Split job on more than one workspace
RESOURCESYou can finde the INSPIRE data and Atom feed here
● https://download.kortforsyningen.dk/● https://download.kortforsyningen.dk/feeds/
THANK YOU!THANK YOU!