Automated Bulk Conversion of 250,000+ CAD and Corresponding Raster Files to PDF

Automated Bulk Conversion of 250,000+

CAD and Corresponding Raster Files to PDF

Courtney MaxsonSpatial Business Systems (SBS)

Hello!Courtney Maxson• Certified FME Professional• Certified FME Trainer• M.S. GIS, Johns Hopkins• SBS Employee – 2.5 years

Spatial Business

Systems• Spatial integration firm based

in Lakewood, CO• FME Gold-level reseller• 5 certified FME professionals

and 2 trainers

Problem

Problem• Utility with 250,000 + MicroStation DGN’s• DGN’s only viewable via Bentley Viewer• Not compatible with mobile devices• Access needed in field• Manual creation process

Project Objectives• Convert DGN’s and associated TIFF’s and Borders

to PDF files• Process multiple DGN’s simultaneously• Produce 1 PDF per DGN with few exceptions• Customize output based on DGN type• Automate PDF conversion process

PDF Production Process Diagram

FME Solution

PDF Production Process Diagram

FMESolutionProcess• DGNfilesarecopiedtoatemporarydirectory,thenmergedwiththeborder.

• ClientdevelopedaC#scriptthatinitiatesMicroStationandmergestheborderintotheDGNaslevel“border”.

• Thefilesarecopiedbeforebeingmergedtoavoidalteringtheoriginalfiles.

• FMEcopiestheDGNfilesandassociatedTIFFfiles,callsthemergescript,thenconvertsthemergedDGNfilesandassociatedTIFF’stoPDFfiles.

• Atthebeginningofeachrun,FMEexecutesaPythonscriptthatdeletesallDGNandTIFFfilesfromthetemporarydirectory.

Runner Workspace

WorkspaceParameters:§ TEMP_DIR§ SOURCE_DATASET_CSV§ SOURCE_DATASET_CTL_DGN§ TYPES_TO_COPY_MERGE§ DELTAS_ONLY

Step 1: Copy Files to Temp Directory

StartupPythonScript

§ Empties Temp directory§ Reads DGN directories via

CSV and Directories and File Pathnames Readers

§ Copies DGN’s to Temp directory

§ Copies TIFF’s to Temp directory

Step 2: Merge Temp DGN’s

C#ScriptFunctions:§ MergeDGN’sandborders§ ExtractvaluesforTIFF

alignment• Temppath• LowerLevelX,Y• DimensionY• PixelsizeX,Y

§ CreateRasterTable.csv

Step 3: Create PDF Files

PDF Conversion Workspace

WorkspaceParameters:§ SOURCE_DATASET_DGNV8§ SOURCE_DATASET_TIFF§ SOURCE_DATASET_CSV

§ Converts DGN to single PDF with associated borders and/or TIFF’s

§ Can process multiple file sets simultaneously

§ Generates multiple PDF’s from single source DGN, when required

PDF Conversion: Read DGN Elements

PDF Conversion: Manipulate DGN TextIssues/Workarounds

§ DimensionTextBug• Manualconversion

§ Degree(°)Notavailableinfont• Converttorichtext

§ XMLescapecharacters• Convertedtoescapestring• &replacedwith&

§ ΩandΦ notsupportedbyPDFWriter• ReplacedwithtextOHMand

PH

PDF Conversion: Manipulate Non-Text

§ Removemulti-textnodes§ ResolveLineStyledomains

PDF Conversion: Manipulate TIFF

Issues/Workarounds§ TIFFalignment

• UseRasterTable.csvcreatedbyC#script§ TIFF’sprocessingwithoutDGN§ MultipleDGN’swithsamename

• AdddirectorypathtoRasterTable.csv• Createfanout_attr

PDF Conversion: Style and Write to PDFForeachnon-ZSubeng DGN:§ QueriesSubeng DBtodetermine

#ofoutputPDF’s§ Removesunneededlevelsfor

eachplantype§ Creates1PDFperplan

PDFFormatWriter Transformer

TransformerParameters:§ FME_PROCESS_GROUP_BY§ TARGET_DATASET

§ Determines extent of all elements per fanout_attr

§ Calculates extent orientation§ Formats PDF based on orientation§ Writes DGN and TIFF elements to

different layers

PDF Results

SubstationOneLine

Initial Run

FME Server Solution

• Distributed environment

• 3 Production Servers• 10 Engines

• ProductionServer:o Dual-coreCPUo 8GBRAMo 300GBHardDriveo 2PermanentEngines

• TemporaryProductionServer1:o Quad-coreCPUo 16GBRAMo 300GBHardDriveo 4TemporaryEngines

• TemporaryProductionServer2:o Quad-coreCPUo 16GBRAMo 300GBHardDriveo 4TemporaryEngines

Estimated Performance

Ctldgn Subeng Trans TOTALTotalDGN 122745 28561 551 151857

AvgTime/Job(Sec) 12 26 15TotalEstHrs- 1Engine 409.15 206.27 2.29 617.71TotalEstHrs - 10Engines 61.771

Automation / Long-Term Solution

Long Term FME Server Solution

• 1 Production Server• 2 Engines• Scheduler - Runs every

hour• DELTAS_ONLY = Yes

Deltas Only SelectionForeachnon-ZSubeng DGN:§ QueriesSubeng DBtodetermine

#ofoutputPDF’s§ Removesunneededlevelsfor

eachplantype§ Creates1PDFperplan

• Onlyprocessesnew/modifiedDGN’s• ComparesmodifieddatesofDGNandPDF

Final Steps

On Production Servers:• Install/ Configure FME

Server• Run subset• Make performance

adjustments• Get final client

approval• Kick-off full initial run• Setup Schedule

Thank you!

Courtney [email protected]

https://www.linkedin.com/in/courtney-maxson-50727983/