Deliverable D 2.4 Report on functional testing of fully ...

G A 730836 P a g e 1 | 32

Deliverable D 2.4 Report on functional testing of fully integrated

multi-sensor obstacle detection system

Reviewed: (no)

Project acronym: SMART

Starting date: 01/10/2016

Duration (in months): 36

Call (part) identifier: H2020-S2RJU-OC-2015-01-2

Grant agreement no: 730836

Due date of deliverable: Month 24

Actual submission date: 23/04/2020

Responsible/Author: Dragan Nikolić , SOVA

Dissemination level: PU

Status: Submitted

Ref. Ares(2020)2209310 - 23/04/2020

G A 730836 P a g e 2 | 32

Document history

Revision Date Description

01 10/01/2019 First draft

02 06/08/2019 Second draft

03 11/09/2019 Submitted version

04 23/03/2020 Revision according to review comments

05 23/04/2020 Submitted revised version

Report contributors

Name Beneficiary Short Name

Details of contribution

Danijela Ristić Durrant UB Initial draft, document structure; Chapters 1,3,4

Milan Banić UNI Chapter 5; Section 7.4

Muhammad Abdul Haseeb

UB Chapter 6; Section 7.1

Dragan Nikolić SOVA Section 7.2; Section 7.3

Danijela Ristić Durrant UB Final integration of all inputs

Dragan Nikolić SOVA Final review

Danijela Ristić Durrant UB 1st draft deliverable revision

Danijela Ristić Durrant UB Section 6.2

Milan Banić UNI Inputs to Section 6.2 and

Dragan Nikolić SOVA Final review

Danijela Ristić Durrant UB Final revised document

G A 730836 P a g e 3 | 32

Table of Contents 1. Executive Summary ................................................................................................................ 4

2. Abbreviations and acronyms .................................................................................................. 5

3. Background ............................................................................................................................. 6

4. Objective/Aim ......................................................................................................................... 6

5. Integrated SMART Obstacle Detection System (ODS) ............................................................ 7

6. SMART dynamic field tests ..................................................................................................... 8

6.1 Pre-run tests ........................................................................................................................... 9

6.2 Functional testing against functional requirements ............................................................ 10

7. Functional testing of the sub-systems of the integrated ODS ............................................. 13

7.1 Multi-RGB cameras ............................................................................................................... 14

7.2 Thermal camera subsystem .................................................................................................. 17

7.3 Night-vision image intensifier subsystem............................................................................. 19

7.4 OD system integration .......................................................................................................... 22

7. Conclusion ............................................................................................................................. 31

8. References ............................................................................................................................ 32

G A 730836 P a g e 4 | 32

1. Executive Summary The main goal of SMART project is to increase the effectiveness and capacity of rail freight

through the contribution to automation of railway cargo haul at European railways. Two SMART

working streams are:

Development of a prototype of an autonomous obstacle detection system (ODS),

Development of a real-time marshalling yard management system.

The SMART solution for obstacle detection (OD) provides prototype hardware and software

algorithms for obstacle detection on the rail tracks ahead of the locomotive. The system

combines different vision technologies: thermal camera, night vision sensor (camera augmented

with image intensifier), multi RGB cameras, and laser scanner in order to create a multi-sensor

system for mid (up to 200 m) and long range (up to 1000 m) obstacle detection during day and

night operation, as well as during operation in poor visibility condition.

This deliverable document, (D2.4), reports the activities, effort and work undertaken in Work

Package 2 (WP2- Development of obstacle detection system prototype) of the SMART project,

focused on dynamic real-world field tests of the integrated obstacle detection system. In

particular, this document includes description of dynamic field tests performed in July 2018 for

the purpose of functional testing of fully integrated SMART OD system.

The following documents provide additional perspectives for the present work:

1. D1.1 Obstacle Detection System Requirements and Specification. 2. D2.1 Report on selected sensors for multi-sensory system for obstacle detection. 3. D2.2 Design of the passive vibration isolation system 4. D2.3 Report on sub systems conformance testing 5. D3.1 Report on algorithms for 2D image processing. 6. D3.2 Report on SMART data fusion and distance calculations 7. D7.1 Report on evaluation of developed SMART technologies

G A 730836 P a g e 5 | 32

2. Abbreviations and acronyms

Abbreviation / Acronyms Description FPS Frames per second

ODS Obstacle Detection System

OD Obstacle Detection

ML Machine Learning

ROI Region of Interest (in an image)

ROS Robot Operating System

RGB RGB (Red Green Blue) camera image

S2R JU Shift2Rail Joint Undertaking

SMART Smart Automation of Rail Transport

WP Work Package

G A 730836 P a g e 6 | 32

3. Background The present document constitutes the Deliverable D2.4 “Report on functional testing of fully integrated multi-sensor obstacle detection system” in the framework of the TD 5.6 Autonomous train operation, task 2, 2016-2019) of IP5 (MAAP version November 2015).

4. Objective/Aim This document has been prepared to provide report on functional testing of integrated obstacle detection system developed within the Obstacle Detection work stream of the project SMART. SMART prototype of a novel reliable on-board system for obstacle detection on railway mainlines has been developed with a long-term goal of integration into planned Autonomous Train Operation (ATO) module over standardized interface. In this way, SMART will make an important contribution to the vision of a fully automated rail freight system (IP5 – TD5.6: “Autonomous Train Operation”).

G A 730836 P a g e 7 | 32

5. Integrated SMART Obstacle Detection System (ODS)

SMART ODS is based on combination of different vision technologies: thermal camera, night vision sensor (camera augmented with image intensifier), multi RGB cameras, and laser scanner (LiDAR). The idea behind is to create a multi-sensor system for mid (up to 200 m) and long range (up to 1000 m) obstacle detection, which is independent of light and weather conditions.

All SMART OD sensors, together with network and power components, are integrated into custom designed sensors’ housing as shown in Figure 1.

(a)

(b)

Figure 1. Integrated SMART OD system. (a) CAD model, (b) photo of the integrated OD system with different components labelled

The meaning of the abbreviations of all the components labeled in Figure 1(b) can be seen in Figure 2. Figure 2 shows power and data flow for SMART OD integrated system mounted onto locomotive Serbia KARGO series 444, which was used as the test locomotive in dynamic field tests

G A 730836 P a g e 8 | 32

as explained in Chapter 6. As it can be seen in Figure 2, sensors data via network switch (NS) go to ODS processing and data storage elements located in the locomotive drive cabin. Also, vibration sensor (VS) data go to ODS processing and data storage units via processing unit (A) placed in the sensors’ housing.

Figure 2. The scheme of the power and data flow for SMART OD integrated system mounted onto locomotive

6. SMART dynamic field tests

The functional testing of SMART integrated system was performed during the dynamic field tests in July 2018. The integrated OD system was mounted onto the moving locomotive SERBIA CARGO 444-018 (Figure 3(a)) running without attached wagons from the “Red cross” station to the Niš Marshalling Yard in length of 3.1 km (Figure 3(b)). The functional testing was performed in a test run environment based on permit issued for test run by Infrastructure of Serbia Railways. During this field tests, the members of the SMART team mimicked possible obstacles (persons) on the rail tracks, crossing the rail tracks according to previously defined scenario at the safe distance from the locomotive bearing in mind the train speed of 40 km/h on the considered tracks section.

G A 730836 P a g e 9 | 32

(a)

(b)

Fig.3. (a) SMART ODS sensors integrated into sensors’ housing mounted on the frontal profile of a locomotive below the headlights. (b)

The sensors’ housing was vibration isolated to prevent transmitting of vibrations from the locomotive onto the cameras as moving vehicle vibration can severely deteriorate quality of acquired images. The vibration isolation system was designed with the rubber-metal springs, as described in Deliverable 2.2

6.1 Pre-run tests After mounting of the sensors’ housing onto the locomotive, with sensors and other necessary components integrated inside the housing, several pre-run tests were performed while locomotive was stationary in order to achieve high performance, failures free, functioning of the ODS during the run tests. Those pre-run tests can be classified as Hardware and Software tests as described in following. Hardware: Sensors Alignment: In order to allow the maximum view of the straight rail tracks as well as of the rail tracks in curves, sensors integrated into sensors’ housing as first were manually aligned with the rail tracks while parallel to the ground level. Software: SMART software modules were firstly individually tested. The SMART data acquisition modules were firstly checked for the reliable data acquisition. The acquisition rate in frames per second (FPS) for all vision sensors were pre-defined. The maximum possible FPS were configured so to avoid overload of the network bandwidth. The RGB and night vision cameras were configured at 6 FPS at full image resolution of 2592x1944, whereas thermal camera was configured at 9 FPS. The zooming factors of SMART RGB cameras were also configured in a way to cover maximum range so to enable viewing of close and distant objects. The zooming factors of three SMART RGB cameras were set-up to 0%, 80% and 25% for RGB cameras C1, C2 and C3 respectively. After testing the data acquisition modules and after configuration of the zooming factors, the data recording module, which enables recording of data during the experiments, was tested.

SMART ODS

G A 730836 P a g e 10 | 32

The available storage size in SMART PC was also checked to enable certain recording of data without any failures. The recording of data from all the sensors was simultaneously performed during the testing. The object detection and distance estimation modules were also tested in pre-run test, while locomotive was stationary. Further, the testing of graphical user interface module was done to enable real-time visualization of processing results, outputs of the object detection and distance estimation module. The above described testing of individual hardware and software modules made possible to troubleshoot and to assure the sub-system’s performance. However, further the complete integrated system, with all individual sub-systems running, was also tested in pre-run tests. As detailed in Deliverable 1.1, the SMART ODS software is based on Robot Operating System (ROS) [3], which supports modular implementation and integration of sub-systems into complex system such as SMART ODS. The SMART ODS system modular framework design enables that each module (sub-system) is independent and does not influence any other module’s (sub-system’s) performance. For example, during any sensors or software failure, the other sub-systems do not get affected and overall integrated system remains running. The advanced ROS tools enabled troubleshooting of SMART integrated ODS in a convenient way.

6.2 Functional testing against functional requirements The functional dynamic tests protocols were defined so to evaluate fulfilling of the functional requirements (FR) defined in Deliverable 1.1. An overview of SMART sub-systems functional testing against functional requirements, defined in D1.1, is given by the compatibility matrix in Figure 4. The green tick symbols in given compatibility matrix denote that the functionalities of particular sub-systems were tested in the dynamic field tests reported in this deliverable D2.4. However, bearing in mind that the functional dynamic tests were timely and geographically limited, it was not possible to test the OD sub-systems against all functional requirements, as defined in D1.1. For example, as here described tests were performed in July 2018, they were limited to particular environment and light conditions (summer, warm weather with environmental temperature of 38°C) so that a full functional testing with respect to FR3-Robust system to environmental conditions could not be performed. Because of this, here described functional tests should be considered as complemented by field tests performed in different environment and light conditions during the SMART project lifetime. The report on full testing against FR3 in all complementary field tests, including those described in this deliverable D2.4, is given in the deliverable D7.1 as marked in Figure 4.

It is important to note that limited time for dynamic field tests particularly influenced possibilities for functional testing of the night vision camera. Namely, as night vision systems uses technology that allows data recording in no light and low-light conditions, the night vision camera images were captured only at the end of the dynamic field tests as the end train station was reached in dusk. In this way, the functionality of night vision sub-system could be tested

G A 730836 P a g e 11 | 32

against FR1 and FR2 (frontal and on-board OD) as well as against FR3 and FR4 (robustness with respect to poor illumination condition and long-range OD). However, night vision sub-system could not be tested against FR6 (sensor fusion) as marked in Figure 4. As reported in deliverable D3.3, the SMART sensor fusion algorithms are based on a novel machine learning-based method for object detection and distance estimation from multiple cameras. As a machine learning-based method, its development demanded a large amount of data synchrony recorded by multiple cameras. As all dynamic field tests performed during the SMART project lifetime were recorded mainly in daily operational conditions, it was not possible to record sufficient amount of night vision camera data to be used for training of SMART sensor fusion machine learning model. Because of this, as reported in deliverable D3.3, the SMART OD sensor fusion functionality was implemented and evaluated on RGB and Thermal vision sensors data. However, the principle for night vision data fusion with other sensors types would be the same as in case of RGB and Thermal cameras, so that the good results presented in D3.3 assures that the good results would be achieved with night vision sensor as well.

Fig.4. (a) Overview of the components of the SMART OD integrated system as tested against

the functional requirements (FR) defined in D1.1

In addition, as functional tests described in this deliverable D2.4 aimed on the first functional testing of the integrated SMART OD system mounted on an operational test locomotive, bearing in mind the limited testing time (limited to the duration of allowed running of the test locomotive) and the focus on safety, the functionalities of individual sub-systems, which already were tested in sub-systems conformance testing and consequently were reported in related deliverables, were not tested in here described dynamic field tests. This is the case with some of

G A 730836 P a g e 12 | 32

the functionalities of SMART software for OD. For example, the rail track detection functionality was switched off during the functional testing in July 2018, as it was tested earlier as described in deliverables D3.1 and D2.3. Also, the Sensor Fusion functionality of SMART OD software, was not included in here described functional tests as it was reported in deliverable D3.3. All these related deliverables that complement this deliverable D2.4, are noted in Figure 4. Abbreviation “n.a.” in Figure 4 stands for “not applicable” and refers to functional requirements, which are not applicable to particular sub-systems. Also, it has to be noted that in following section the sub-systems functional testing tables were given only for the sub-systems, which were actively used in dynamic field tests. Namely, as one of the outcomes of sub-system conformance testing (Deliverable 2.3) was that selected laser scanner could be used as ground truth distance measure in available range 100 m – 300 m, this sensor was mainly employed in creating datasets used for development of machine learning (ML) based algorithms for object detection and distance estimation (as described in Deliverable 3.2), and it was not used in online/real-time system implementation in here described dynamic field tests. Because of this, the following section does not include the functional testing table of laser scanner. Also, as detailed in Deliverables 3.1 and 3.2, stereo-vision distance estimation was prone to errors due to uncertainty in stereo-vision based methodology. In order to overcome these problems of stereo-vision, as detailed in Deliverable 3.2, all SMRT RGB cameras were finally used as mono cameras and SMART ML-based methods were applied to these mono cameras to estimate object distance. Because of this, multi stereo-vision system defined in D1.1, D2.1 and D3.1, whose testing was reported in D2.3, in here described dynamic field tests was considered through three individual (mono) RGB cameras. Regarding the FR4-long-range OD, it is important to note that because of the rail-tracks configuration on the route approved for dynamic field tests, there were no long segments viewed with onboard sensors during the running tests where accidental obstacles appeared. In order to test the OD functionalities, during the dynamic field tests, the members of the SMART team mimicked possible obstacles (persons) on the rail tracks, crossing the rail tracks according to previously defined scenarios at the safe distance from the locomotive bearing in mind the train speed of 40 km/h on the considered tracks sections. These scenarios were defined on the/near the level crossings and when approaching/in the train stations where only mid range segments were viewed by onboard sensors. Because of this, only mid-range object detection (up to and about 200 m) in real-world environment could be demonstrated. However, as achieved mid-range obstacle detection is beyond state-of-the-art range in obstacle detection for autonomous driving (about 100 m), here presented results on the OD range could be considered as fulfilling the FR4. Here presented results are to be considered as complemented with long-range OD, achieved in different SMART static field tests that have been reported in D7.1.

G A 730836 P a g e 13 | 32

7. Functional testing of the sub-systems of the integrated ODS The objective of the functional testing of the integrated ODS was to test the sub-systems of the integrated ODS in pre-run as well as in run tests during the dynamic field tests described in Chapter 6. For a unified documentation of the SMART OD sub-system functional testing, a table was made for each sub-system describing it, its parts, acceptance tests and documentation. Each partner is responsible for specifying tests and documentation for their own sub-modules; however their requirements must be accepted by the other partners. Below is an example for a sub-system table:

SUB-SYSTEM

ABBREVIATION

Status

Description Sub-system integrated into

SMART ODS

Responsible Responsible partner

Functional tests Short description of

functional tests done for the

sub-system

If available illustrating photo or/and additional

information

Acceptance test result Functional test approved

What kind of

performances could be

expected for the sub-

system based on the

requirements of D1.1

Summary of sub-system

functional tests results with

respect to requirements

defined in D1.1

G A 730836 P a g e 14 | 32

7.1 Multi-RGB cameras

Three zooming cameras from the imaging source (TIS) were selected for SMART obstacle detection (OD) system due to their long range and high resolution [1]. The zoom camera DFK Z12GP031 are categorized as GigE interface cameras which provides high data transfer rate and high bandwidth.

Table 1

RGB MONO Status

Description Three zoom

RGB cameras

integrated into

SMART ODS

Responsible UB

Functional

tests

Setup

(configuration):

- Setting up the

parameters for

mono cameras;

different focal

lengths

(zooming

factors)

Functional

tests:

- Connectivity

test between

cameras and

the PC

- Confirming

the data

received from

cameras

-(Images to the

right) dynamic

field tests on

the rail track in

Serbia

(approved by

Serbian

Railways). C1

camera image,

zooming factor

0; C2 camera

image,

zooming factor

80; C3 camera

image,

zooming factor

G A 730836 P a g e 15 | 32

25;

Acceptance

test result

Functional test

approved

(images to the

right) dynamic

field tests on

rail tracks in

Serbia

(approved by

Serbian

Railway).

Detected object

(person) in

three

subsequent

frames of

camera C2 with

estimated

object

distances of

121.74 m,

114.16 m and

86.37 m

respectively as

opposed to

ground truth

distances of

100.12 m,

91.95 m and

83.66 m

respectively.

The ground

truth distances

were calculated

using GPS

coordinates of

G A 730836 P a g e 16 | 32

train, Google

maps GPS

coordinates

(e.g. crossing)

and railway

infrastructure

information

(e.g. distance

between

pillars)

.

What kind of

performances

could be

expected for

the sub-

system based

on the

requirements

of D1.1

Mid (Long)

range detecting

of objects,

potential

obstacles, on

the rail tracks

and near the

rail tracks

ahead of the

running

locomotive

with on-board

vision sensor

(FR1. FR2 and

FR4).

Different

zooming

factors can be

adjusted to

enable

covering of

different

distance ranges

with individual

cameras

(requirements:

FR4); Live

camera image

are transmitted

to the SMART

G A 730836 P a g e 17 | 32

PC and

processed by

SMART OD

software (FR7

and FR8)

7.2 Thermal camera subsystem

Thermal camera integrated into SMART ODS is FLIR TAU2 model with resolution of 640x480 pixels and 100mm objective lens [2]. Its small size and the fact that sensor requires no cooling, together with possibility to expand it with adapter for gigabit Ethernet communication, make it suitable for this project.

Table 2

THERMAL Status

Description Thermal camera

integrated into

SMART ODS

Responsible SOVA

Functional tests Tests performed

during dynamic

field tests with

thermal camera

integrated into

sensors’ housing,

with Germanium

glass protective

window.

Images on the

right: Tau2 thermal

camera integrated

into sensors’

housing before

mounting on the

locomotive (top)

and after mounting

on the frontal

locomotive profile

(bottom) as

marked with red

circles.

G A 730836 P a g e 18 | 32

Acceptance test

result

Functional test

approved

(images to the

right) dynamic

field tests on rail

tracks in Serbia

(approved by

Serbian Railway).

Images of good

quality for further

processing

captured in day

condition at

temperature of

38°C.

Detected objects

(persons) on rail

track in two

subsequent frames

of on-board

thermal camera

with estimated

object distances of

185.74 m and

166.94 m (person

1) and 227,08 m

and 209,39 m

(person 2) as

opposed to ground

truth distances of

191.48 m and

175,06 m (person

1) and 235.93 m

and 211,25 m

(person 2). The

ground truth

distances were

calculated using

GPS coordinates of

train, Google maps

GPS coordinates

(e.g. crossing) and

railway

infrastructure

G A 730836 P a g e 19 | 32

information (e.g.

distance between

pillars)

What kind of

performances

could be expected

for the sub-system

based on the

requirements of

D1.1

Mid (long)

detecting of

objects, potential

obstacles, on the

rail tracks and near

the rail tracks

ahead of the

running

locomotive with

on-board vision

sensor (FR1, FR2

and FR4).

Live camera image

are transmitted to

the SMART PC

and processed by

SMART OD

software (FR7 and

FR8)

7.3 Night-vision image intensifier subsystem

SMART night vision camera subsystem consists of 4 main parts: objective lens, image intensifier, coupling optics and CMOS camera, as illustrated in Figure 4.

Figure 4 Night vision sensor: Camera augmented with image intensifier

G A 730836 P a g e 20 | 32

Table 3

NIGHT

VISION

Status

Description Night Vision

camera integrated

into SMART

ODS

Responsible SOVA

Conformance

tests

Tests completed

in poor visibility

condition, in

dusk.

Images on the

right show Night

Vision camera

integrated into

sensors’ housing

before mounting

on the locomotive

(top) and after

mounting on the

frontal

locomotive

profile (bottom)

as marked

prototype (red

circle).

Acceptance test

result

Functional test

approved

(image to the

right) dynamic

field test on rail

tracks in Serbia

(approved by

Serbian Railway).

Detection of

objects (persons)

on the/ near to

rail tracks with

night vision

camera image.

This image was

G A 730836 P a g e 21 | 32

captured in poor

visibility

conditions- dusk-

at the end of the

dynamic field

tests (the end

train station was

reached in dusk).

What kind of

performances

could be

expected for the

sub-system

based on the

requirements of

D1.1

Mid (long)

detecting of

objects, potential

obstacles, on the

rail tracks and

near the rail

tracks ahead of

the locomotive

with on-board

vision sensor

(FR1, FR2 and

FR4).

Object detection

in poor light

conditions

(requirements:

FR3);

Live camera

image are

transmitted to the

SMART PC and

processed by

SMART OD

software (FR7

and FR8)

G A 730836 P a g e 22 | 32

7.4 OD system integration

Table 4

NETWORK Status

Description Network

component

integrated into

SMART ODS

Responsible UB, UNI

Conformance tests Ethernet switch

Setup:

- all on-board

devices

connected to

the Ethernet

Switch using

CAT7 Ethernet

cables of

appropriate

length

- switch

connected to

processing and

data storage

computer

located in

locomotive

driver cabin

using CAT7

cable with 7.5

m length.

- static IP

addresses

configured on

each device

Functional

tests:

- connectivity

tests between

ODS sensors

and switch

(ping)

- connectivity

tests between

ODS sensors

and processing

and data

storage

NETGEAR XS708T

G A 730836 P a g e 23 | 32

computer

(ping)

- real time

sensor

connection

- testing of

maximal frame

rate at full

resolution of all

sensors

Acceptance test result Units were

operative

What kind of

performances could be

expected for the sub-

system based on the

requirements of D1.1

The established

network

enables the

real-time

acquisition of

data from all of

the sensors for

further

processing as

there was no

frame loss

during

capturing of

sensor data (as

rosbags).

Furthermore,

the network

performance is

sufficient to

provide real-

time sensor

visualisation

for the HMI

(requirements:

FR7)

Table 5

Processing Status

Description Processing and data storage

integrated into SMART ODS

Responsible UNI

G A 730836 P a g e 24 | 32

Conformance tests Processing and data storage

PC

Hardware setup:

- Processor: INTEL Core i9-

7900X

- Motherboard: ASUS

RAMPAGE VI EXTREME

- Memory: KINGSTON

DIMM DDR4 32GB

(4x8GB)

- GPU: 2 x ROG Strix

GeForce GTX 1080 Ti OC

edition 11GB GDDR5X

- Storage: HDD SSD M.2

NVMe Samsung 500GB 960

Pro

Case: MasterCase H500P

- Power supply: LC Power

LC1000 v2.4 80 plus

Platinum

- Cooling: COOLER

MASTER MasterLiquid

ML240L

- Connected to network

switch using CAT7 cable

with 7.5 m length

Software setup (installation

of required software

packages):

- Ubuntu 14.04 64-bit

- Drivers for 10 GB Lan and

dual GPU

- ROS Indigo Igloo Full

Desktop

- Qt 4.8.1

- CUDA 8.1

- OPEN CV

- Team Viewer 13

Setup (configuration):

- setting up a static IP

- configuration of ROS

environment

- configuration ROS nodes

for all the sensors

G A 730836 P a g e 25 | 32

Functional tests:

-- basic connectivity tests

between ODS sensors and

processing and data storage

computer (ping)

- real time sensor connection

- testing of maximal frame

rate at full resolution of all

sensors

- testing remote desktop

connections to the device

- testing of developed ROS

nodes

- CUDA testing

- OPEN CV testing

- OPENGL testing (glxgears)

- real time processing of data

from sensors

Acceptance test result Functional test

approved

What kind of

performances could be

expected for the sub-

system based on the

requirements of D1.1

The ODS processing and

storage system successfully

acquired and stored data for

further processing aimed at

obstacle detection, long

range obstacle detection and

rail track detection. The ROS

(Robot Operating System)

implementation was made

which enabled recording and

re-playing of sensor data

from rosbags. The recording

of sensor data enables the

offline work and testing of

software for obstacle

detection with real-world

experiments (requirements:

FR1, FR2, FR4, FR5, FR7,

FR8)

Table 6

TRL 5 Demonstrator Status

Description Integrated TRL4

prototype (ODS

housing with

G A 730836 P a g e 26 | 32

integrated sensors,

power supply

elements and

network

components) is

mounted onto the

test vehicle for

dynamic tests

(tests with running

train)-TRL5 OD

Demonstrator

Responsible UNI

Conformance tests Images to the

right: mounting

construction

(construction for

mounting the

sensors’ housing)

assembled with

the locomotive;

levelling tests for

mounting

construction; final

manual

adjustments of

sensor positions

inside the housing

before the initial

test run; housing

mounted on the

frontal locomotive

profile (connection

with locomotive

cab through the

window marked

with green circle);

location of triaxial

accelerometer on

the mounting

construction

(green circle) and

rubber-metal

elements used for

vibration

suppression (red

circles).

Hardware:

- integrated

SMART ODS

G A 730836 P a g e 27 | 32

system

- mass ballast for

locomotive

mounting

- construction for

locomotive

mounting

- 4 rubber metal

mounts Trelleborg

M50

- 2 triaxial

accelerometers

Functional tests:

- locomotive

mounting

- determination of

time necessary for

vehicle mounting

- power supply of

integrated system

- connection to

ODS processing

and data storage

- housing vehicle

driver visibility

disruption

power loss during

change of power

section

- performance

assessment of the

passive vibration

suppression

system

G A 730836 P a g e 28 | 32

Acceptance test result Functional test

approved

Functional tests were successfully performed during the

test run. There was no damage to the locomotive or

mounted equipment, as well as to the railway infrastructure,

during the test performed. The mounting of the whole

system onto the locomotive was performed in 25 minutes

which was below the requirement of 30 min issued by

owner of the locomotive (SERBIA Kargo). The

dismounting was performed in 15 minutes which was again

below the requirement imposed by the locomotive owner.

During the run-tests, the system was able to detect

obstacles on the track. The designed passive vibration

suppression system lowered significantly the level of

vibrations transmitted from the locomotive to the ODS

sensor housing in all three directions. Functional testing

results were accepted by the engineering and safety

commission of locomotive owner and infrastructure

manager, so they issued permit for dynamic testing in

regular cargo haul operation (as it will be described in

D7.1-Evaluation).

What kind of

performances could

be expected for the

sub-system based on

the requirements of

D1.1

Images to the

right: Diagrams of

acceleration in

vertical,

longitudinal and

lateral direction

for two measuring

positions, on the

mounting plate

(without

suppression) and

in the ODS

housing (with

suppression).

The integrated

system TRL 4

ODS demonstrator

was mounted onto

the locomotive

during dynamic

field tests. The

system was

mounted onto the

locomotive

without any

modification of

the locomotive.

The mounted

system housing

The designed passive suppression system successfully

lowers the level of vibrations transmitting from locomotive

during test runs to the ODS housing (requirement: P1).

G A 730836 P a g e 29 | 32

doesn’t

interfere/obstruct

the sensor field of

view and provides

protection of the

sensors during

testing from

damage and

environmental

effects. During the

train run it was

demonstrated that

the system does

not obstruct the

driver view. The

protective front

transparent part of

the housing did

not produce

reflection in

vision-based

sensors with direct

sunlight exposure

and in unfavorable

lightning

conditions during

test run.

The power supply

from the

locomotive cab

provides electric

power supply to

all the sensors.

UPS integrated in

the power

subsystem

provides reserve

during power loss

due to change of

power section

during the train

run. The system

successfully

detects the

obstacles on the

rail tracks, as well

as the rail tracks.

(requirements:

FR1, FR2, FR3,

FR7, FR8, INT1,

INT2, INT3,

G A 730836 P a g e 30 | 32

INT6, INT11).

TRL5 SMART

ODS demonstrator

dimensions are

smaller than

frontal profile of

the locomotive

series 444 used in

evaluation

(requirement:

INT4).

TRL5 SMART

ODS prototype

mass with ballast

and mounting

construction is

negligible in

comparison to

locomotive series

444 and doesn’t

influence axle load

distribution

(requirement:

INT5)

During the design

stage the housing

and mounting

construction were

subjected to

random vibration

load in simulation

according to the

EN 61373:2010 –

Rolling stock

equipment –

Shock and

vibration tests for

a Category 1 Class

B device so, they

are dimensioned to

resist the vibration

load (requirement:

INT7).

The installed

equipment is

compatible in

regard to electric

power supply and

electromagnetic

emissions and

immunity as it was

G A 730836 P a g e 31 | 32

not interfering

with other

locomotive system

or ODS system

was influenced in

any way by

electromagnetic

train emissions

(requirement:

INT11).

All TRL5 SMART

ODS prototype

components are

manufactured

from corrosion

resistant materials

(requirement:

INT9).

The TRL5

SMART ODS

prototype is not

connected to any

of the control &

command systems

of the vehicle

(requirement:

INT10).

7. Conclusion This deliverable reports on testing of the complete, integrated system against functional requirements, as defined in SMART deliverable D1.1. The presented functional tests were performed during the dynamic field tests in July 2018. During the dynamic field tests, the SMART integrated OD system was mounted onto the moving locomotive SERBIA CARGO 444-018 running without attached wagons on the 3.1 km route approved by a permit issued for test run by Infrastructure of Serbia Railways. Due to time and geographical limitations of dynamic field tests, it was not possible to test the OD sub-systems against all functional requirements defined in D1.1. Because of this, here described field tests should be considered as complemented by other field tests performed in different environment and light conditions during the SMART project lifetime. In this way, this deliverable D2.4 should be considered as complemented by other related SMART deliverables: D2.3, D3.1, D3.2 and D7.1, as marked in functional testing compatibility matrix given in Section 6.2. All complementary SMART field tests, including here described functional dynamic field tests

G A 730836 P a g e 32 | 32

performed in July 2018, demonstrated satisfactory OD system functionalities, the entire system could be released for the WP 7 evaluation activities.

8. References [1] T. I. S. E. GmbH, "The Imaging Source Europe GmbH," [Online]. Available: https://www.theimagingsource.com/products/zoom-cameras/gige-color/dfkz12gp031. [2] FLIR TAU2 camera[Online]. Available: http://www.flir.com/uploadedFiles/OEM/Products/LWIR-Cameras/Tau/FLIR-TAU-2-Datasheet.pdf. [3] Cousins S (2010) Welcome to ROS Topics, IEEE Robotics & Automation Magazine, Vol. 17, Issue 1.