Programme - asd-ssg.org€¦ · Web viewProgramme Integrating and Strengthening the European Research Strategic Objective Networked business and government Integrated Project / Programme

Programme

Integrating and Strengthening the European ResearchStrategic Objective

Networked business and governmentIntegrated Project / Programme Title

Advanced Technologies for Interoperability of Heterogeneous Enterprise Networks and their Application

Acronym

ATHENAProject No

507849ATHENA – Project Name

Piloting including Technology Testing Coordination and Pilot Infrastructure ATHEN A - Project No

B5

WORKING DOCUMENT

Use Case for test piloting

Work package – B5.5Leading Partner: CRF

Security Classification: Project Participants (PP)

May, 2005

Version 2.0

IP- Project ATHENA IP- Project - No 507849ATHENA - Project Piloting Including Technology Testing Coordination and Pilot Infrastructure ATHENA - Project Number B5 Document Working Document WD.B5.5 Date 10.05.23

Versioning and contribution historyVersion Description Comments

0.0 CRF Initial draft Preliminary version sent before Lisbon meeting December 13-15, 2004

0.1 Document Template and CRF Test Case – First draft -

0.2 CRF Test Case – Details on Test Scenarios 1 and 2 – First draft of Test Scenarios 3 and 4

-

0.3 SIEMENS Test Case. Suggestions for improvement of Test Scenarios description, assumptions, pre-conditions.

Jörg Müller contributions and feedback on wd framework, received and integrated January 18, 2005

0.4 AIDIMA Test Case. Òscar Garcia contributions of January 18, 2005. AIDIMA Test Case on e-procurement.

0.5 COMPUTAS suggestion on data sources and use case platforms. INTRACOM Test CaseAIDIMA Test Case update

Frank Lillehagen contributions of January 19, 2005. Maria Anastasiou contributions of January 24 2005. Òscar Garcia contributions of January 24, 2005. AIDIMA Test Case on e-procurement.

0.6 CRF contents analysis, Test Case improvement, metrics CRF revision of January 26, 2005.

0.7 CRF document revision TXT comments .CRF revision of February 3, 2005.

0.8 Siemens test case completed Jörg Müller contributions on SIEMENS Test Case, Introduction and Conclusion of February 15, 2005

0.9 CRF document revision CRF revision of February 21, 2005.

1.0 CRF final editing CRF final revision of February 22, 2005

2.0 Aerospace Use /Test Case for Pilot “Aerospace CPD based on Semantic Mediation”

EADS contributions on Test Case April 11, 2004

document.doc CONFIDENTIAL Page ii / 80


WD process scheduleNo Process step Responsible Timing Comments

1 Work document initial draft including structure and Template for Test Case CRF 03.12.2004

2 Official initial draft and first round of contributions.

SIEMENSAIDIMAINTRACOMEADSCRF

27.12.2004

3 Collection of final contributions

SIEMENSCRFAIDIMAINTRACOMEADS

10.01.2005

4 Consolidation of all contributions CRF 24.01.2005

5 Work Document approval CATS 31.01.2005

document.doc CONFIDENTIAL Page iii / 80


Table of contents

1 Summary....................................................................................................................................... 26

2 Introduction................................................................................................................................... 27

3 Templates and references...........................................................................................................283.1 Test Case Template................................................................................................................... 283.2 Metrics........................................................................................................................................ 28

4 Use /Test Case for Piloting..........................................................................................................294.1 Automotive Use /Test Case for Piloting : “Interoperability for Testing Management in Integrated Experimentation Plan” ( CRF )..............................................................................................................294.2 Automotive Use /Test Case for Piloting : “Interoperability for event-driven workflow management in Automotive Collaborative Product Development” ( SIEMENS )...................................404.3 Furniture Use /Test Case for Piloting: “Interoperability for optimisation of Furniture e-Procurement” (AIDIMA)......................................................................................................................... 544.4 Telecom Use /Test Case for Piloting (INTRACOM)....................................................................634.5 Aerospace Use /Test Case for Pilot “Aerospace CPD based on Semantic Mediation” (EADS CCR) 70

5 Conclusion.................................................................................................................................... 79

6 Glossary......................................................................................................................................... 80

document.doc CONFIDENTIAL Page iv / 80


List of FiguresFigure 1......Overview of the Product Development Process from an OEM perspective......................40Figure 2......Supplier side project development process overview.......................................................41Figure 3......Overview of the Sourcing process from an OEM perspective..........................................42Figure 4......Use cases for test scenario- Distributed search and subscription handling (TS_1)..........45Figure 5......Use cases for test scenario - Collaborative RfQ Processing (TS_2)................................46Figure 6......Dynamic handling of update events (TS_3).....................................................................47

document.doc CONFIDENTIAL Page v / 80


1 Summary“Use case for test piloting” aims to provide guidelines and support tools to prepare and carry out

the tests on the available Solutions.

Three main activities must be performed by the involved partners belonging to the four domains (Aerospace, Automotive, Furniture, Aerospace).

The first is the definition of use case of Business Scenarios and the statement of the Test Cases on which to build the Test Scenario. The Test Scenarios are the typical situations on which the Solutions will be tested.

The second is to develop the testing on the basis of Test Cases observing methodologies and metrics defined in “Methodology & Technology Testing Report”. The test activities will be carried out taking into account a predefined plan of experiments.

The third is the evaluation of test results and validation of the available Solutions. The evaluation and validation results will provide recommendations for further developments and will feed the activity of analysis on pilots’ results.

document.doc CONFIDENTIAL Page 26 / 80


2 Introduction

The present work document for which is foreseen a repeated planned edition (M12 – M24) contains in this issue a set of information useful to predispose the test environment.

It provides a set of templates for Test Cases and Test Scenarios formalization in order to illustrate the different descriptions in a uniform format, to yield the contents understandable for both internal and external partners. Besides, it contains the descriptions of the Test Cases of CRF, AIDIMA, EADS, SIEMENS, and INTRACOM.

The document (WD.B5.5 M12) is articulated in two main parts.

The first part (“3.Templates and references”) is dedicated to the description of documentation of support for Test Case definition and Validation.

The second part (“4.Use /Test Case for Piloting”) consists of the declaration of whole and specific test cases.



3 Templates and references

All information related to the individual test cases is compiled using a template. This chapter contains a set of guidelines to compile the proposed template.

3.1 Test Case TemplateTo gather the contents of the test cases a template is suggested and formalized in this document.

The template compounded of several parts, is reported in different sections dedicated to each partner (4.1 to 4.5).

A set of test case templates have to be compiled in relation to a certain number of considered test cases.

The suggested template includes the following sections: “1. General”, provide basic information about the test case, such as name for reference, issues

considered, and the solution/s that will be tested on these issues; “2. Introduction” where the test case representing a real work situation is described; “3. Glossary” to interpret the terms used in the test case; “4. Actors”, to describe the actors that are involved in the test case (human resources,

applications, systems, etc. involved in the real work situations); “5. Test/Use Case diagram” to better explain the contents, to represent the test case in a

graphical way; “6. Assumptions”, that are necessary to take into account for the test case to be valid; “7. Pre-conditions”, that are necessary to be satisfied for the test case to be applicable /

executable; “8.Test Scenarios”, (contained in this test case) with a short description for each test scenario.; “8.1. Test Scenario – TS1/2/3/n” where include details concerning the test scenarios involved; “9. Test procedures” with brief description for each test procedure used during the test scenario

execution; “9.1 Test procedure – TP1/2/n”. Detailed list of steps that must be followed to perform the test

scenario using the test procedure; “10. Metric”. A list of coded metrics that will be used to validate the test case, considering the

aspects regarding Validation methods identified in DB5.1 (“Technical Method” – T -; “Business Validation Method” B1 – B2);

“11. Matrix Test Procedure / Test Scenario” (e.g. TP1 and TP2 used for TS1 or TP1 and TP3 used for TS2, both belonging to test case 1)

“12. Matrix Test Scenario / Validation Methods” “13.Matrix or relationship issue / Test Scenario”3.2 Metrics

In the context of each test case, a set of metrics will be defined that will provide guidelines for validating each test procedure involved in test scenarios belonging to the test case.

Section “10. Metrics” of the template will be used to store this information, to formalize the metrics, and validation methods.



4 Use /Test Case for Piloting

4.1 Automotive Use /Test Case for Piloting : “Interoperability for Testing Management in Integrated Experimentation Plan” ( CRF )

1. GeneralAuthor Dario LeoRole Business Engineering Methodologies SpecialistDate 04/12/27 Scenario of reference CPDTest Case name CPD – Integrated Experimentation PlanTest Case number AUT_TC_1Issue 1.Produce automatic models translation

2.Produce models execution3.Produce automatic data translation and querying on demand4.Construction of an ontological system that will interact with disparate data sources and unify them by comprehending the semantic meaning underlying the data

Athena Solutions [Initial date for the Test]

Athena Solution Athena Solution delivery Date

Date of Test

MPCE [ / / ] [ / / ]EKAs [ / / ] [ / / ]MGWP [ / / ] [ / / ]

2. Introduction

The Case is integral part of the Automotive Scenario. It treats relevant aspects of interest tightly interconnected with the process of Collaborative Product Development.


Systems and applications

Test CaseArea

CATnet

WelcomHome

PSI LIVELIK


Test CaseArea

CATnet

WelcomHome

PSI LIVELIK

Process interestedby “Test Case 1”


Test CaseArea

CATnet

WelcomHome

PSI LIVELIK


Test CaseArea

CATnet

WelcomHome

PSI LIVELIK

Process interestedby “Test Case 1”


The core process regarding this case is the parallel process (respect to Target Setting, Sourcing and Design) of

“Testing” (Virtual and physical Testing), involving FIAT (OEM) and Suppliers (1st and 2nd Tier).

THE CHOICE

The reasons of this choice reside in the Target Setting process.

Starting from the Target Setting process:

1. OEM: has to define Technical Objectives (engineering specifications), therefore a set of Tests must be performed;

2. Supplier: the established laboratories must provide test results;

3. OEM: the Test Results are Technical Objectives that target Setting must reach.

THE PROBLEM

These three steps characterize the problem that can be represented in this way:

Inside the CPD there are different applications that manage vehicle testing information in different ways;

The data are stored in different databases using different formats;

Application integration is accomplished through the definition/management of point to



point translations that are computationally difficult and expensive

THE EXPECTATIONS

The expectations are:

Integrate these heterogeneous applications to obtain an environment on which is possible to connect and synchronize all these applications;

Exchange enterprise models in a multi-language environment;

Exchange enterprise models in a multi-application environment;

Execute enterprise models;

Integrate enterprise models on existing and execute them.

The Test Case that will be carried out concerns a specific situation characterized by the following features:

The model take into account two distinct actors: on abstract an applicant and an executor;

The information flow is characterized by levels that identify the information status (compiled, forwarded, executed, published);

The data are only management data (without particular calculations).

TEST CASE IN FOCUS:

The Prototype Plan Manager requests a Test Execution to Test Manager.The Test Manager prepares Test Packets and requests Test Execution to Lab Manager.In order to perform the Test, the Test executor, for each component/system, has to follow a

test procedure.The Test procedure is made by steps, with performances and target values.The test results are published by Test executor

In day-by-day operations, a Manufacturer would have to send test procedure/process to its supplier.

In this test scenario, we assume that this process is supported through the use of an Enterprise Modelling Tool, an Enterprise Project Management System and a Business Process Engine.

3. Glossary of Terms , Acronyms, abbreviations of Test CaseTerm Description

CatNet System

System

PSI SystemWelcomHome ApplicationLivelink SystemPrototype Plan Manager

Human resource

Test Manager Human resourceLab Manager Human resourceTest executor Human resource

4. Actors of Test Case (Roles, Systems, Resources)Actor Description



CatNet System

Testing management application being used by laboratories, testing managers, and prototypemanagers, and enables the creation of sessions to group test requests by laboratory andtracks testing documentation, results, and status throughout the individual testing cycles.

PSI System designed for testing management, it contains several methods for accessing vehicledevelopment testing and allows the creation of test packets, for submission to the CATNETsystem. Testing procedures are available to enable prototype managers to determine what sort of tests should be performed, and when, during the whole vehicle development cycle.

WelcomHome Project management application that extends Microsoft Project. Each macro project refers toa Motor Vehicle Make (Fiat Punto). Micro projects track the development of a specific MotorVehicle Model (Fiat Punto 1.3 8v Bi-Power). Project activities follow a fairly rigid prescribedsequence across the approximately 44 month timeline of a new car model.

Livelink Document management system which will be used to store test detail documents, test resultdocuments, and other types of documents shared by the future integrated Vehicle development system. Schema for communication with this application via XML messaging is employed for the purposes of COG.

Prototype Plan Manager

Manager and coordinator of the Experimentation Plan. He manages all tests on prototypes and decides the tests of systems, sub-systems and components on pre-defined prototypes.

Test Manager Manager and coordinator of a sub set of tests that must be performed by laboratories. He knows all interested laboratories and involves giving the Lab manager the responsibility of doing tests.

Lab Manager The Lab Manager coordinates the activity of Test executorsTest executor The Test Executor performs the Test on component and system. He follows pre-

defined procedures composed of several steps that driving him during the test activity.

5. Test/Use Case Diagrams

TEST CASE


OEM

Request Test

Model Test Procedure

Receive Results<<use>>

<<use>>

EM1

Process Catalogue Translate Model

MPCE

<<us

e>>

<<use>>

Supplier

EM2Receive Test

Procedure

Fill Test ProcedureAttributes

Send TestResult

<<use>><<use>>

<<use>>

<<use>>

<<use>>

<<use>>

<<use>>

OEM

Request Test

Model Test Procedure

Receive Results<<use>>

<<use>>

EM1

Process Catalogue Translate Model

MPCE

<<us

e>>

<<use>>

Supplier

EM2Receive Test

Procedure

Fill Test ProcedureAttributes

Send TestResult

<<use>><<use>>

<<use>>

<<use>>

<<use>>

<<use>>

<<use>>


Note: for Supplier we intend Test Laboratories situated at OEM home or Suppliers home

TEST SCENARIO 1 and 2

TEST SCENARIO 1


Prototype Manager(OEM)

Test Manager(OEM)

Lab Manager(Supplier)

Test Executor(Supplier)

Instantiation

Test ProceduresRepository

Testprocedure n°#

Testprocedureinitialised

Filling

Testrequest

initialisedForwarding

Test request Receiving Test Execution

ResultsForwarding

Testrequest

forwarded

Test Results

Test resultsforwarded

Testrequest

assigned

Result Validation and pubblication

StartTest

request

Resultsissue



Test Manager(OEM)

Test Manager(OEM)





Instantiation


Testprocedure n°#


Filling

Testrequest

initialisedForwarding


ResultsForwarding

Testrequest

forwarded

Test Results


Testrequest

assigned


StartTest

request

Resultsissue


Test Manager(OEM)


Instantiation


Test Proceduren°#


FillingTest

requestinitialised

Forwarding

Test request Receiving

Testrequest

forwarded

StartTest

request



Test Manager(OEM)

Test Manager(OEM)


Instantiation


Test Proceduren°#


FillingTest

requestinitialised

Forwarding

Test request Receiving

Testrequest

forwarded

StartTest

request


TEST SCENARIO 2


Test Manager(OEM)




ResultsForwarding

Testrequest

forwarded

Test Results


Testrequest

assigned


Resultsissue

Test Manager(OEM)

Test Manager(OEM)






ResultsForwarding

Testrequest

forwarded

Test Results


Testrequest

assigned


Resultsissue

TS1

TS2

Test Manager

Lab Manager

Test Executor

Test ProcedureInstantiation

Test ProcedureFilling

Test Request Forw arding

Test Results Receivingand Validation

Test Request Receiving

Test Assignement

Test Execution

Test ProcedureFilling w ith results

Test ResultsForw arding

AUT_TC_1

TS1

TS2

Test Manager

Lab Manager

Test Executor






Test Assignement

Test Execution



TS1

TS2

Test Manager

Lab Manager

Test Executor






Test Assignement

Test Execution



AUT_TC_1

TS1

TS2

Test Manager

Lab Manager

Test Executor






Test Assignement

Test Execution




6. Assumptionsn. Description1 A component/system has 6 test procedures.2 There are three test laboratories (Lab1 , Lab2, Lab3).3 Each lab tests exactly two test procedure.4 For demo purposes are considered two different Enterprise Modelling tools.5 A test procedure contains multiple steps, where each step relates to a specific

performance where a value is required.6 Each performance has an unit of measurement7 To maximize interoperability testing, a different unit measurement should be used in

at least one lab.8 The suggestion is : at least one Test lab use an English/American unit measurement.

7. Pre-conditionsn. Description1 The Laboratories can be internal (FIAT and Suppliers or only FIAT) or external (FIAT

at Supplier’s home or only Supplier).2 The Manufacturer would have to send test procedure/process to its supplier.

In these test scenarios, we assume that this process is supported through the use of an Enterprise Modelling Tool.

8. Test Scenarios n. Description

TS1 Select and Transmit test procedures to a Test LabTS2 Select and Transmit test procedures executed to a ManufacturerTS3 Execute the enterprise model on a BP EngineTS4 Automatic models translation towards Enterprise Project Management system

8.1. Test Scenario – TS1 : Select and Transmit test procedures to a Test LabItem DescriptionGoal A Test Manager goes to the Test website with the intent of request and

transmit three tests to a test Lab for its executionPre-conditions 1.Test Procedure Catalogue Exists

2.Test Procedure contains predefined target values3.Service Test details for each Lab are known



Test Creator MIData Creator 04/12/27Data Test 05/Test executor DLInternal Test Assistant GSExternal Test assistant PCSuccess Post Conditions 1. All test procedure steps are recognized by the Labs EM tool

2. All predefined performance attributes are recognized by the Labs EM tool3. All different units of measurement for attribute are converted

Failed Post Conditions -Results -Comments -8.2 Test Scenario – TS2: Select and Transmit test procedures executed to a ManufacturerItem DescriptionGoal A Test Executor goes to the Test website with the intent of request and

transmit three executed tests to a ManufacturerPre-conditions 1. Test Procedure Model Catalogue Exists

2. Test Procedure Model contains requested performance valuesTest Creator MIData Creator 04/12/27Data Test 05/Test executor DLInternal Test Assistant GSExternal Test assistant PCSuccess Post Conditions 1. All test procedure steps are recognized by the Manufacturer tool

2. All calculated performance attributes are recognized by the Manufacturer EM tool3. All different units of measurement are converted back

Failed Post ConditionsResultsComments8.3 Test Scenario – TS3: Execute the enterprise model on a BP Engine Item DescriptionGoal A Test Executor goes to the Test website with the intent of start a test

procedurePre-conditions 1. Test Procedure Model Catalogue Exists

2. Test Procedure Model contains requested performance valuesTest Creator MIData Creator 05/01/11Data Test 05/Test executor DLInternal Test Assistant GSExternal Test assistant PCSuccess Post Conditions 1. All test procedure steps are recognized by the BP Engine tool

2. All steps are executed and the right people are notified by the BP Engine or the right e-service are properly called 3. The enterprise model is correctly integrated with the already existing and running processes

Failed Post ConditionsResultsComments8.4 Test Scenario – TS4: Automatic models translation towards Enterprise Project Management systemItem DescriptionGoal A Project Manager goes to the process website with the intent of

performing a Project simulation (resource allocation and costs) based on



an existing model template.Pre-conditions 1. Process Model Catalogue Exists

2. Process Model contains all roles, KPI, duration needed for a taskTest Creator MIData Creator 05/01/11Data Test 05/Test executor DLInternal Test Assistant GSExternal Test assistant PCSuccess Post Conditions 1. All process tasks are recognized by the EPM system

2. All process tasks are assigned to the right people as well as the other resources3. All process duration are assigned to the right tasks

Failed Post ConditionsResultsComments

9. Test Proceduresn. Description

TP1 Sequence for– TS1: Select and Transmit test procedures to a Test LabTP2 Sequence for – TS2: Select and Transmit test procedures executed to a ManufacturerTP3 Sequence for –TS3: Execute the enterprise model on a BP EngineTP4 Sequence for – TS4: Automatic models translation towards Enterprise Project

Management system

9.1 Test ProcedureTP1 Description

Step1 Test Manager accedes to CatNETStep2 Test Manager examines Tests Catalogue (Test Procedures are modelled with EM1

tool)Step3 Test Manager distributes the Tests to Lab Manager (Workflow in progress. Sent the

notification)Step4 Lab Manager receives Test requests and Test procedures (The procedures are

recognized by EM2 tool)Step5 Lab Manager verifies the consistency of data and unit of measurement


Step1 Test executor accedes to CatNetStep2 (Executes the Tests) input Test results in CatNetStep3 The EM2 tool models the data Step4 Test executor sends dataStep5 The MPCE transforms EM2 data to EM1 data


Step1 TbdStep2Step3Step4Step5


Step1 TbdStep2Step3Step4Step5



10. MetricMetric n. Validation

Type (Method)

Description

M1

Test Procedure steps recognised : 80 % (after 10 test replications : at least 8 test must be completed)

B1 Predefined Performance attributes recognised : 80 % (after 10 test replications : at least 8 test must provide 100 % of recognised attributes)Calculated performance attributes recognised : 80 % (after 10 test replications : at least 8 test successfully completed must provide 100 % of recognised attributes)

M2

Unit of measurement : 100% of conversionProcess tasks recognized by the EPM system : 100%

B1 Process tasks assigned to the right people : 100%Process duration assigned to the right tasks :100 %Enterprise model correctly integrated with the already existing and running processes : 100%Steps executed and right people notified by the BP Engine or right e-service properly called : 100%

M3

M4

M5

M6

M7

M8



M9

11. Matrix Test Procedures – Test ScenarioTEST SCENARIO 1

TEST PROCEDURE

METRIC 1 METRIC 2 METRIC 3 METRIC 4 METRIC 5

TP1 x xTP2TP3TP4TP5TP6TP7TP8TP9

TP10TEST SCENARIO 2

TEST PROCEDURE


TP1TP2 x xTP3TP4TP5TP6TP7TP8TP9

TP10TEST SCENARIO 3

TEST PROCEDURE


TP1TP2TP3 x xTP4TP5TP6TP7TP8TP9

TP10TEST SCENARIO 4

TEST PROCEDURE


TP1TP2TP3TP4 xTP5TP6TP7



TP8TP9

TP10

12. Matrix Test Scenario – Validation Methods

Test Scenario T B1 B2TS1 xTS2 xTS3 xTS4 xTS5TS6TS7TS8TS9

TS10

13. MATRIX OF RELATIONSHIP – ISSUE TEST SCENARIO

I1 I2 I3 I4 I5 I6 I7 I8 I9TS1 X XTS2 X XTS3 X XTS4 X XTS5TS6TS7TS8TS9TS10

4.2 Automotive Use /Test Case for Piloting : “Interoperability for event-driven workflow management in Automotive Collaborative Product Development” ( SIEMENS )

1. GeneralAuthor Jörg MüllerRole Automotive SupplierDate 05/02/16Scenario of reference CPDTest Case name Event-driven workflow management in automotive CPDTest Case number AUT_TC_02Issue Event-driven distributed business process interoperability

Proactive workflow management and automationAdaptive robust and resilient ICT solution for distributed

business systems Efficient and secure management of decentralized

information / business resourcesAthena Solutions [Initial date for the Test]


Date of Test

P2P Resource Management; Intelligent Workflow Automation

[ / / ] [ / / ]

[ / / ] [ / / ][ / / ] [ / / ]



2. Introduction

The test case described in this section is an integral part of collaborative product development. It is located in the early phase collaborative product development, during which an OEM, its (first-tier) suppliers, and selected suppliers of the first-tier suppliers collaborate in order to verify (and where necessary to amend) a request for quotation (RfQ). The RfQ contains business data (derived from the target setting process) as well as a technical specification of the automotive subsystem (e.g. safety electronics subsystem) which is subject of the collaboration. The strategic sourcing process from the perspective of the OEM is depicted in Figure 1.

Figure 1. Overview of the Product Development Process from an OEM perspective

The corresponding supplier side process view is shown in Figure 2. As the red circle in Figure 2 indicates, this test case focuses on Phase 1 (acquisition and quotation) and Phase 2 (concept), which are drawn sequentially in Figure 2, but which in reality are strongly overlapping and which are performed in close collaboration with the OEM and with 2nd tier suppliers.



ProjectManagement ManufacturingDivision

Change Control / Change Management / Claim Management

Acquisition Development Production

ProductionMaintenance

Idea

Phase 0OpportunitySelection

M0 •Business Case / Opportunity Evaluation•Planning for Acquisition•Acquisition Leader Named•Review Minutes

Phase 1Acquisition& Quotation

M1 •Project Authorization•Risk Assessment•SV Offer Is Made•Quotation Package•Initial Project Plan•Initial BOM Distributed•Initial Concept•Intellectual Property•Business Case•Project Manager Nominated•Documents for Decision Base•Review Minutes

Phase 2Concept

M2

•Project Agreement•Customer Commitment•Detailed Project Plan•Kick Off Meeting Minutes•Project Team Established•Review Minutes

Phase 3Sample Verification &

Design Validation

M3•Production Capacity Plan•Approved DV•Production Ramp Up Plan•Agreement with Plant•Pre-Launch Control Plan•Finalized Supplier Plan•Packaging Plan•Preliminary Capability Studies•Product and Process Documentation Released into Production System•Review Minutes

Phase 4Introduction

to Production &Validation

M4•Production Release•Production Control Plan•Validated Production Parts (int/ext) PV Report•Validated Supplier Components•Approved Tools and Equipment•Completed Measurement System Analysis•Updated Preliminary Capability Studies•Review Minutes

Phase 5ProductionRamp Up

M5

•Plant’s Formal Acceptance of Transition Signed by Project Team and the Plant•Completed Project Final Report and Budget Closed•Project Closed and Project Team Dissolved•Documentation and Experience Transferred to Plant•Capability Studies Completed•Monthly Metrics at M 5 Until 6 Months (max)•Non Conformance Cost Report•Review Minutes

M2a •Final Concept Defined•BOM Defined and Distributed•Main Suppliers Determined•Initial Supplier Plan•Pre-Design Freeze for Long Lead Time Parts•Review Minutes

M3c

•Design Committed and Frozen

•Process Committed and Frozen (Equipment Spec. Packages Complete)•Supplier Contracts •Training Plan•Customer Commitment for Tooling Purchase•Review Minutes

–BOM Released–Product Design Package Released

M3a.x •Sample Loops Plan•Sample X Quality Plan•Sample X Definition Released•Supplier Sample Manufacturing Process•Sample X Manufacturing Plan•Sample Tooling Released•Review Minutes

M3b.x •Sample X Quality Report•Sample X Built Analysis Report•Sample X Test Report•Sample X Delivery•Review Minutes

0

OpportunityApproved

1

ProjectAuthorization

2

Project Agreement

3

Product / ProcessDesign Release

4

ProductionPart Approved

5

ProjectClosure

Relea

se fo

r Buil

t

3a.x

Conc

ept C

ommi

tmen

t

2a 3b.x 3c

Deliv

ery

Produ

ct D e

sign F

re eze

Receiving Plant

Figure 2. Supplier side project development process overview

In the automotive sector, major portions of RfQ related input specifications are driven/owned by the OEM. A major problem and obstacle to interoperability between an OEM, its first-tier suppliers, and the suppliers’ suppliers are modifications made to the specifications after the specifications were published together with the RfQ.

There are different reasons for these modifications:

Availability of new business-level information (e.g., sales forecast data) that change business parameters (e.g. sales target, profit target, volume targets) and that may affect the technical specifications (e.g. less budget for a certain subsystem may enforce a less-quality version; a need for increased competitiveness may enforce an additional feature to be included in the specification)

Inconsistencies and technical problems observed as the RfQ is discussed between the OEM and its suppliers may force changes in the specification. These problems can be observed by any partner in the CPD process.

In some cases, the availability of new technology (e.g. distance keeping assistant) may lead to changes being made.

Sometimes, business relationships change during the process, and the specification needs to be adapted to the capabilities of new suppliers. (In automotive, this happens rarely due to the strong position of the OEMs).

Thus, the RfQ handling workflow is less process-centred than event-centred. A basic process description (see Figure 3 for the OEM perspective of this) serves as the guiding environment, but within this process there is a considerable amount of dynamic action and reaction by the partners.In the current automotive practice, this process is executed in a fairly ad-hoc way with only limited IT support. This leads to basic interoperability problems, leading to delays and planning errors and inconsistencies that are only discovered at a much later stage of the CPD process, leading to costly and time-consuming iterations. For instance, events such as change requests initiated from a suppliers’ supplier may be lost during the process; modifications made by the OEM may not be communicated to a suppliers, and partners may hence do their planning and offer preparations based



on outdated versions.

Figure 3. Overview of the Sourcing process from an OEM perspectiveA seemingly easy solution would be a central document management system that is used by the OEM, its suppliers, and the suppliers’ supplier to synchronize specifications, to report and manage specification change requests, and ultimately to publish and discuss offers. This system could receive, produce and propagate event notifications to the partners. However, there are some problems with this solution:

Firstly, there is a strong competition between first tier suppliers in the strategic sourcing process. The thought of sharing a platform with other suppliers that contains sensitive data (e.g. about the product innovations, but also business data), does not appeal to suppliers. An at least partially decentralized solution could reduce the severity of this problem.

Secondly, the CPD scenario described in this test case is a cross-sector scenario; while the OEM-supplier relationship is a pure automotive scenario in which a high degree of standardization can be achievable due to the market power of the OEM, the supplier-supplier relationship is much less asymmetric: e.g. a screw manufacturer will have customers from the automotive sector, but also many other customers from other sectors; hence, it is much less willing to spend resources for joining a central system than the OEM’s first-tier supplier may be. Interestingly, from the perspective of the first-tier supplier, the essence of this Test Case is how to smoothen its processes, shorten time-to-market, and perform cost savings by linking its OEM-side customer relationship management processes with its supplier-side supplier relationship management processes. Due to the heterogeneity of this supply network, a decentral solution which helps preserving partners’ autonomy and which can be adapted to the respective sector market structures, seems appropriate.

Finally, from a mere ICT perspective, central solutions carry the intrinsic risk of becoming performance bottlenecks; large hardware and software investments are required to provide a high degree of resilience, robustness, and availability. A more decentralized approach has the potential of satisfying these non-functional requirements in a cost-saving way. Therefore, the test case will validate a decentral event-driven workflow management solution based on an ATHENA R&D result: the Distributed Resource Management Framework (BRMF). The objective is:

o To provide improved interoperability while preserving heterogeneity at the level of business objects / documents without necessarily requiring central data repositories or registry structures.

o To provide the ability to dynamically discover services / suppliers without requiring central structures



o To preserve autonomy and to be adaptable to different sectorial market structures and market regimes, hence providing a smooth flow of information and process control across the boundaries of sectors (from a supplier view, as has been said above, this translates to supporting the link between CRM (OEM-side) and SRM (2nd-tier supplier side).

The essential test case is centred around the lifecycle of an RfQ document and focuses on the management of process-critical events during the processing of the RfQ and allocated discussion and – if necessary – modification of the technical specification attached to the RfQ.

THE CHOICE

The reasons for this choice reside in the dynamics of the strategic sourcing process and in the potential for optimisation and interoperability improvements, which make it a suitable case for applying ATHENA technology:

1. Peer-to-peer decentralized resource management for event-driven workflow, loosely coupled cross-market collaboration, and maintenance of partner’s autonomy

2. Model-driven development for reduced IT maintenance and a potentially higher degree of automation.

THE PROBLEM

The problem is how cross-sectorial interaction between OEM, 1st tier supplier, and Nth-tier supplier in the sourcing phase can be supported to reduce time-to-market, to save cost and increase the quality and accuracy of the final specifications and the offers made by the supplier(s). This problem is complicated by the following constraints:

The easy way, i.e. a centralized solution is not feasible, at least in some cases There are no direct business relationship between the OEM and the suppliers’ suppliers;

however, events initiated by some partners may affect other partners, even if these do not know each other.

There are different heterogeneous applications (e.g. CRM, document management, SRM) involved that manage the process in different ways.

New partners may enter the process (e.g. new suppliers) dynamically.

THE EXPECTATIONS

The expectations from the participating actors (in particular here focusing on the supplier side) are: Provide a way to manage business objects in an interoperable cross-enterprise way Maintain security and data protection, in particular across multiple competing suppliers Enable the goal-directed propagation of changes across the collaborative network to minimize

time delay and cost caused by coordination, versioning, and synchronization problems Improve the interleaving of the suppliers’ CRM and SRM processes. Provide robustness and resilience of the underlying ICT infrastructure at affordable cost Integrate new processes seamlessly with existing applications.

TEST CASE IN FOCUS1. The supplier’s Sales Project Manager can search for / subscribe for new RfQs in the area of car

safety electronics.2. The supplier’s Sales Project Manager receives a new RfQ (as result of a search or via notification)

and distributes it to business administration and engineering3. The supplier’s Technical Project Manager selects a critical 2nd tier supplier and initiates a

collaboration with the supplier, making a selected part of the RfQ visible to the 2nd tier supplier.4. Asynchronous event 1: A change request is made by the OEM while the RfQ is being processed;

all relevant parties are informed and re-adjust their work5. Asynchronous event 2: An inconsistency is detected by the supplier’s engineering department, a

change in the technical specification is proposed and communicated to all relevant parties. The OEM accepts the change, all parties are informed and the work is re-adjusted.

6. Asynchronous event 3: The document server at the OEM side, which logically hosts the master version of the technical specification, is temporarily unavailable due to a system crash. Using P2P replication, partners are still able to get access to the most recent version of the RfQ / Technical



Specification information [NOTE: This test case step will be postponed until a later version of the working document].

7. The supplier sends an offer to the OEM, based on the consolidated version of the technical specification

In the current day-by-day solution (as-is), RfQ/technical spec information is exchanged between OEM and suppliers using word documents. In our test case (to-be), this process is supported by a model-based representation (which may still contain semi-structured elements such as MS word documents, but in addition contains metadata with a clear and agreed upon semantics). In addition, parties use proprietary software systems. We shall assume the existence of some type of web-service based wrappers that allow partners a well-defined access of some parts of the required functionality of these systems at the ICT level.

3. Glossary of Terms, Acronyms, abbreviations of Test CaseTerm Description

DE Development Engineer (human resource)SPM Sales Project Manager (human resource)SRM Supplier Relationship Manager (human resource)TPM Technical Project Manager (human resource)BDMS Business document management system (system / application)BRMF Business resource management framework (system / middleware)CRMS Customer relationship management system (system / application)DPMS Development project management system (system / application)SRMS Supplier relationship management system (system / application)


DEVELOPMENT ENGINEER (DE)

Responsible for checking technical feasibility of a certain aspect of the RfQ and for preparing the corresponding part of the offer / response at the 1st tier supplier side. The DE reports to the TPM.

SALES PROJECT MANAGER (SPM)

Responsible for OEM-side Processing of RfQ at 1st-tier supplier side; primary contact to OEM purchasing unit.

SUPPLIER RELATIONSHIP MANAGER (SRM)

Responsible for 2nd-Tier Supplier side communication and strategic contracting issues at 1st tier supplier side. Responsible for overall co-ordination with 2nd-tier supplier during RfQ processing and offer/response preparation.

TECHNICAL PROJECT MANAGER (TPM)

Responsible for technical coordination of RfQ processing and technical adequacy of response to RfQ at 1st tier supplier side. Validation of technical feasibility; preparation of technical section of offer / response to RfQ.

BDMS System Business document management system built on top of the BRMF system, developed in ATHENA; maintains links to DPMS, CRMS, and SRMS.

BRMF System Decentral Business Resource Management Framework, developed in ATHENA; basic technology platform facilitating the event-driven workflow solution.

CRMS System Customer relationship management system (supplier proprietary) used for offer / RfQ response preparation

DPMS System Development project management system: a supplier proprietary solution used to store CPD project planning and management information (customer, organisation chart, members, resources, project plan, milestones, reports, status information)

SRMS System Supplier relationship management system (supplier proprietary) contains information about skills, capabilities, history, offerings, pricing, and capacities of 2nd

tier suppliers as well as ongoing supplier side processes.




The test case described in this section consists of three different test scenarios, each of which is described by a set of use cases:

Test scenario TS_1: Distributed search and subscription handling (see Figure 4) Test scenario TS_2: Collaborative RfQ processing (see Figure 5) Test scenario TS_3: Dynamic handling of update events (see Figure 6)

In the following, the use case diagrams for each test scenario are illustrated and briefly described.

1. TS_1: Distributed search and subscription handling

This test scenario is described by the following features / steps:

SPM can search using BDMS for suitable RfQs SPM can register for RfQs satisfying certain criteria SPM is notified if a RfQ satisfying the subscription is published SPM can view RfQs

The use cases corresponding to this test scenario are shown in the use case diagram in Figure 4.

Figure 4. Use cases for test scenario- Distributed search and subscription handling (TS_1)

2. TS_2: Collaborative RfQ Processing

This test scenario is described by the following features / steps: Starts on notification of RfQ receipt SPM sets visibility for RfQ, involving the TPM, business administration and other

parties (on the TPM side this involves use case “Receive RfQ information” described in TS_1. This is not depicted in Figure 5).



TPM assigns RfQ processing to DEs DEs evaluate RfQ for selected components in collaboration with second tier suppliers.

These suppliers are retrieved using the SRM; a collaboration is established The offer (response to RfQ) is prepared and submitted to the OEM


Figure 5. Use cases for test scenario - Collaborative RfQ Processing (TS_2)

3. TS_3: Dynamic handling of update events:

This test scenario is described by the following features / steps: DE recognizes inconsistency After exchanging information with its suppliers (if necessary) DE proposes change request Change request is approved by TPM and SPM


SPM

TPM

SRM

DE

Set up RfQdistribution

Find 2nd tiersupplier for component

Collaborate with2nd t ier supplier

«uses»

Extractsupplier-specif ic component requirements

Prepare RfQResponse

«uses»

«uses»

Using CRMS

BRMF/BDMS

BRMF/BDMS

Using SRMS

Using SRMS


Change request is submitted to OEM OEM commits RfQ update as consequence of change request OR denies change request Alternatively, a change request may be published by the OEM for reasons beyond the

influence of the 1st tier supplier. Information about the change request is routed to all relevant parties, in particular SPM,

TPM, and corresponding partners at 2nd tier supplier side.


Figure 6. Dynamic handling of update events (TS_3)

6. Assumptionsn. Description1 OEM, 1st tier supplier and 2nd tier suppliers may use different formats or models for

RfQ and technical specifications. The necessary transformations are beyond the scope of this test case; they are defined in a different test case (see e.g. CRF_TS_1)

2 Note: Our solution assumes / allows for the presence of corresponding document/project management systems at OEM and 2nd tier suppliers. However,



these systems lie beyond the control of the 1st tier supplier and are made available in a virtualised form using the BRMF solution. As such, they are not modelled in the test case, in particular, for the 2nd tier suppliers; where necessary, they will be simulated.

3 The preparation of a response to RfQ by a supplier involves both a technical and a business part. In this use case, we focus on the technical part, i.e. on the review and discussion of the technical feasibility and solution for the offer. The business part is neglected at this stage,

4 We do not assume the availability of a central server that contains all RfQ related information and that is physically accessed by all participants, nor do we assume a globally accessible central model or business object repository or registry

5 We assume an appropriate domain ontology for all business entities; where different ontologies / formats are used by the different users, we assume the existence of an appropriate transformation / mapping. This mapping is beyond the scope of the test case.

6 For test-scenario specific assumptions regarding TS_1 (Distributed search and subscription handling), see Section 8.1 Pre-conditions / assumptions

7 For test-scenario specific assumptions regarding TS_2 (Collaborative RfQ processing), see Section 8.2 Pre-conditions / assumptions

8 For test-scenario specific assumptions regarding TS_3 (Dynamic handling of update events), see Section 8.3 Pre-conditions / assumptions

7. Pre-conditionsn. Description1 Interconnection of OEM and 1st tier suppliers regarding RfQ publication and

processing; if no coupling of live systems is feasible, this can be simulated in offline mode using real data.

2 Interconnection of 1st tier and 2nd tier suppliers. Since the ATHENA consortium does not contain a suitable 2nd tier supplier, it may turn out to be necessary to simulate the systems and behaviour of that role during test case execution.

3 We assume secure electronic connectivity within the collaborative network, e.g. via a VPN network, and the availability of the required applications and services via secure web-service based interfaces.

4 For test-scenario specific preconditions regarding TS_1 (Distributed search and subscription handling), see Section 8.1 Pre-conditions / assumptions

5 For test-scenario specific preconditions regarding TS_2 (Collaborative RfQ processing), see Section 8.2 Pre-conditions / assumptions

6 For test-scenario specific preconditions regarding TS_3 (Dynamic handling of update events), see Section 8.3 Pre-conditions / assumptions


TS1 Distributed search and subscription handlingTS2 Collaborative RfQ ProcessingTS3 Dynamic handling of update events

8.1. Test Scenario – TS1: Distributed search and subscription handlingItem DescriptionGoal Supplier-side business users (in particular SPM and TPM) can find and

inspect new RfQs of interest to the supplier online. The SPM can set up a persistent search request (so-called subscription for notification) using appropriate search criteria.

Pre-conditions / assumptions

1. All participating users in the test scenario have access to a software client for or a web-based gateway to the BDMS.

2. A base ontology for the RfQ exists which is shared by the users, or users use different ontological representations, and a transformation is made. The performance of this transformation is beyond the scope of this test scenario.

3. BDMS has access to all physical business object / document / service



repositories required (subject to access policies and user permissions).

4. See general pre-conditions and assumptions, Sections 6 and 7.Test Creator JPMData Creator 05/01/13Data Test 05/Test executor VöInternal Test Assistant TFExternal Test assistant NNSuccess Post Conditions 1. SPM finds one valid RfQ or multiple valid RfQs of interest.Failed Post Conditions 1. No RfQs found, because wrong / poor search criteria used.

2. No RfQs found, because none exist.Results See Post Conditions.Comments --8.2 Test Scenario – TS2: Collaborative RfQ ProcessingItem DescriptionGoal Supplier-side business users are enabled to prepare a response to an

RfQ (offer) in collaboration with the OEM and selected 2nd-tier suppliers. For this purpose, the management users (in particular SPM) can set access policies to the RfQ and segments of it. These policies will determine visibility and flow of information during RfQ processing.


1. The SPM has received a valid RfQ from an OEM by performing Test Scenario TS_1.

2. A suitable tool for visualization, discussion, and annotation of the RfQ and the technical specification between the suppliers development engineers and the 2nd tier suppliers is available. The description and operation of this tool itself is beyond the scope of the test scenario.

3. Visibility policies are set to deny access to the RfQ for a specific 2nd-tier supplier or Development Engineer (DE)

4. See general pre-conditions and assumptions, Sections 6 and 7.Test Creator JPMData Creator 05/01/13Data Test 05/Test executor VöInternal Test Assistant TFExternal Test assistant NNSuccess Post Conditions 1. The OEM has received a valid response to the RfQ (offer) from the

supplier; receipt of the offer is documented in the OEMs supplier management system; the technical specification is free of major inconsistencies and errors.

2. Alternatively, the supplier has decided not to submit an offer and has informed the OEM to this effect.

3. Attempts to access the RfQ’s SSTS made by suppliers for which access is not provided by the access policy are unsuccessful

Failed Post Conditions 1. The supplier fails to submit the response in time.2. Alternatively, the supplier may have submitted the offer in time, but

the offer may violate certain requirements in the RfQ due to synchronization problems, or the RfQ / Technical specification may contain unrecognised inconsistencies or errors.

3. A registered party for which access to the RfQs SSTS is not granted manages to access the SSTS via the system.

Results See Post Conditions.Comments This test scenario is subsequent to Test Scenario TS_1.

This Test scenario contains test scenario TS_3 as a sub-scenario.8.3 Test Scenario – TS3: Dynamic handling of update eventsItem DescriptionGoal Enables business users at supplier side to quickly react to incoming

changes to the RfQ and technical specification, and to quickly and reliably



create and propagate change requests to partners in the collaborative network, with the goal to minimize overall RfQ processing time.


1. A collaborative RfQ processing scenario as described in test scenario TS_2 (Section 8.2) is active.

2. In particular, access policies and visibilities were set properly in TS_2.3. An interface to the CRMS System is provided to access and store role

information required for the approval process, as well as the workflow history (i.e. the fact that change requests were received / created / approved). This information can be stored temporarily within the DBMS system, but will be synchronized with CRMS for archiving purposes.

4. See general pre-conditions and assumptions, Sections 6 and 7.Test Creator JPMData Creator 05/01/13Data Test 05/Test executor VöInternal Test Assistant TFExternal Test assistant NNSuccess Post Conditions 1. Variant 1: A change request identified by the supplier has been

transmitted to the OEM in time (i.e. before the offer submission deadline), has been agreed and committed.

2. Variant 1: A change request identified by the supplier has been transmitted to the OEM, has been discussed and rejected by the OEM.

3. Variant 2: A change request committed by the OEM has been received and propagated to all relevant parties in time to be processed / implemented before the offer submission deadline.

Failed Post Conditions 1. Networking or server problems prevent a timely delivery of update event notifications.

Results See Post Conditions.Comments This Test Scenario is part of Test Scenario TS_2.


TP1 Sequence for– TS1: Distributed search and subscription managementTP2 Sequence for – TS2: Collaborative RfQ processingTP3 Sequence for – TS3: Dynamic handling of update events


Step1 SPM logs into BDMS client application.Step2 SPM enters content-based search criteria for suitable RfQs.Step3 SPM scans search results and inspects individual results.Step4 If no suitable results are found, the SPM saves its query by creating a subscription.Step5 At a later point in time, when logging into the system, the SPM can see that new RfQs

have been published that match one of its subscriptions.Step6 The SPM accesses and views the RfQ.Step7 The SPM can make annotations to the RfQ model (annotation feature outside the

scope of this test scenario; supposedly provided by other partners).9.2 Test Procedure

TP2 DescriptionStep1 SPM identifies responsible TPM and other parties (e.g. Business Administration

Manager).Step2 SPM uses BDMS to set access policy / visibility for RfQ, involving the TPM, business

administration and other parties.Step3 TPM can now access the RfQ (analogous to test scenario TS_1) using the BDMS.Step4 TPM identifies responsible DEs (different DEs are responsible for different sub-



components to be addressed in the offer related to the RfQ.Step5 TPM assigns RfQ processing to DEs.Step6 DEs can now access the RfQ (analogous to test scenario TS_1) using the BDMS. It is

possible to restrict the visibility of the RfQ to DEs at the section or sub-component level. While this is rarely done if DEs are in-house, it may be necessary if e.g., external consultants are used.

Step7 DEs evaluate RfQ for selected sub-components.Step8 DEs discover/select potential 2nd tier suppliers for selected component. These

suppliers are retrieved using the SRMS.Step9 A collaborative process is established during which requirements and possible

inconsistencies of the RfQ and technical specification are discussed / identified using a suitable collaboration channel / tool.

Step 9a A second tier supplier who has not been authorized to participate in the collaborative process attempts to access the Subsystem Technical Specification

Step10 If necessary, change requests are initiated and propagated (see TS_3).Step 11 The offer (technical response to RfQ) is prepared by the SPM and TPM and

submitted to the OEM. Alternatively, the SPM and TPM may decide not to submit an offer, in which case the OEM is informed.


Step1 DE recognizes technical inconsistency in the Technical Specification or RfQ.Step2 After exchanging information with its suppliers (if necessary), the DE proposes

change request.Step3 Change request is approved by TPM and SPM.Step4 Change request is submitted to OEM.Step5 OEM commits RfQ update as consequence of change request OR denies change

request.Step6 [Variant 2:] Alternatively to 1.-5., a change request may be published by the OEM for

reasons beyond the influence of the 1st tier supplier. This variant of the test case starts at Step 6.

Step7 Information about the change request is routed to all relevant parties, in particular SPM, TPM, and corresponding partners at 2nd tier supplier side. The definition of “relevant parties” is according to steps 2, 5, and 8 of test scenario TS_2.




TypeDescription

General approach:For this test case, we shall focus on five aspects of interoperability, each of which will be measured by a set of metrics. These aspects are:- Quality of search (see M1)- Communication efficacy (see M2)- Resilience and robustness (see M3)- Security and data protection (see M4)- Usability (see M5)

NOTE: Where reference is made to average values, this generally refers to the average from a total of ten iterations. We are aware that this number is too low to fully ensure statistical significance of the results. However, the fact that human users are involved prevents us from carrying out a higher number of iterations. We do believe that the validation procedure still provides a fair indication of the qualitative and quantitative performance of the system in the test cases. For the quantitative measures, additional simulation experiments will be carried out with the aim to provide a satisfactory level of statistical significance.

M1

Search quality will be measured with reference to a manually compiled reference set (“caucus”) using the IR measurements of search precision (SP), search recall (SR), and search accuracy (SA);1. Search recall SR (= #relevant RfQs returned by the search / #total relevant RfQs) (target: > 0.6)

T 2. Search precision SP (=#relevant RfQs returned by the search / #total RfQs returned by the search) (target: > 0.6)3. Overall search accuracy = (SP + SR)/2; target: 0.7 4. Average time for online search to return results; target: 10 seconds5. Average time for subscriptions to yield results after they were created or changed; target: 1 minute

M2

Communication efficacy will be measured using the speed and reliability of information dissemination through the partner network.1. Average time for an event notification to be propagated through the network of partners; 1 minute (assuming ten test iterations)2. Average time for a partner / resource to be visible in the network after being published less than five minutes (assuming ten test iterations)

T 3. Share of event notifications not delivered after ten minutes; target: : 0.054. Share of partners / resources not visible ten minutes after registration: : 0.05

M3

Resilience and robustness will be measured in terms of successful overall execution of the test procedures, as well as based on the ability of the system to keep operational in the face of (simulated) failures of selected partner IT systems / nodes. 1. Overall successful completion rate for test scenarios; target: 0.8 (given ten iterations)

B1 2. Overall successful completion rate for test scenarios given that at most one partner is not available for max. 20% of test case duration; target: 0.8 (given ten iterations)3. Average time for an event notification to be propagated through the network given an availability probability of 0.9 and an average downtime of 1 minute; target: 2 minutes (see metrics M2.1 for 100% availability)



M4

Security and data protection will be measured based on the observed probability of failure for attempts to intrude into the system, i.e. access information that is not visible according to the access policies1. Average share of denied unauthorized accesses to BRMF resources; target: 100%

B1

M5

Usability will be measured based on the participants judgement on the test case system. The following aspects will be acquired from the participants from an interview (a) after the first round of usage; (b) after completing all test cases1. Ease of use / intuition

B2 2. To what extent does the system help with getting the work done?3. Does the user get easily acquainted to using the system (compare answers after first round to answers after completing all test cases)4. Completeness / correctness / helpfulness of test case documentation.


TEST PROCEDURE


TP1 X X XTP2TP3

TEST SCENARIO 2TEST

PROCEDUREMETRIC 1 METRIC 2 METRIC 3 METRIC 4 METRIC 5

TP1 XTP2 X X XTP3

TEST SCENARIO 3TEST

PROCEDUREMETRIC 1 METRIC 2 METRIC 3 METRIC 4 METRIC 5

TP1TP2TP3 X X X


Test Scenario T B1 B2TS1 XTS2 X XTS3 X X X


I1 I2 I3 I4 I5 I6 I7 I8 I9TS1 X X XTS2 X X XTS3 X X X



4.3 Furniture Use /Test Case for Piloting: “Interoperability for optimisation of Furniture e-Procurement” (AIDIMA)

1. GeneralAuthor AIDIMARoleDate 05/01/11Scenario of reference e-ProcurementTest Case name e-ProcurementTest Case number FUR_TC_1Issue 1. Build and/or use documents suitable for data and services exchange

between actors such as RFQ, Quotation or Orders. This would improve efficiency and communication and would save time and money due to the integration between their systems.

2. Build and/or use an ontological system that will allow different actors such as the manufacturer and the supplier to exchange data sharing a common semantic solving the problems of communication between actors of different countries, cultures, scope, etc.

Athena Solutions [Initial date for the Test]


Date of Test

Integration between furniture ERP systems

[ / / ] [ / / ]

Exchange data sharing common semantics

[ / / ] [ / / ]

[ / / ] [ / / ]

2. Introduction

The Test Case for the e-Procurement Scenario is based on a major Spanish office furniture manufacturer named Perfil Madera S.A., (Permasa – http://www.permasa.es) and placed close to Valencia. Permasa is a medium sized furniture manufacturer with strong European and domestic sales. Currently they are looking at implementing new technologies to assist its interactions with both customers and suppliers.

The Furniture e-Procurement scenario is divided in two parts depending on the role that Permasa is playing. In each case, apart from them there is another company implied: either a Supplier either a Retailer. Each of the sub applications identified in the above diagram (Logistics, Sales Management, Manufacturing and Purchasing) are custom built applications running on an IBM AS/400 system programmed in ILE (Integrated Language Environment)


Database (OS400)

Sales Mgmt. Manufacturing Purchasing

Web Server(IIS)

Customer Order processing

Manufacturing Procurement

Logistics

Design

ClientSupplier

http://www.permasa.es/


E-PROCUREMENT SCENARIO N.1 - SUPPLIER SIDE

Permasa detects a raw material needed for manufacturing a piece of furniture. For this reason, Permasa starts its procurement process following these steps:

1. Provider selection2. Material specification3. Quotation processing4. Order processing

Provider selection

Prior to perform the procurement activity, Permasa has to select the provider according to an internal category:

1. A1. Provider that is Acceptable & necessary2. A2. Provider that is Acceptable but not necessary3. B1. Provider that is Not acceptable but necessary4. B2. Provider that is Neither acceptable nor necessary

To purchase from any of the B class providers, the purchasing department need to seek permission from the managing director.

The grading mechanism to assign points to providers implemented by Permasa includes: ISO Certification Prepared Questionnaire by Permasa Premises Inspection Assignation of negative points for various faults found with the provider Conducts Quarterly reviews of the provider Ability to deliver on time If the provider falls out of scope of the acceptable range, they are dropped

Material specification

Once the provider is selected, Permasa can purchase two different categories of products from them: Serial (bulk) products: Usually used for the generic catalogue and for pre-manufactured

products Specialised products (Made-To-Order components): These involve a much deeper level of

interaction with the supplier and actually, Permasa does not have in mind to computerise this aspect as all the orders placed to the provider are done by paper/telephone/fax/email.

Quotation processing

Permasa sends an RFQ with the desired material to the selected supplier, and they send back a quote together with a product sample. After receiving the quote and if the product sample sent are approved by Permasa’s Quality Department, Permasa launches a Purchase Order to the Supplier, waiting for Supplier’s respond.

Orders processing

Once the order has been approved it is manually entered into Permasa’s system, giving it a number and specifying all the mandatory fields specified by Permasa’s system.

E-PROCUREMENT SCENARIO N.2 - CLIENT SIDE

Although Permasa deals with numerous clients throughout the world and some have technological implementations to better facilitate the purchasing of goods, Permasa currently does not have the relevant technological infrastructure to utilise these interfaces. Technologically speaking, Permasa receives an order for its goods by phone/fax/e-Mail with the same limitations that in the Scenario #1.

Permasa receives an RFQ asking for some pieces of furniture. This RFQ is processed by the Product Design Department which checks the ability on manufacturing the requested pieces and pricing the



pieces themselves. Afterwards, Permasa sends to the Retailer the Quotation with all these data and corrections or suggestions in case there is any error or Permasa is not capable to produce a specific piece.

After sending the quotation, the Retailer formalizes an Order by sending to Permasa the correspondent order. The order is manually checked for errors. Actually, Permasa currently has 3 people processing orders full-time. The bulk of their time is spent checking/revising orders - (this is due to the fact that very often the client make an erroneous order unknown to them).

Once the order has been checked, processed and internally approved by Permasa, they send to the Retailer an Order Confirmation with the price of the products and the delivery date.

By the time that the pieces of furniture are being manufactured, the administration department prepares the Delivery Note to be sent together with the products and the Invoice with the payment conditions.

THE PROBLEM

The following points are the main issues identified:1. Repetitive manual process for regular bulk orders. Much of the manufactured products are

generic and this involves repeated periodic processing of similar or identical orders2. Confusion resulting from poor product descriptions. Clients very often order the wrong

products!3. Missing information (both from supplier and buyer!). Permasa has 3 people employed on the

client side and 1 person employed on the supplier side to ensure the integrity of orders and RFQs

4. Lag. Time from product order to delivery could be shorter. Shortening time from ordering to receiving raw materials from the supplier has a direct effect on the delivery date of the finished product

5. Time spent rating supplier. Permasa conducts tri-monthly reviews of their suppliers to ensure that standards are kept

THE EXPECTATIONS

The expectations are:1. Major reduction in false/incorrect orders2. Dramatic shortening of time from order to delivery3. Dramatic reduction in surplus stock in warehouse4. Ability to search new providers and apply Permasa criteria to rate those providers5. Better integration between internal systems. (i.e. Stock, purchasing, manufacturing, invoicing

sub-systems)

3. Glossary of Terms , Acronyms, abbreviations of Test CaseTerm Description

Manufacturer Enterprise which its main activity is the creation of furniture, from its conceptualisation (Product Design) to the delivery.

Suppliers Enterprise which its main activity is to manufacture and to provide raw material, varnishes, paints, fabrics, … to the furniture manufacturers.

The raw material providers are not considered as furniture-makers but as woodworkers.

The products as varnishes, paints, fabrics, … are used to “finish” the furniture once manufactured.

Retailers Business which its main activity is to exhibit and to sell the furniture to the end-user.RFQ Request for QuotationILE Integrated Language Environment. Programming Language specific for IBM AS/400

systems.IIS Internet Information Server



Manufacturer Enterprise which its main activity is the creation of furniture, from its conceptualisation (Product Design) to the delivery.


Design: Technical Office (AutoCAD/Inventor)

All design work is done using AutoCAD and Inventor from AutoDeskAutoCAD/Inventor are NOT directly linked to the Custom Built suite of applications built by Permasa. There exists modules within Inventor that when cross referenced with stock database, can show the designer how much their design will cost in real time. (but this is out of scope for the moment)

Purchasing: Custom Built Application (AS/400 ILE)

Custom Built Application to manage the Purchases from the suppliers.Requires 1 person to interface between the "Permasa Purchasing Application" and the supplier. This person uses the "Permasa Purchasing Application" to make and track orders sent to suppliers. They also update the system once the goods have been received.

Manufacturing: Custom Built Application (AS/400 ILE)

Custom Built Application to manage the Factory floor processes, machines scheduling, man-power scheduling, products components manufacturing, etc.

Sales Management: Custom Built Application (AS/400 ILE)

Custom Built Application to manage the Sales to the retailers. It processes RFQs, orders, invoices, delivery notes, etc.Requires 3 people to manage

Web: Windows2000 server running IIS

Currently this is unrelated to the legacy company applications. Permasa currently have an application that a retailer can download to facilitate the correct ordering of products.

Logistics: Custom Built Application (AS/400 ILE)

Custom Built Application to manage the shipping of the products. Out of the scope of this scenario.

Design: Technical Office (AutoCAD/Inventor)

All design work is done using AutoCAD and Inventor from AutoDeskAutoCAD/Inventor are NOT directly linked to the Custom Built suite of applications built by Permasa. There exists modules within Inventor that when cross referenced with stock database, can show the designer how much their design will cost in real time. (but this is out of scope for the moment)




1. Integration between furniture ERP systems

2. Exchange data sharing common semantics

6. Assumptionsn. Description1 There exists a "wrapper layer" or some other sort of "Supplier Facing Application" to

interface between the Suppliers systems and the existing Permasa legacy system2 There will be a person to monitor the transaction



7. Pre-conditionsn. Description12345


TS1 Detection of errors in a Commercial e-DocumentTS2 Integration of a Commercial e-Document into the ERP systemTS3 Validation of price listsTS4 Integration of price lists

8.1. Test Scenario – TS1: Detection of errors in a Commercial e-DocumentItem DescriptionGoal Manufacturer-side applications are capable to communicate any error

from the Commercial e-Documents receivedPre-conditionsTest Creator AIDIMAData Creator 05/01/17Data Test 05/Test executor TBDInternal Test Assistant TBDExternal Test assistant TBDSuccess Post Conditions 1. The e-Documents with semantic errors are detected

2. The Retailer is warned about the errors detected3. The e-Document is not processed

Failed Post Conditions 1. The e-Document is processedResultsComments8.2 Test Scenario – TS2: Integration of a Commercial e-Document into the ERP systemItem DescriptionGoal Manufacturer-side business users are capable of integrate into their ERP

a Commercial e-Document that has been received through an Internet-based application from the Retailer-side without human intervention

Pre-conditions 1. The e-Document has no errors in terms of non-existent productsTest Creator AIDIMAData Creator 05/01/17Data Test 05/Test executor TBDInternal Test Assistant TBDExternal Test assistant TBDSuccess Post Conditions 1. The e-Document is successfully inserted in the system

2. The e-Document is successfully processedFailed Post Conditions 1. The e-Document is rejectedResults Manufacturer-side business users are capable of integrate into their ERP

a Commercial e-Document that has been received through an Internet-based application from the Retailer-side without human intervention

Comments 1. The e-Document has no errors in terms of non-existent products8.3 Test Scenario – TS3: Validation of price listsItem DescriptionGoal Manufacturer-side applications are capable to communicate any error

from the price lists receivedPre-conditions 1. Use of the Semantic system



Test Creator AIDIMAData Creator 05/01/17Data Test 05/Test executor TBDInternal Test Assistant TBDExternal Test assistant TBDSuccess Post Conditions 1. The semantic errors in the price lists are detected

2. The Provider is warned about the errors detected3. The price lists are not processed

Failed Post Conditions 1. The price lists are processedResultsComments8.4 Test Scenario – TS4: Integration of price listsItem DescriptionGoal Manufacturer-side system is capable to integrate the price lists received

through an Internet-based application from the Provider-side systemPre-conditions 1. The price lists have no semantic errorsTest Creator AIDIMAData Creator 05/01/17Data Test 05/Test executor TBDInternal Test Assistant TBDExternal Test assistant TBDSuccess Post Conditions 1. The price lists are successfully inserted in the system

2. The price lists are successfully processedFailed Post Conditions 1. The price lists are rejectedResultsComments


TP1 Processing an e-DocumentTP2 Processing the price lists

9.1 Test Procedure TP1 Description

Step1 Receive an e-DocumentStep2 Check the structure for any possible syntactic errorsStep3 Check the content for any possible semantic errorsStep4 Insert the e-Document into the ERP systemStep5 Integrate the content of the e-Document into the AS/400Step6 Processing of the content of the e-Document (data interpretation, …)Step7 Generate the appropriate e-Document to respondStep8 Send the responding e-Document

9.2 Test Procedure TP2 Description

Step1 Receive the price lists in an electronic formatStep2 Check the semantic of the price listsStep3 Identify the Provider for inserting the dataStep4 Insert the price lists into the AS/400 database


TypeDescription



M1 T

M2 B1

M3 B2

M4

M5

M6

M7

M8

M9


TEST METRIC 1 METRIC 2 METRIC 3 METRIC 4 METRIC 5



PROCEDURETP1TP2TP3TP4TP5TP6TP7TP8TP9

TP10


Test Scenario T B1 B2TS1TS2TS3TS4TS5TS6TS7TS8TS9

TS10


I1 I2 I3 I4 I5 I6 I7 I8 I9TS1 XTS2 XTS3 XTS4 XTS5TS6TS7TS8TS9

TS10



4.4 Telecom Use /Test Case for Piloting (INTRACOM)1. GeneralAuthor INTRACOM S.A.RoleDate 2005/01/24Scenario of reference PPMTest Case name PPM – Model Generated Workplaces Test Case number TEL_TC_01Issue Considered issueAthena Solutions [Initial date for the Test]


Date of Test

MPCE [ / / ] [ / / ]EKAs [ / / ] [ / / ]MGWP [ / / ] [ / / ]

2. Introduction

The following paragraphs present INTRACOM’s test case for the first period of ATHENA programme. The test case is based on the “to-be” scenario related to the Product Portfolio Management process. It initially focuses on the use of knowledge-based workplaces dynamically generated on top of enterprise models to support collaborative work. Models Generated Workplaces are being developed in A1 project. Therefore, this is the work performed in collaboration with A1 partners. Further collaboration with the rest of AL A projects is in progress, investigating how complementary aspects could be addressed.

The problem: Actual Situation – General DescriptionThe management of product portfolio is of significant importance especially to large enterprises with many business units and complex products. The development of a telecommunication product requires the collaboration of many different engineering teams inside the same business unit or corporate environment, as well as within a business network of collaborative enterprise. The focus of the specific case under study is at an intra-enterprise level. However, even inside the corporate environment the collaboration among the actors involved in the different projects is inadequate. Actors require updated information from various sources (marketing, project execution) and from different phase of the product life cycle (BOL, MOL).Information gathering is a time consuming activity, while in many cases critical information is often lost although it exist somewhere. In addition, due to the involvement of various teams, it is difficult to spot the persons associated to specific activities or persons that worked in related projects in the past.Furthermore, holistic views of the different activities are missing, and thus it is difficult to identify how changes and delays propagate into the various activities. One more very important problem is that business decision-making activities are not sufficiently linked to the enterprise operational system.

ExpectationsThe PPM use case investigates into the use of Model Generated Workplaces to support simultaneous project, resource and results management, performance measurement through work management views, and the provision of shared project and work monitoring views. The overall objective is to support collaborative work by providing the actors in the enterprise with the tools, information and communication support they need to efficiently perform their work. This could be splitted down into the following expectations: Support faster initiation and assessment of collaboration Support integration of the various associated enterprise processes, information and roles / people

through a single point of entry Provide the different actors with an integrated view of the project, products or development

initiatives depending on their roles Provide the actors with up to date information related to sales, products, project plans and

progress status, available and consumed resources Improve the communication between the different participants and the coordination work Facilitate more justified decisions.



INTRACOM’s test case focuses on the telecommunication business sector of the company. The requirements analysis and validation is based on the development project of a specific product, the WIBAS (Wireless Broadband Access System). WIBAS is a new generation LMDS system supporting fixed wireless broadband services over packet technology (ATM/IP). Therefore, the business application team that provides requirements and will validate the solution is formed around this product including the related product and project managers, a team leader, a business development consultant and a quality engineer.

The initial test scenarios are likely to focus on three actors and the services they need to perform their every day tasks. More specifically, the test cases may focus on the Project Manager, the Quality Engineer and the Business Development Manager, the final selection of which will ultimately depend on their weighted importance, resulting from our internal assessment. As it has already been mentioned, these scenarios are part of the “to-be” scenario related to PPM. Therefore, they are subject to future alterations, because the “to-be” scenario is currently under development. As soon as the first version of the “to-be” scenario is available, the test scenarios will be described in more detail, taking into consideration the results of our internal assessment.

3. Glossary of Terms , Acronyms, abbreviations of Test CaseTerm DescriptionPPM Product Portfolio ManagementPM Product Manager – Human ResourcePjM Project Manager – Human ResourceTM Team Manager – Human ResourceQE Quality Engineer – Human Resource

BDM Business Development Manager – Human ResourceERP Enterprise Resource Planning System


Product Manager

The Product Manager (PM) is assigned to a product or product family and is responsible for developing or overseeing all aspects of the product including product definition, product development, product launch, current product management, and product obsolescence. Therefore, the PM is the connective link among all the different units and functions involved in the different phases of the product life cycle.

Project Manager

The Project Manager (PjM) conducts a product development project. The project manager has to exercise the proper control in order to bring about the project deliverables on time and within the budget. The activities involve the definition of the project deliverables, organisation of the project team, planning and allocation of the project team task/responsibilities/dependencies, financial planning and control, monitoring, assessing and reporting of the project implementation execution, quality planning and assurance.

Team Manager

The Team Manager (TM) is a member of a section that provides services to project management and coordinates a specific sub-project that is assigned to his/her section. This is especially the case in large projects that sub teams need to set up to handle and implement part of the development project. The TM has also to keep the Section Manager informed about the sub-project progress.For the particular stage of the project development (e.g. software development) that the team manager is responsible, he/she has to provide the first draft specification taking care the design of the sub-module to allow its reusability. Most often the Team Manager is also involved in engineering work.

Quality Engineer

The Quality Engineer (QE) provides services and guidance to the PjMs and supports them to achieve their objectives. The goal of the QE is to drive project management through a quality mandate and ensure customer satisfaction no matter if it is about an internal customer (Product Development Projects) or an external one (Customer



Delivery Projects).Business Development Manager

The Business Development Manager (BDM) is responsible for the management of customer relationships. The BDM is mainly involved in the pre-sales phase having to contact Intercom’s customers, inform them about the company’s products, identify their needs and how to satisfy them.The BDM is also involved in the process of preparing a BID to respond to a request for tender released by a potential customer, as well as in the contract preparation in the case of successful BIDs.The BDM needs to be provided with product folders comprising all the related product information to support the meeting with the customers. This information includes product presentations, technical descriptions, white papers etc. In addition, the BDM needs to be provided with different views including customer view, product, project, and resource view in order to fulfil the different activities s/he is involved in.

ERP The enterprise resource planning system that needs to be integrated with the workplaces of the majority of the actors to follow-up the budget of the project, the available resources

Microsoft Project

The project management tool used in the case


Actual Test/Use Case Diagrams to be derived upon selection and finalisation of actors/roles/test scenarios.

6. Assumptionsn. Description1 The business application team is formed around the WIBAS project



TS1TS2TS3TS4TS5

8.1. Test Scenario – TS1Item DescriptionGoal To support the Project Manager in his/her every day tasksPre-conditionsTest Creator INTRACOMData Creator 05/01/20Data Test 05Test executor TBDInternal Test Assistant TBDExternal Test assistant TBDSuccess Post Conditions 1. The PjM sets up the project database that is shared by all the



involved actors and workplaces.2. An updated view of each sub team tasks is available in the PjM’s

workplace.3. Project History is updated with the ongoing and pending tasks,

including also current cost information coming from the ERP.The actual cost of the project is accordingly updated in the ERP system.

Failed Post ConditionsResultsComments8.2 Test Scenario – TS2Item DescriptionGoal To allow the Quality Engineer to have a view in the current status of the

projectPre-conditionsTest Creator INTRACOMData Creator 05/01/20Data Test 05Test executor TBDInternal Test Assistant TBDExternal Test assistant TBDSuccess Post Conditions The QE’s workplace is provided with the latest project reports

and updated plans. Failed Post ConditionsResultsComments8.3 Test Scenario – TS3Item DescriptionGoal The Business Development Manager to be supported in his/her

everyday tasks.Pre-conditionsTest Creator INTRACOMData Creator 05/01/20Data Test 05Test executor TBDInternal Test Assistant TBDExternal Test assistant TBDSuccess Post Conditions 1. The Business Developer manager is provided with the latest

information about the status of the product (under development, production etc), the supported documents coming from the product management and the technical specifications.

2. The workplace is updated with the latest information regarding the contracts with a specific customer, the products involved and the related sales data.

3. The workplace is updated with the latest information about the availability of the personnel that potentially will form a bidding team.

Failed Post ConditionsResultsComments


TP1TP2TP3TP4TP5




Step1Step2Step3Step4Step5








TypeDescription

M1 T

M2 B1

M3 B2

M4



M5

M6

M7

M8

M9


TEST PROCEDURE


TP1TP2TP3TP4TP5TP6TP7TP8TP9

TP10


Test Scenario T B1 B2TS1TS2TS3TS4TS5TS6



TS7TS8TS9

TS10


I1 I2 I3 I4 I5 I6 I7 I8 I9TS1TS2TS3TS4TS5TS6TS7TS8TS9

TS10



4.5 Aerospace Use /Test Case for Pilot “Aerospace CPD based on Semantic Mediation” (EADS CCR)

1. GeneralAuthor EADS CCRRoleDate 2005/04/10Scenario of reference Collaborative Aerospace Product development within networked

organisationTest Case name Aerospace CPD based on Semantic MediationTest Case number AER_TC_01Issue Considered issueAthena Solutions [Initial date for the Test]


Date of Test

Ontology Authoring and Management System for informational knowledge and Business Processes

M12 (ATHOS) M13

Semantic Annotation language and tool for information and Business Processes

M26 M26

System for reconciliation rules specification, storage and management.

M28 M28

A reconciliation and mediation engine, capable to efficiently process semantic mediation and reconciliation rules

M30 M30

2. Introduction The following paragraphs present EADS CCR’s test case for the second period of ATHENA programme. The test case is based on the “to-be” scenario related to the Collaborative Aerospace Product development within networked organisation, in particular for the NECD and PDM Coupling processes.

They are motivated by the fact that semantic mediation is already used in Aerospace industry, using STEP based or XML technologies, and based on usage of STEP based applications protocols that are neutral standardised ontologies build by international experts.

Even if initially A3 is not targeting directly AL B, but other AL A projects, A3 results appear as a key component of ATHENA solutions, and it is of prime importance for Aerospace to validate that such a component will be able to be used with such ontologies, and to validate/demonstrate that it provides real added value and innovation compare to existing solutions and used solution

It is consequently targeted to provide to A3 a Reference Ontology built from STEP application protocols to check if it is usable with the different results of A3, in particular OPAL and ATHOS.

The problem: Actual Situation – General DescriptionToday, in particular between Aerospace Integrator and Equipment providers, exchange/sharing of product data and related organisational data can’t be achieved by the mean of specific proprietary formats between the heterogeneous tools that are used. In order to avoid multiple mappings, some protocols are established based on STEP application protocols. Exchanges are then done based on STEP technologies (SDAI, Express, Part 21) or other technologies for which bindings are provided



(XML, programming languages, etc). The protocol by itself is reference ontology, and is technology used independent, being implementation or modelling.

Today, product data exchange/sharing strategy for Product Life Cycle management is targeting to use STEP Application protocols, and it is consequently a constraint to ensure that these protocols can be used in conjunction with ATHENA solutions.

As currently used in operational context, it is also important to demonstrate that there is no regression with ATHENA solutions compare to what is already running.

ExpectationsExisting in Aerospace and Manufacturing for product data exchange is already based on usage of semantic mediation: STEP application protocols are used exactly the same way proposed by A3.

EnterprA EnterprB

EnterprC

Semantic Hub

STEP Application Protocol

STEP Application Protocol is an standardised ontology in manufacturing area

Based on STEP technologies , that arebased on the same principle of interchange (Express, Express-X)

The used technologies are those coming ISO STEP technological framework, implemented by commercial products as Express Data Manager. The schema description language is Express-X while Express-X is used for transformation. As an alternative, it is also possible to use XML (an XML binding is provided by ISO STEP) and XSLT for transformation.

The A3 proposed approach is the following one: a semantic hub is provided, based on ATHENA A3 innovative ontology technologies in one hand, in the usage of a Reference Model from experts of the field in the other hand. The used tools are ATHOS, based on OPAL and OWL, and transformation based on A*. These tools are implementation of the targeted results from 13, i.e. Ontology Authoring and Management System for informational knowledge and Business Processes, Semantic Annotation language and tool for information and Business Processes and System for reconciliation rules specification, storage and management. They correspond functionally to authoring and management system for Express Schema (e.g. Express Data Manager from EPM technology), mapping description and execution (with Express-X and rules description). This last approach is precisely annotation but it can very precisely target subset of information and provide meta-data that consist in additional information and rules. The syntax for metadata is not different from those of data. It is also possible to use XML technologies (using Part 28 STEP-XML binding).A reconciliation and mediation engine, capable to efficiently process semantic mediation and reconciliation rules



EnterprA EnerprB

EnterprC

Semantic Hub

RO

RO from expertsof the fieldsRO from expertsof the fields

Based ATHENA A3 innovative ontology based technologies and meta models (RO formalized using ATHOS, based on OPAL and OWL, transformation based on A*,…)Based ATHENA A3 innovative ontology based technologies and meta models (RO formalized using ATHOS, based on OPAL and OWL, transformation based on A*,…)

What is expected is to be able to use A3 semantic hub using as Reference Ontology from experts of the fields, any STEP application protocol, in order to exchange product data between two heterogeneous PDM application, following a classical manufacturing change management process.

As OPAL is based on OWL, EADS CCR will develop and provided transformation of STEP application protocols in OWL, as input for ATHOS. It is expected to be able to be able to reuse OWL file to create Reference Ontology without a lot of effort.

It is also awaited to be able to annotate documents or information from one PDM application using A3 tools on the basis of the application protocol.

It is awaited then to be able to define and execute reconciliation rules based on the ontology of reference.

3. Glossary of Terms , Acronyms, abbreviations of Test CaseTerm DescriptionAP Application Protocol

CPD Collaborative Product DevelopmentOWL Ontology Web LanguageOPAL

RO Reference OntologyPDM Product Data ManagementEDM Express Data ManagerXML

STEPPLM Product Lifecycle ManagementPDM Product Data ManagementCC Conformance ClassUoF Unit of Functionality




PDMS 1 PDMS 2WindchillEDM Express Data Manager, tool that manage STEP models and Schema, their validation,

their transformationATHOS Semantic mediation tool provided by A3, implementing A3 concepts.


6. Assumptionsn. Description1 The two PDMS are two different applications, that can be or not implemented using

the same software product.2 The two PDMS allow exporting data and the related Application schema, respectively

with the format P21 and Express.3 The change management process of the two different concerned organisation is not

the same4 The change management process is implemented by the mean of a workflow engine5 Description of roles is different for the two organisation6 The two organisations are not in the same country, and the native language is not the

same7 In the concerned Aerospace network, exchange protocols are based on AP233

(system engineering), AP214 (Product Data Management and mechanical Digital Mock Ups), PLCS is used for the customer support. The CAD models can be both in native formats or in STEP format

8 The exchange/sharing information is performed on the Internet (B2B)9 Exchange/shared data are confidential10 Each partner want to expose as few as possible concerning their internal process11 Quality of exchange/sharing is very important. No loose of data or incoherency is

allowed.12 The PDM software Conceptual Model can be made available in UML13 The workflow process model can be made available in XPDL14 The services of the PDM Systems can be made available as CORBA or WEB services15 The PDM systems implement interfaces of PDM Enablers and PLM services16 The PDM applications manages multiple engineering views (System design, product

design, production, support) and multiple disciplines (simulation, mechanical, electrical, system)

17 Simulations are also managed18 The network used CMII as reference model for Change Management and

Configuration Management





TS1 Import of AP214, AP233, AP239 Application Protocol (OWL) as Reference Ontology from expert of the field

TS2 Import of AP214, AP233, AP239 Application Protocol (OPAL) as Reference Ontology from expert of the field

TS3 Annotation of PDMS1 and PDMS2 using Semantic Annotation language and tool for information and Business Processes

TS4 Definition of reconciliation rules specification between each PDM System for the change management processes and the product data

TS5 Execution of the reconciliation rules for automatic workflow interconnection and product data exchange/or sharing

8.1. Test Scenario – TS1Item DescriptionGoal To be able to use Standardized Ontology of the field as Reference

Ontology and to measure adaptation for efficient usage of OPALPre-conditions Reference ontology for each application protocol available in OWL

formatTest Creator EADS CCRData Creator EADS CCRData Test 05Test executor EADS CCRInternal Test Assistant CNR IASIExternal Test assistant Success Post Conditions Possible to visualize the RO with ATHOS, with no lost of OWL objectsFailed Post Conditions No possible to visualize the RO with ATHOSResults OWL can be successfully imported (standard for Ontology on WEB), and

most of the Application Protocol Entities can be used for annotation. It should allow to identify and measure efforts to adapt transformation targeting directly OPAL

Comments At this stage, as OPAL is a very rich ontology, there will probably a lot of adaptation to perform in order to fully match between the Application Protocol. The mapping tested there is at the meta level, so not really rich. TS2 should allow to go to the next step.

8.2 Test Scenario – TS2Item DescriptionGoal To be able to use Standardized Ontology of the field as Reference

Ontology in OPALPre-conditions Mapping between OPAL and the Application Protocols, implementation

of the mapping with the STEP Mapper, import schema based on OWL defined and implemented with ATHOS. TS1 successful.

Test Creator EADS CCRData Creator EADS CCRData TestTest executor EADS CCRInternal Test Assistant CNR IASIExternal Test assistantSuccess Post Conditions OWL/OPAL can be successfully imported (standard for Ontology on

WEB), and all the Application Protocol Entities can be used for annotation.

Failed Post Conditions Not possible to import the OWL/OPAL file. Some information of the application protocols entities are lost in the import.

Results Application protocols can be used as Reference Ontology with ATHENA ontology Authoring and Management System for informational knowledge and Business Processes.

Comments



8.3 Test Scenario – TS3Item DescriptionGoal Annotation of information of each PDMS information using ATHENA

annotation system and Application protocols as RO.Pre-conditions PDMS 1 and 2b information are accessible to the annotation system

(possible way to proceed to be defined and eventually implemented based on other AL A project – A2? A5? A6?). TS1 to TS2 successful.

Test Creator EADS CCRData Creator EADS CCRData TestTest executor EADS CCRInternal Test Assistant CNR IASI?External Test assistantSuccess Post Conditions Information of the main contractor being annotated according

manufacturing RO.Information of the sub-contractor being annotated according manufacturing RO.(with the agreed access method)

Failed Post Conditions Not possible to annotate one of the information of PDMS1 and PDMS2ResultsComments Annotation should concern “customised” information, i.e. to target the

parameterisation of the PDM Systems8.4 Test Scenario – TS4Item DescriptionGoal Definition and formalisation of reconciliation rules using system for

reconciliation rules specification, storage and management.Pre-conditions System for reconciliation rules specification, storage and management

available, reconciliation rules defined on the base of RO. TS1 to TS3 successful.

Test Creator EADS CCRData Creator EADS CCRData TestTest executor EADS CCRInternal Test Assistant CNR IASI?External Test assistantSuccess Post Conditions Possible to enter and store the defined reconciliation rules based on RO

with the system.Failed Post Conditions Not possible to enter nor to store the defined reconciliation rulesResults Possible to formalize mapping by the mean of reconciliation rulesComments No regression compare to usual solution, for example with Express-X. If

any advantage, should be precisely identified and promoted.8.4 Test Scenario – TS5Item DescriptionGoal Execution of semantic reconciliation of process interconnection and

product data exchange or sharing.Pre-conditions System for execution available. TS1 to TS4 successful.Test Creator EADS CCRData Creator EADS CCRData TestTest executor EADS CCRInternal Test Assistant CNR IASI?External Test assistantSuccess Post Conditions Possible to enter and store the defined reconciliation rules based on RO

with the system.Failed Post Conditions Not possible to enter nor to store the defined reconciliation rulesResults Possible to formalize mapping by the mean of reconciliation rulesComments No regression compare to usual solution, for example with Express-X. If



any advantage, should be precisely identified and promoted.


TP1 AP schema and models oTP2TP3TP4TP5









10. MetricGeneral approach:For this test case, we shall focus on five aspects of interoperability, each of which will be measured by a set of metrics. These aspects are:- Usability of Ontology of the field for each AP, that can be related to data requirements according

VOLERE classification described in DB4.1(see M1)- Usability (see M2)- Performance- Openess (see M5)- Integration with other ATHENA solutionsMetric n. Validation

TypeDescription

M1

Ratio number of OWL items imported in the RO (ATHOS)/OWL AP itemsRatio number of OWL items mapped in the OPAL RO (ATHOS OPAL)/OWL AP items

B Ratio number of class and properties transformed/ number of source classes and properties for different AP



Ratio number of individuals transformed/ number of source individuals for different models based on targeted APRatio number of UoF, CC or modules fully covered/ number of UoF, CC or modules per APComparison of these ratio with already existing alternative solutions (EDM)

M2

Time required to quickly and accurately use ATHOS for creating an ontology of reference, to navigate in an already created ontology, to perform annotation, to produce a set of set of robust transformation rules of a module (statistically).Should be an average of one month to be fully operational.Ease of remembering – how much is the casual user expected to remember about using the product

B Error rates to produce satisfying set of transformation rules : number of change requests for a validated set of rules due to error related to the usage of tool (scale to be defined from experience and comparison with other tools)Overall satisfaction in using the method and prototype (number of user satisfied after using the product) - An anonymous survey shall show that 75% of the ATHENA partners are regularly using the product after one month familiarization period.

M3

Response time : time to display a complete RO <10 secondsResponse time : time to display details of an RO<5 seconds

T Time to perform a transformation for a set of information should be some second for a little population (<100), and minutes for a huge (build on a comparison with other transformation tools). Should be able to transform models with 100 000 individuals and with RO of 1000 classes (to be validated with figures from industry).

M4

Number of standardised formats that can be used for import/exports for ontologies

B Number of standardised formats that can be used for import/exports for transformation rules

M5

Number of enterprise modelling solutions that can be connected through A1 solutionNumber of enterprise modelling ontologies from that can used as source/target ontologies

T Number of CPD solutions that can be integrated (from A2).Number of services that can be made available though Services Bus (A6)Number of MDA applications that can be source/target for semantic reconciliation in application and software engineering (A5)


TEST PROCEDURE


TP1TP2TP3TP4TP5TP6TP7TP8TP9

TP10




Test Scenario T B1 B2TS1TS2TS3TS4TS5TS6TS7TS8TS9

TS10


I1 I2 I3 I4 I5 I6 I7 I8 I9TS1TS2TS3TS4TS5TS6TS7TS8TS9

TS10



5 Conclusion

The partners involved in the process of the test case definition have provided useful collections of real-world problems inside their organizations and formalized the description of these challenges. The problems pointed out during the scenario definition are described in detail in this document. In this document, also a first step has been done in overlaying this problem-driven view with a perspective of technological possibilities created through ATHENA innovation.

The main results obtained from this activity are a set of expectations, a perspective to a way of solving these problems, and the solutions that the partners intend to adopt in order to reach their expressed objectives and satisfy the expectations.

In the automotive domain, SIEMENS and C.R.F. have highlighted problems related to the presence of different applications that manage information in different way, data stored in different databases using different formats, open issues in applications integration. In particular, C.R.F. would integrate heterogeneous applications to obtain an environment on which it is possible to connect and synchronize all the applications, to have the possibility to exchange enterprise models in a multi-language environment, exchange enterprise models in a multi-application environment and execute these enterprise models after their integration on existing platforms. The solutions, on which the tests will be performed and that should allow reaching the objectives are MPCE, EKAs and MGWP.

SIEMENS proposes “Adaptive P2P resource management” and “Pro-active workflow management and automation” as two technical approaches to address interoperability problems in a cross-enterprise context. The problem pointed out concerns the data exchange (accuracy and quality of final technical specifications) among OEM, supplier and supplier of supplier. The main aspects analysed are the limitation of centralised application architectures in the context of cross-company and cross-market process flows, and the presence of heterogeneous applications that manage processes in different way. Some expectations are: facilitate decentralized management of business objects in a cross-enterprise way, observe security aspects, enable the goal-directed propagation of changes across the collaborative network, improve the interleaving of the suppliers’ CRM and SRM processes, and integrate new processes seamlessly with existing applications through an open service-oriented IT architecture.

In the e-Procurement domain, AIDIMA bear evidence on problems regarding the repetition of manual processes and redundancies of data orders, integrity of data and aspects concerning the conformity with expected data standards. AIDIMA would obtain integration between ERP systems and use a common semantics in data sharing. The ATHENA solutions should allow solving these two relevant needs.

INTRACOM notices that the collaboration among the actors involved in the different projects is inadequate because actors require updated information from various sources (marketing, project execution) and from different phase of the product life cycle (BOL, MOL). Relevant aspects stressed are: technical information loss, technical chronicle of activities and projects, knowledge of the impact of delays in others enterprise activities and its management, decision-making activities not directly linked to the enterprise operational systems.

The objectives of INTRACOM are to use the solution like MGWPs, MPCE, EKAs to reach its expectation. The pilot on which the solutions will be tried is WIBAS (Wireless Broadband Access System) LMDS system supporting fixed wireless broadband services over packet technology (ATM/IP).



6 Glossary

Assessment: An action of applying specific documented assessment criteria to a specific software module, package or product for the purpose of determining acceptance or release of the software module, package or product. (ISO 9126: 1991, 3.1)

Level of performance: The degree to which the needs are satisfied, represented by a specific set of values for the quality characteristics. (ISO 9126: 1991, 3.4)

Pilot: The implementation of use case solutions in a real context of industrial users. Pilots are employed to test in a limited context the functionality and appropriateness of a use case solution before its extension on the whole enterprise reality. [ATHENA]

Piloting: is the best offered by providing guidelines approaches on how to solve, design and engineering applications and challenges that until now have been poorly or not at all solved. [ATHENA]

Testing : Activity performed on architectures, components and solutions at all three level of the Enterprise Architecture.[ATHENA]

Test Case: [1]: A set of inputs, execution preconditions, and expected outcomes developed for a particular

objective, such as to exercise a particular program path or to verify compliance with a specific requirement [ATHENA].

[2]: A test scenario can be defined as a Test Case instance[ATHENA].

Test Scenario: A set of inputs, execution preconditions, and expected outcomes developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement [ATHENA]

Test Procedure: is the step-by-step process needed in order to verify that the product meets all the interoperability requirements established [ATHENA]

Use Case: is a collection of possible sequences of interactions between the system under discussion and its Users (or Actors in Business Processes), relating to a particular goal.

A use case defines a goal-oriented set of interactions between external users and the system under consideration or development. Use cases allow capturing functional requirements for a Business case. [ATHENA]

Validation:[1]: confirmation by examination and provision of objective evidence that specifications conform to

user needs and intended uses, and that the particular requirements implemented can be fulfilled. The primary focus of validation is customer satisfaction. [ATHENA]

[2]: Confirmation by examination and provision of objective evidence that particular requirements for a specific intended use are fulfilled” (ISO/IEC 9126)
