V0.3 | 2020-06-08 EMCC 2020 Virtual Edition, 30 th of July 2020 Towards Continuous Software Development for AUTOSAR Systems
V0.3 | 2020-06-08
EMCC 2020 Virtual Edition, 30th of July 2020
Towards Continuous Software Development for AUTOSAR Systems
2
Introduction
Applying Cx to AUTOSAR Development
AUTOSAR Continuous Integration
AUTOSAR Continuous Delivery
Summary & Outlook
Agenda
3
All Tesla vehicles with our FSD computer have been updated with new software that can better detect new details in their environments […]. We are currently validating this functionality before releasing to customers, and we look forward to its gradual deployment. We also introduced in-app purchases, where our customers can buy various software updates, such as basic Autopilot, Full Self Driving […]. Software will continue to play a growing role in our business model.
[Source: Tesla:“Q4 and FY2019 Update”]
Example of a Modern Software Product
Introduction
Tesla Financial Report Q4/2019:
All Tesla vehicles with our FSD computer have been updated with new software that can better detect new details in their environments […]. We are currently validating this functionality before releasing to customers, and we look forward to its gradual deployment. We also introduced in-app purchases, where our customers can buy various software updates, such as basic Autopilot, Full Self Driving […]. Software will continue to play a growing role in our business model.
[Source: Tesla:“Q4 and FY2019 Update”]
4
Maturity Levels of Continuous Software Development
Introduction
Plan Code
Agile
SCRUM, Feature Driven Development, etc.
Feature backlog with ranked tasks
Sprint based feature planning
5
Maturity Levels of Continuous Software Development
Introduction
… as before, + Server based system with:
Plan Code Build Test
Feedback
Agile
Continuous Integration
Automatic software
configuration
Software generation (ASW, RTE,
BSW)
Software compilation
Software linking
Unit-testing (Integration
Tests)
Short feedback channel for developers
+
6
Maturity Levels of Continuous Software Development
Introduction
Plan Code Build TestValidate & Pack
Test
Feedback
Agile
Continuous Integration
Continuous Delivery
… as before, + Server based deployment system with:
Automatic functional
tests
Rollout to staging
area
Automatic non-
functional tests
Automatic regression
tests
Automatic acceptance
tests
Product packing (executable,
docu, release-notes, …)
+ +
7
Maturity Levels of Continuous Software Development
Introduction
… as before, +
Plan Code Build TestValidate & Pack
Test Release Test
Feedback
Agile
Continuous Integration
Continuous Delivery
Continuous Deployment
automatic software deployment to fleet
8
Plan Code Build TestValidate & Pack
Test Release Test Operate
Maturity Levels of Continuous Software Development
Introduction
Feedback
… as before, +
Agile
Continuous Integration
Continuous Delivery
Continuous Deployment
Continuous Development & Operation (DevOps)
Feedback
+
automatic collection of feedback from operation
Including feedback in product planning
9
Continuous Software Development (Cx)
Good approach for dynamic times with many changes (e.g. technological) & flexible customers
What is Continuous Software Development in Automotive Industry about?
Introduction
Traditional Software Development
Good approach for stable times & highly reliable products
Product Update
Planning
Feature Structure
Stop of Planning
Testing
Release
New product with major features Small feature increment updates for released product
Long term (3-6 y) + detailed break-down, all stakeholders approve
Abstract vision + short term & flexible ranking (1-4 sprints), product owner considers stakeholders
Top-down decomposition of functionality Cross-structure features
Early feature freeze Dynamic changes in function scope
Long testing interval, manual tests (SIL, HIL, car) Fast testing, nearly 100% automation
New major software in 1-3 years Tiny software increment with weekly/daily updates
11
Introduction
Applying Cx to AUTOSAR Development
AUTOSAR Continuous Integration
AUTOSAR Continuous Delivery
Summary & Outlook
Agenda
12
… where it brings the most benefit
When activities are done with a high frequency, e.g. per commit
When activities are time-critical, e.g. before release/testing
When activities are repetitive and error prone
(Duration/Effort is relative – depends on time criticality)
Where to start with Cx
Applying Cx to AUTOSAR Development
traditional
objective
13
Plan Code Build TestValidate & Pack
Test Release Test Operate Test
Feedback
Test
Where to focus at AUTOSAR Projects to advance in Cx
Applying Cx to AUTOSAR Development
Our focus for this talkState of the art For future
Checking timing requirements
e.g. meeting reaction constraints
Validating capacity boundaries
Staying inside predefined boundaries for software parts / ECU resources (CPU, memory, stack)
(functional testing is not considered here)
Cross-Team/Organizational Development
Integration of 3rd Party Software
ASW to BSW integration
Manual job by integrator (often bottleneck for executable software)
Providing fast testing possibilities
I – AUTOSAR Continuous Integration II – AUTOSAR Continuous Delivery
14
Introduction
Applying Cx to AUTOSAR Development
AUTOSAR Continuous Integration
AUTOSAR Continuous Delivery
Summary & Outlook
Agenda
15
For Developers
Getting testable software in hours instead of weeks
Reducing number of manual & monotonic tasks
For Integrators
Reducing repetitive tasks and focus on more complex system integrations (conflicts)
For Project Manager
Having early & continuous transparency on progress of implemented functionality
Objectives for Continuous Integration
AUTOSAR Continuous Integration
Plan Code Build Test
Feedback
Continuous Integration
Efficiency
Speed
Change of Mindset
16
Eliminating exclusive operations on central artefacts
Preventing blocking of other engineers
Applying Decision Frontloading for integration
For automatizing integration activities
Use virtual testing
Executing Smoke Test for minimal executability check
Major Concepts for AUTOSAR Continuous Integration
AUTOSAR Continuous Integration
17
Current Approach
System Architect
Updates Communication-Matrix or Diagnostic Description
SW Developer
Modifies SWC description according to changed software
Integrator
Applies all changes by integrating application software to BSW> Runnable to Task mapping
> Runnable Positioning inside Task
> Port Mapping
Integrator is the bottleneck for SW Developers
Cross-Team/Organizational Development
AUTOSAR Continuous Integration
System Description
ECU-C
OEM / System Architect
changes
Com-Matrix & Diagnostic Description
update SWC Description
SW Developer
SW Developer
SW Developer
Integrator
applies changes
18
New Approach
Cross-Team/Organizational Development
AUTOSAR Continuous Integration
ECU-C (Root)
SWC Description
(cutout)
Inte-gration Instr.
Auto-Integration
Diagnostic Description
Communication
Description
System Description
SWC Description
ECU-C
(Base)
SWC Description
(cutout)
Inte-gration Instr.
SWC Description
(cutout)
Inte-gration Instr.
ECU-C Root’
Auto-Integration
modified modified modified
build
build
ECU SW Architect/Integrator
BSW/Platform Expert
SW Developer
Modify & implement
Auto-Integration
build
Modify & implement
Auto-Integration
build
Modify & implement
organize
Executable with local changes
Executable with all changes
clone
19
Workflow
AUTOSAR Continuous Integration
Root Configuration
DPA project configured for the used ECU, without apps and RTE
Integration Result
RTE configuration based on integration Instructions
App-dependent BSW configuration
App Package
SWC (atomic or composition)
Integration Instructions
Implementation (Source-Code)
App SWC
App SWC
App SWC
Platform BSW
App SWC
App SWC
App SWC
RTE
Platform BSW
Option CI
20
Major activities in case of changed Application Software
1. Runnable to Task Mapping & Positioning
2. Port Mapping
3. Data Mapping
Integration Instructions for CI
AUTOSAR Continuous Integration
21
SWC A SWC B
SWC C
1. Via Direct Integration Instructions
Explicit Runnable to Task Mapping
2. Via Abstract Integration Instructions
Describing only in abstract way> Activation Period
> Safety Level
Mapping via matching properties
3. Heuristic Based
Using AUTOSAR model information> Runnable Trigger
> Safety Level
> Data Dependencies
> Resource Needs
Matching to Task definition via solving algorithm
ASW Integration Instruction for CI – Runnable-to-Task Mapping
AUTOSAR Continuous Integration
R1
R2
R4
R5
R3
Trigger: 10ms
Safety:QM
Trigger: init
Safety: ASIL AResource: Stacksize/Runtime
Execution Order
Task 1 Task 2 Task 3
OS-Application:QM OS-Application:ASIL A
Task 4
22
Task 1
1. Via Direct Integration Instructions
Defining Execution Sequence of all Runnables
2. Via formal Constraints
Using Execution Order Constraints or Data Age Constraints for describing time critical dependencies between Runnables
3. Via Heuristics (Auto-Map)
Analyzing dependencies between Runnables and minimizing backward references
ASW Integration Instruction for CI – Runnable Positioning
AUTOSAR Continuous Integration
R-A
R-B
R-C
R-D
R-E
R-(to be mapped) ?
?
23
ASW Integration Instruction for CI – Port Mapping
AUTOSAR Continuous Integration
SWC A SWC B
SWC C
1. Direct mapping via Integration Instructions
Mapping SWC port to > Ports of other SWC
> Service ports (e.g. Trigger Ports, …)
Non-existing SWCs and Ports in integrated App Packages can be stubbed
2. Via Heuristics (Auto-Map)
Matching of Port Names
24
Using DaVinci Configurator Pro.CI in Agile Development Process
Project Application & IT Integration
Preparation of Root Config
Application Software Development
Updates of ECU Extract
Build System
RC-v1.0 RC-v1.1 RC-v1.2
Sprint A Sprint B
ECU Extract Updates get integrated in Root Configuration at dedicated dates (e.g. till sprint start)
App Developers work during sprint on prepared Root Config (Baseline)
In case of urgent ECU Extract Updates, a manual Baseline Update is required
Commit of App Package to Build Server produces Executable, based on specified Baseline
CI
ExecutablesE E E E E E E E E E
ECU Ext
ECU Ext
ECU Ext
ECU Ext
ECU Ext
ECU Ext
ECU Ext
25
Result 3 Types of Builds (offered by Option .CI)
1. Virtualization on VFB Level (vVIRTUALtarget Pro)
For testing ASW functions standalone (e.g. for Unit-Tests)
2. Virtualization on BSW Level (vVIRTUALtarget Basic)
For testing ASW functions in combination with BSW (e.g. for Integration-Test with BSW)
3. Hardware
For functional HIL tests
For trace measurements (see later)
Pipeline & Testing
AUTOSAR Continuous Integration
Initialize Integrate Generate Build Check
26
Introduction
Applying Cx to AUTOSAR Development
AUTOSAR Continuous Integration
AUTOSAR Continuous Delivery
Summary & Outlook
Agenda
27
For Developers & Testers
Validate all functional and most non-functional properties automatically (feedback on malfunction)
Increasing confidence in committed software
For Project Architects
Be sure all system and function specific timing requirements have been met
Getting early notice of potential future resource exceeds
For Project Manager
Be sure that all resource restrictions have been met
Making software directly deliverable
Objectives for Continuous Delivery
AUTOSAR Continuous Delivery
Quality
Confidence
Fast Function Prototyping
28
Formalize Performance & Timing Requirements
For executing non-functional tests automatically
Perform Model-based Timing & Performance Analysis
For having fast feedback and being not limited to physical resources during tests
Major Concepts for AUTOSAR Continuous Delivery (CD)
AUTOSAR Continuous Delivery
29
Composition 1
SWC A
SWC B
Application Software Requirements
e.g. utilization boundaries for Compositions, Software Components or Runnables
Hardware Requirements
e.g. maximum core utilization
CPU Resource Requirements
AUTOSAR Continuous Delivery
R1
R2
R1
CPU: 30%
CPU: 20%
CPU: 10%
CPU: 10%
CPU: 5%
CPU: 15%
Max CPU Core Load
Core 1 Core 2 Core 3
80% 80%50%
30
Application Software Requirements
e.g. maximum execution time for Runnables
e.g. data ages for communication between Runnables
Operating System Service Requirements
e.g. Task response times
e.g. Start-to-Start durations
System Requirements
e.g. Event-Chains on critical dataflow / End-to-End times
Timing Requirements
AUTOSAR Continuous Delivery
31
In contrast to normal (non CD) testing …
Testing should cover many different Scenarios
e.g. Burst-modes with many service-requests
e.g. increased load scenario, when function amount increases
e.g. different execution modes
Fast test execution is required
Tests should run in hours (max. a day) instead of weeks
Simulation allows fast exploration
Objectives for Testing
AUTOSAR Continuous Delivery
32
1. Model-based Simulation
+ Not limited to hardware, therefore theoretical unlimited parallelization
+ For Stress-tests, system stimulation scenarios can be arbitrary extended in order to cover many situations
- Requires execution times for Runnables
2. Hardware Trace Measurements
+ Allows detailed analysis of error scenarios on unlimited level of granularity
+ Execution times can be measured
- Limited through physical hardware
Types of Timing & Performance Validation
AUTOSAR Continuous Delivery
33
Model-based SimulationModel-based
SimulationModel-based Simulation
Procedure
AUTOSAR Continuous Delivery
Hardware Measurement
Model-based SimulationModel-based
SimulationModel-based Simulation
Build
System Description
ECU-C
Test Report
Runnable Execution Times
Timing Report
(alternative) Supplement Runnable Execution Times to SysDescr
Parametrized random parameter generator
34
Hardware Measurement Overview
AUTOSAR Continuous Delivery
1. Configure MICROSAR Tracing Hooks (OS and RTE)
2. Configure iSYSTEM
3. Trace the ECU and export a BTF
4. Evaluate the Traces against Requirements
TA Tool Suite
TA.Inspection
Extraction of aScheduling Trace
BTF Scheduling Trace Import
Timing Report Generation
MICROSAR
winIDEA
iSYSTEM
Requirements InputiSYSTEM Profiler XMLfor OS Awareness
1.
2.
3. 4.
35
Example of Timing Report
AUTOSAR Continuous Delivery
Showing measurement, tool and input file information
Validating system requirements (meet/failed)
Showing further timing metrics for later evaluation
Inputs such as Task Stack Consumptioncan also be stored
36
Junit Reports for automated testing
Publish direct on the CI Server
Combination with other testing results in one overview
JUnit Reports and Continuous Timing Evaluation
AUTOSAR Continuous Delivery
Showing CPU Load average, as well as minimum and maximum value (in a x ms timeframe)
Extrapolation shows CPU Limit will potentially be reached within two releases
Performance improvements should be applied now, in order to prevent a blocking of delivery
37
Introduction
Applying Cx to AUTOSAR Development
AUTOSAR Continuous Integration
AUTOSAR Continuous Delivery
Summary & Outlook
Agenda
38
Summary & Outlook
Summary & Outlook
Conclusion
CI brings already a lot of benefits for the agile software development, especially for developers and integrators
Decision Frontloading enables fast integration
Independent local integration solves blocking
CD makes sense, when software is often released (e.g. sprint based, ~quarter based)
Automatic non-functional testing, e.g. timing or performance is essential
What’s Next?
Standardization of Timing Hardware Measurement (AUTOSAR ARTI)
Over-the-Air Updates (OTA)
39 © 2020. Vector Informatik GmbH. All rights reserved. Any distribution or copying is subject to prior written approval by Vector. V0.3 | 2020-06-08
Author:Deubzer, MichaelVector Germany
For more information about Vectorand our products please visit
www.vector.com