Embedded software validation best practices with NI and RQM

Embedded Software Validation Best Practices with National InstrumentsKyle PerkuhnPlatform Marketing Engineer: Real-Time Test National [email protected]

Paul UrbanBusiness Development Manager IBM [email protected]

Scott FairbrotherSenior Software EngineerISV Technical EnablementRational Ready for Rational Software [email protected]

10-20 defects produced per 1,000 lines of code*10-20 defects produced per 1,000 lines of code*

6.5 Mil6.5 Mil

Commercial Aircraft

10 Mil10 Mil

Luxury Automobile

100k100k

Smart Washing Machine

1k – 2k1k – 2k 65k - 130k 65k - 130k 100k – 200k100k – 200kDefects

Lines of Code

* The Economists.May 16th, 2010. Tech.View: Cars and software bugs. www.economist.com

Embedded Software Quality Challenge

10-20 defects produced per 1,000 lines of code*10-20 defects produced per 1,000 lines of code*

6.5 Mil6.5 Mil

Commercial Aircraft

10 Mil10 Mil

Luxury Automobile

100k100k

Smart Washing Machine

1k – 2k1k – 2k 65k - 130k 65k - 130k 100k – 200k100k – 200kDefects

Lines of Code

* The Economists.May 16th, 2010. Tech.View: Cars and software bugs. www.economist.com

Embedded Software Quality Challenge

How can you ensure that a possiblemalfunction will not harm anyone?

A “safety” approach is needed

Industry safety standards examplesAvionics/aerospace

DO-178B/C / ED-12B (RTCA/EUROCAE)DO-254

MedicalFDA 510(k), IEC 62304, IEC 60601

Automotive ISO 26262

Functional safety in process industryIEC 61508

Nuclear power plants IEC 60880

Railway SystemsEN50128 and EN50129

4

5

Standards often initially increase project costs Example: DO-178B

+25-40%

+60 – 100%

Common issues

Inadequate formal plans or not following them

Inadequate level of detail and process for Requirements

Inadequate or non-automated Requirements Mgmt and Traceability Mgmt

Lack of automated testing

Improper Tool Qualification (too much or too little)

Weak process and checklist management

6

IBM helps you turn product development into a competitive advantage through our Rational Systems Engineering approach

• Improve systems engineering to tackle growing product complexity

• Improve software development to deliver innovation faster

• With an open, integrated systems approach that enables access to all engineering information across the engineering disciplines.

7

ECU Design/Devvia AUTOSAR

Functional Safetyfor Automotivevia ISO 26262

Functional Safetyfor Aerospacevia DO-178B

Functional Safetyfor Medical Devices

via IEC 62304 (future)

Defense Architecturesvia DoDAF

F u n c t i o n a l S a f e t y

Automotive

HW-SW Co-Design

Aerospace Electronics

Focused on Industry safety standards

IBM Automotive Solution for ISO 26262 Content

DashboardsPractice library

Auto generation of practice work items

Starting templates

Tool mentors

Introduction to National Instruments

Diversity of Applications

No Industry >15% of Revenue in 2012

SemiconductorsEnergy

Big Physics

Life Sciences

Consumer Electronics

Mobile Devices

AutomotiveAdvanced Research

Defense/Aerospace

Academic

NI CompactRIOPXI PC NI CompactDAQ

Graphical Programming

Analysis and Reporting

Test Automation

Real-Time Testing

Enterprise Connectivity

Tools for Next Generation Development

Modeling

Multi-Chassis Systems

NI TestStand—Test Management Software

• Graphical sequence editor environment

• Automate tests written in any language

• Multithreaded sequence execution

• ASCII, HTML/Web, XML, and ATML report generation

• Access, Oracle, SQL Server database connectivity

Connecting Design and Test

© 2013 IBM Corporation 04/07/23

Accelerating Product and Service Innovation

Execution Records, Defect Management, Test Plan Documentation

Quality EngineerQuality EngineerPlan tests and link to requirements

Schedule tests

Submit defect

Execute tests

Implement test, analysis, and reporting

Generate results

Test EngineerTest Engineer

Manually Manage Results

© 2013 IBM Corporation 04/07/23

Accelerating Product and Service Innovation

National Instruments Test Components (.seq, .vi, .nivs, …)

Requirements Documents (DOORS, .docx, .xlsx, …)

Execution Records, Defect Management, Test Plan Documentation

Test Results(.html, .pdf, .tdms, .jpg, …)

© 2013 IBM Corporation 04/07/23

Accelerating Product and Service Innovation

National Instruments Test Components (.seq, .vi, .nivs, …)

Requirements Documents (DOORS, .docx, .xlsx, …)

Execution Records, Defect Management, Test Plan Documentation

Test Results(.html, .pdf, .tdms, .jpg, …)

National Instruments

Test Integration Adapter

for IBM Rational

Quality Manager

© 2013 IBM Corporation 04/07/23

Accelerating Product and Service Innovation

Next Generation Development ProcessSystems Systems EngineerEngineer

DevelopmentDevelopment Quality Quality EngineerEngineer

DOORS Rhapsody Team Concert

Quality Manager

Project Project ManagerManager

Modify requirements and evaluate impact

System Design & Development

Submit defectAssess progress

Create change request

Derive system requirements

Plan tests and link to requirements

Test EngineerTest Engineer

Schedule tests Execute tests

Implement test, analysis, and reporting

Generate results

NI LabVIEW | VeriStandNI TestStand NI

DIAdem

Test Components

Models Analysis Reports

User Interfaces Stimulus Requirements

Consistency and Efficiency throughout the Process

Data Loggin

g Stimulus

Profile

Requirements

Physical TestingDesign

Deployment

Prototyping HIL Validation

Increasing Development Quality and Efficiency with Test Component Re-use

Re-use of test components reduces cost and risk by enabling correlation of test procedures and results across all development phases

Continuity of test components reduces test application development and maintenance costs

Data Loggin

g Stimulus

Profile

Requirements

Data Loggi

ngStimul

us Profile

Data Loggin

g Stimulus

Profile

Data Loggin

g Stimulus

Profile

Physical TestingDesign

Deployment

Prototyping HIL Validation

Data Loggin

g Stimulus

Profile

Test Asset Management and Automation

IBM Rational Quality Manager

MIL Test System #1

MIL Test System #2

MIL Test System #3

HIL Test System #1

HIL Test System #2

HIL Test System #3

Integration System #1

Integration System #2

Integration System #3

Test Cell System #1

Test Cell System #2

Test Cell System #3

Field Test System #1

Field Test System #2

Field Test System #3

Busy Available Offline

IBM Rational Quality Manager

Defect Results and Verification Items

Defect Resolution Collaboration

Defect Management

Project Dashboards

TestResults

IBM Rational Quality Manager

Project Documentation and Tracking

• Traceability all the way to test and back• Collaboration between ALL teams, including test• All test results available to all teams and linked to test cases and requirements• Quality and test considered from the outset, not an afterthought, reducing cost and risk of identifying and correcting defects• Test component re-use throughout project phases and between projects providing operational efficiency and accuracy

IBM Rational and National Instruments are teaming together to provide an end-to-end quality management solution

Learn More: Web pages:

IBM.com: IBM Rational and National InstrumentsNI.com: Integrating Test Into Development CycleNI Test Integration Adapter Product page

References:Solution Brief: Break down engineering silos to improve qualityArticle: To Build a Smarter Product, Stop Separating Design & Test Ian Cannings, Danfoss Power ElectronicsBlog: Design News blog

Media:Webinar: Integrating National Instruments testing with RQM (demo starts at min 24) Webcast:: Embedded System Quality: Top Competitive Techniques Revealed Video: Interview Innovate 2013- Chris Washington, National Instruments...Video: Innovate 2013 Interview and Demo at NI BoothVideo: NI Week 2012 Keynote: Embedded Software Quality ManagementWebcast: Taming the Complexities of Software Driven Innovation to Reduce Project Cost and RiskPodcast: Integrate testing into the development lifecycle

© Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.

www.ibm.com/software/rational

Backup

DOORS Template for ISO 26262• Capture Severity, Probability and

Controllability attributes• Automatically determines ASIL

• Working on matching these attributes to attributes in Rhapsody

• Developing requirements module structure to capture relationships between

• Stakeholder (Item Definition) Requirements

• Functional Safety Requirements• Technical Safety Requirements• System Safety Requirements• HW and SW safety requirements

• Automatic propagation through Safety Requirement Hierarchy of ASIL

• Delivered as a DOORS project archive

30

Rhapsody Kit for ISO 26262 and IEC 61508• Overview Doc: describes the contents of the Rhapsody kit

• Rhapsody Reference workflow : provides an exemplary workflow for modelling, code generation and verification in safety critical

• Rhapsody TestConductor Add On Workflow: describes testing activities and objectives

• Rhapsody TestConductor Safety Manual: provides additional information for using TestConductor in safety related applications

• TÜV SÜD Certificate for Rhapsody TestConductor Add On

• TÜV SÜD Report on Certificate for ISO 26262 and IEC 61508

• Rhapsody TestConductor Add On Validation Suite: separately available test suite for Rhapsody TestConductor to help in qualification efforts

• Certification kits for the SXF (C++) and SMXF (C) frameworks