RCE Platform Technology (RPT) Sergio Maldonado ([email protected]) Software Architecture Development Environment.

RCE Platform Technology (RPT)

Sergio Maldonado ([email protected])

Software ArchitectureDevelopment Environment

2

RCE Platform Technology – Software Overview

Topics Covered:

• Embedded System Software- Runtime Environment: The Core

- Service Libraries

- System Configuration

• Software Development- SDK

- Development Model

• Protocol Plugin Software- Socket Abstraction Services (SAS)

3

RCE Embedded System Software

The RCE system software consists

of:

• Bootloaders- Xilinx FSBL

- U-Boot

• RTEMS operating system- Zynq board support package

- OS primitives: semaphores,

message queues, timers, tasks, etc

- POSIX support, FAT filesystem

• Kernel extensions and

enhancements- Provided by the RPT group

• Linux Operating system- Filesystem administration and

system maintenance

Core

Services

Applications

Bootloaders

4

RCE Embedded System – Core and Services

Applications

Boot Loaders

Services

Core

NFS Telnet Shell RTS DSL ATCA

Network 1/10 GE drivers SAS

Kernel

SD & UART drivers Memory & task mgmt Dynamic linker/loader SVT

Console

5

RCE Runtime Core

The Core is the set of software that provides the base level

of runtime operation

• RTEMS kernel

• C runtime libraries

• RPT additions to kernel functionality- SD device driver

- UART device driver

- Dynamic linker/loader

- Memory management utilities

- Task utilities

- Configuration using Symbol/Value tables

- Filesystem abstraction: Namespaces

6

RCE Runtime Services

A set of services are built on top of the Core to provide:

• Embedded console

• Shell and extensions

• 1-GE or 10-GE network and TCP/IP stack

• C++ runtime library

• Telnet

• NFS

• Distributed Services Lookup (DSL)

• ATCA Services

• Application Initialization

Shareable Images

8

Shareable Images

The RCE runtime environment operates by executing code

located in ELF formatted, dynamically loaded/linked

shareable objects. The internal format of these ELF objects

has been crafted specifically for the RCE runtime

environment. These shareable images are separated into

three categories.

• Shared Library - implementation

• Executable - tasks

• Symbol/Value Table - configuration

9

Shareable Images - Shared Library

• Implementations for general consumption by other shared

images

• Not associated with a specific task

• Define an initialization entry point- Executed by linker prior to memory fixup

- Used for initialization of library prior to use by other shared images.

• Can be installed as memory resident- Memory resident libraries are loaded once but may be linked

against by many other shareable images.

• Can be linked against any other shared library

• Use the .so filename extension

10

Shareable Images - Executable

• Defined as a pure RTEMS task along with its corresponding

dynamically loaded/linked executable code.

• Must satisfy certain Core requirements- Task entry hook – Task_Start

- Task exit hook – Task_Rundown

- Task specifications (name, priority, etc) are defined in a data

structure, which may be located in SVT tables. – Task_Attributes

• Can link against any shared library and may utilize any

installed SVT.

• Use the .exe filename extension

11

Shareable Images - Symbol Value Table

• Data only files containing values (data structures) mapped to

symbol names (character string).

• Storage of system and application configuration data

structures

• Do not link against shared libraries

• Use the .svt filename extension

12

Shareable Images – Linker and Loader

The dynamic linker/loader is a fundamental component of the

Core.

• Locates image files on the target filesystem

• Loads image segments into appropriate memory regions

• Relocates and fixes up symbols

• Dynamically resolves library dependencies- Recursively loads and links all required images

13

Shareable Images - Benefits

The usage of shareable images provides the maximum

flexibility for development and maintenance of both system

and application software.

• Code can be modularized to encapsulate common functionality

• Software interfaces can be extended and deployed without

requiring a full recompilation of code bases

• The Core and all system service libraries can be developed

and maintained independent of deployed application code

• Development of applications can be more easily distributed

amongst several functional groups.

14

Shareable Images – Namespaces (I)

The dynamic loader locates images by employing the binding of a logical

name, or namespace, to a specific directory on the target filesystem.

• A namespace is a string identifier used to associate a set of target images

with some user defined category or organizational structure

• The namespace string must- be unique over the set of all assigned namespaces in a running system.

- not contain the reserved character ":”

• The namespace gets mapped at runtime to a path on the target filesystem.- This mapping may be stored in a configuration file

• The set of images associated with a namespace must all reside in the

same directory on the target filesystem

• All dynamic images must specify a namespace and filename at link time

using the -soname linker flag

• The format of the argument to the -soname linker flag must be: - "<namespace>:<image_filename>”

15

Shareable Images – Namespaces (II)

The benefits of using namespaces include:

• Separation of software images into logical functional groups.

• Image portability throughout the filesystem

• Execution of images over a network filesystem (NFS)

• Deployment of images to the target filesystem need not take

place until the test and debug cycle is complete.

16

Shareable Images – Namespaces (III)

User-DefinedNamespace(s)

ConfigNamespace

SystemNamespace

/rtemsapp. user.so. user.exe. user.svt..../config..../rtems

sys.svt app.svt

rtems.so dsl.sorts.so xaui.soshell.so sas.so

These files have shared ownership by Applications and System

These files are System owned

These types of files are owned by Applications

Core Utilities

18

Core Utilities - Memory Management

The Core provides the following memory management features

• Separate memory regions each with a unique set of MMU

attributes- Read/write/execute

- 4k or 1Mbyte page size

- Cacheability

• Allocations from managed regions

• Region address lookup by name

• Cache operations- Flush

- Invalidate

• Atomic Resource Management- Allocate memory resources reentrantly

19

Core Utilities - Task Management

• A Core Task is defined as a pure RTEMS task along with its

corresponding dynamically loaded/linked executable code.

• The Core provides the infrastructure to dynamically load and

link executable shared images using namespaces- This occurs before its corresponding RTEMS task is started

• Task Control- Start

- Stop

20

Core Utilities - Symbol Value Tables

• Loading and installation of SVT shared images

• Symbol translation- Resolves the object pointer corresponding to the symbol

• System and application SVTs are loaded at Core initialization- These SVTs are available for immediate use by applications and

do not need to be explicitly installed.

21

Core Utilities - Tools

The Core provides additional low level utilities for general

use:

• Linked lists

• Key value tables

• Byteswap routines

• ELF image utilities

• Hash tables

• System logging

System Libraries

23

System Libraries - Runtime Support

The runtime services library, rts.so, provides full C++ support

and the following features

• System message logging classes

• Exception classes

• String utilities

24

System Libraries – Distributed Services Lookup

Distributed Services Library, or DSL (dsl.so) represents a

lightweight client server model

• Resolves a service name string to a shareable image

• Lazy deferral of image loading

• Host side and embedded interfaces

• Host side client message is delivered over ethernet

• Server executes entry point in registered service library, with

the message as an argument

• Service processes the message and returns a response to

the Server

• The Server delivers the service specific response to the client

25

System Libraries - ATCA

The ATCA library (atca.so) is a registered DSL service.

• Server side component “looks up” networking information

corresponding to an ATCA location- IP address

- MAC address

• Host side client interface allows for building of requests and

processing of server side responses

• Requests are broadcast to all servers on a given subnet.

• Only the node that has the matching ATCA address will

respond to the request.

Configuration

27

System Configuration - SVT

Components of the Core and system services are configured at

startup using the SVT mechanism (sys.svt). The system

configuration is owned by the RPT and is suitable for most

deployments.

• System task specifications- Console

- DSL

• Network stack configuration

• Telnet configuration

• System DSL definitions (currently only ATCA)

28

Application Configuration - SVT

Activation of system services is configured at startup using the SVT

mechanism (app.svt). This is considered to be jointly managed by the

system and applications. Applications may tailor the service set to its

needs, but the svt filename and location is fixed and must be present

in order for the system to successfully boot.

• Services to start at boot- Console

- Shell

- Network Driver

- NFS

- DSL

• Activation of DSL services- ATCA service is required by the system

• Activation of user application specific image- appinit.so

29

Application Initialization

At the completion of starting up system services, application

code can then be conditionally loaded. The supported method

is to create a dynamic image named appinit.so, install it in the

rtemsapp partition, and add it to the services portion of app.svt.

The appinit.so image can be tailored to the needs of the

applications.

• Define application specific namespaces

• Mounting of scratch filesystem

• Mounting of NFS directories

• Loading and activation of application executable images

• Any other required initialization

RCE Software Development Kit (SDK)

31

RCE Software Development Kit - SDK

The SDK contains host side utilities to interact with a running

RCE Core, but its primary focus is to build shareable images

for test and deployment in embedded execution environments.

Execution environments are the pairing of an operation system

with a hardware platform. Here, these are referred to as the

target (ARM RTEMS RCE) and host (I86 Linux 32/64).

• C/C++ code must be cross compiled on the host for

execution on the target.

• C/C++ code may be compiled natively for execution on the

host.

32

SDK Development Model

Both the SDK and the Core runtime environment contain

support for developing software with speed and agility. The

phases presented here were employed by the RPT group in

the development of RCE system images, and constitute our

recommended methodology.

• Shareable image implementation

• Test and debug

• Image deployment and installation

33

RCE Software Development Kit - SDK

Host System Target System

Filesystem

SDK1) Compile source files against

SDK include files (produces *.o)

2) Link object code against SDK libraries (produces *.so, *.exe, or *.svt)

Filesystemscp

(Linux)

orNFS mount

.so

.exe

.svt

.so

.exe

.svt

Runtime Core• Runtime Core provides the

dynamic linker/loader to activate shareable images from the target filesystem

34

SDK Development Model – Implementation

• Develop source code, compile, and link shareable images.

• Takes place on a supported Linux host machine

• Software can first be prototyped using native host compilers

• Allows images to be partially tested and debugged prior to

cross compilation (algorithmic routines, data structures).

• Creation of application specific .exe .so and .svt images

• Images can be examined for size constraints and machine

code optimization using standard ELF utilities.

35

SDK Development Model – Test and Debug

• Cross compiled images are executed on the target using NFS

mounted directories mapped to namespaces

• Library loading and executable activation using shell utilities

• Debugging using the console or telnet shell

• Application images are not required to reside on the target

filesystem

• Does not affect deployed software.

36

SDK Development Model – Deployment

• At this phase, images are fully tested and ready to be deployed

to the target filesystem.

• Configuration and initialization components on the deployed

system are updated to support activation of new images from

the target resident filesystem- SVT configuration files and/or application specific initialization

libraries must be updated to map namespaces, task start directives,

and any explicit library loads required for the new images.

• Images are copied to the appropriate directories on the target

filesystem

• Images are activated at startup, after system services are

available.

37

RCE Software Development Kit - Contents

• RTEMS SDK- RTEMS cross compilers

- Core and service libraries to link code against

- Header files

- Bootloaders

- Firmware bitfiles

• I86 Host Linux SDK- Host development header files and libraries

- Administration and maintenance tools

• ArchLinux SDK- Utilizes Xilinx tools

- Cross-development for execution on an embedded linux platform

- Work in progress

38

RCE Software Development Kit

• Must be integrated into existing build systems- Compilers and linkers sourced from SDK installation path

- Include files - use –I on the compiler command line

- Library paths - use –L: or –l linker command line

• A single SDK installation may be shared by one or more

developers

39

RTEMS SDK Compilers and Linkers

The SDKs make use of the RTEMS cross-compiler tools and

native gcc installations. SDK compilers and linkers are

essentially wrappers around these tools. The wrappers contain

the command line arguments required to support the dynamic

image format used by the Core runtime. User specific

arguments are appended to the default SDK arguments.

• SDK RTEMS Compilers- rtems-gcc, compile C object code

- rtems-g++, compile C++ object code

• SDK RTEMS Linkers- rtems-ld, link object code into shareable images (.so)

- rtems-task, link object code into task images (.exe)

- rtems-svt, link object code into symbol value tables (.svt)

40

I86 Linux Host SDK Compilers and Linkers

Host executables and libraries can be compiled using an SDK.

In this case, the SDK commands are merely indirections to the

native gnu tools.

• Host Linux SDK Compilers- linux-gcc

- linux-g++

• Host Linux SDK Linkers- linux-ld

Protocol Plugin Software

42

Protocol Plugin Support – SAS (I)

Support for the software side of the protocol plugin socket is handled

by the Socket Abstraction Services (SAS). The implementation is

provided by the service library sas.so.

SAS interacts with plug-in firmware by reading and writing AXI

memory mapped I/O addresses.

SAS exposes plug-in functionality using mailboxes. A mailbox is a

container, implemented in firmware which obeys FIFO (first-in, first-

out) discipline. A mailbox contains file descriptors or messages.

• SAS supports three type of mailboxes.- Inbound mailboxes

- Outbound mailboxes

- Unnamed mailboxes

43

Protocol Plugin Support – SAS (II)

…

Task

Ob Mbx

Ib Mbx

Mbx 0

Mbx 31

Session

Plug-in Port

FIFO

FIFO

FIFO

FIFO

SAS

44

Protocol Plugin Support – SAS (III)

SAS uses sessions to wait for input from mailboxes.

• A session is the association of software with a socket.

• Mailboxes are bound to sessions at runtime.

The types of mailbox input are:

• Arrival of an inbound frame- 32-bit SAS file descriptor

• Notification of placement of an inbound frame- 32-bit SAS message

• Notification of placement of an outbound frame- 32-bit SAS message

Waiting on input from a session is always done in the context of a task. A session is

associated with a task.

• Processing of inbound frames use two sessions1. Wait for the arrival of an inbound frame descriptor.

- Also used to post a file descriptor to initiate placement of inbound frame payload

2. Wait for the completion of placement of an inbound frame

• Transmission of an outbound frame uses one session:1. Wait for the completion of placement of the outbound frame

Thank You. Questions?

Backup Slides

47

RCE Software – SD Storage

All software images reside on an SD flash. The 32 GB (27.9 usable) microSD is formatted

with 6 partitions

• Boot - /boot- Bootloader files boot.bin, fpga.bit, uboot.env

- 512 MB FAT32

• RTEMS system - /rtems- The Core, service libraries, system executables

- 512 MB FAT32

• RTEMS application - /rtemsapp- Application specific images

- System/application shared configuration

- 4GB FAT32

• user/application scratch - /scratch- 16 GB FAT32

• Linux Kernel - /linux- System maintenance/recovery

- 512 MB FAT32

• ArchLinux Filesystem - /archlinux- System maintenance/recovery

- 8 GB EXT3

48

RCE Bootstrap Process

• Xilinx first stage bootloader (FSBL)- SOC initialization

- CPU register configuration

• U-Boot- Xilinx distribution includes Zynq hardware support

- Firmware bitfile loader

- BSI handshake – IPMI ready signal

- Networking parameter configuration

- Rescue operations

• Core bootstrap- Board support package, MMU, and kernel initialization

- Relocation/fixup of Core elf image

- Kernel initialization - RTEMS

- Memory and driver initialization

- Loading of system and application SVTs

- System service load and startup

- Application load and startup (optional)

49

Protocol Plugin Support – SAS Configuration

SAS configuration for protocol plugins is defined in an SVT file.

• Inbound/outbound header sizes

• Inbound/outbound buffer count

• Plugin name

• Memory map for plugin register space

50

Protocol Plugin Support – Firmware Registers

Most plugin firmware will expose a set of configuration

registers. This register block is typically mapped to memory

locations in the AXI space.

• SAS interface to configure the AXI register address mapping

• SAS interface to retrieve a pointer to the memory mapped

register block.

51

Protocol Plugin Support - XAUI

Plugin software has been implemented using SAS to support

the operation of the XAUI plugin. This plugin is used by system

software to enable 10-GE TCP/IP stack. The plugin also

supports transmit and receive by other applications that need

to take advantage of the high speed link using other protocols.

• Data receive enabled by registering a protocol callback

routine with the XAUI driver.

• Data transmit is performed by directly posting to the XAUI

SAS outbound mailbox.