BUD17-202: AOSP Toolchains

AOSP ToolchainsBernhard “Bero” Rosenkränzer

ENGINEERS AND DEVICESWORKING TOGETHER

Overview of current work● Making AOSP master userspace build with gcc (6.3 and pre-7) again

○ Still building gcc based AOSP toolchains from TCWG’s releases every month○ Early testing with gcc 7 snapshots -- gcc 7 release expected soon

● Making the kernel build with Clang (4.0)○ Target kernels: 4.4-HiKey, 4.9-HiKey, Mainline

● CI efforts: Testing AOSP daily builds with clang daily builds● To be done: Investigate use of the LLD linker

ENGINEERS AND DEVICESWORKING TOGETHER

Reversing the world: Building AOSP with gcc, kernel with clang● Why?

○ Different compilers show different warnings - both can be useful○ Can’t do meaningful comparison of compilers if there’s only one option

ENGINEERS AND DEVICESWORKING TOGETHER

Kernel with clang status● Done for HiKey 4.4, HiKey 4.9 and Mainline post-4.10 kernels● HiKey works, no known problems that matter right now

○ Compile issues in ARMv8.1-a specific code (for which the hardware doesn’t exist yet): Makefiles force a number of gcc specific compiler flags that don’t have a direct equivalent in clang (and clang developers oppose adding them).

● Mainline kernel fails to boot during early-bootup on some x86 hardware when built with clang

● We’re down to 26 needed patches, 19 of which are relevant for Linaro work (3 are in x86 code, 4 in MIPS code)

ENGINEERS AND DEVICESWORKING TOGETHER

Interesting problems getting the kernel to build with clang● Clang is much more picky about inline assembly

-fno-integrated-as helps a lot, but even when not using the integrated as, clang runs syntax checks.This breaks the kernel’s abuse of inline assembly to generate asm offset tables (see include/linux/kbuild.h, scripts/mod/Makefile -- use of asm statements to generate files that will be processed with sed rather than as) .

● Solution: Still use inline assembly to generate the offset tables - but make the output file valid (even if unused) assembly code: It becomes a file full of asm comments.

● Clang is also more picky about using the right constraints and size-specific instructions (e.g. mulq as opposed to plain mul on x86)

ENGINEERS AND DEVICESWORKING TOGETHER

Interesting problems getting the kernel to build with clang● Reliance on gcc extensions in some functions:

○ Nested functions○ Variable-length arrays in structs○ __attribute__((externally_visible))

○ Removal of references to functions called from unreachable code

● EFI libstub: Mix of code that must be PIC and code that must not be PIC○ It’s not yet 100% clear whether the code works with gcc by accident or clang is doing

something wrong, but it’s looking more like it works with gcc by accident and a future gcc version may expose the same problem.

● gcc plugins for code analysis (obviously) don’t work with clang - implementing equivalent plugins will probably not be too difficult, but will such an effort be accepted upstream?

ENGINEERS AND DEVICESWORKING TOGETHER

Interesting problems getting the kernel to build with clang● The kernel is still C89 code (and enforcing -std=gnu89)

○ Clang supports C89, but has defaulted to C99 and newer for a long time. C89 support in clang has not received the same amount of testing as C99 or C11 (or C++14) mode, and may result in obscure bugs or missing optimizations

○ Moving the kernel to C11 (or at least C99) may be a good idea even when not thinking about clang (a lot of kernel code already uses C99 initializers and other C99 functions that are available in gnu89 as extensions to C89)

○ gcc no longer defaults to C89 either

ENGINEERS AND DEVICESWORKING TOGETHER

AOSP with gcc status● AOSP master built with gcc 6.3 in January, but upstream changes make new

patches necessary.● ABI problem: gcc -std=gnu++14 emits calls to __cxa_throw_bad_array,

which doesn’t exist in compiler-rt (upstream bug 25022512) -- but code has moved on to require C++14 language features

● gcc 7 is still a moving target and used to error out a lot, but it’s getting ready. Not many additional changes to AOSP needed.

ENGINEERS AND DEVICESWORKING TOGETHER

Interesting problems getting AOSP to build with gcc● __attribute__((unused)) is for functions only in gcc -- in clang, it can

also be used for struct members.○ (may make sense to implement this in gcc)

● gcc warns (or errors out with -Werror) when doing an extra check for a pointer being NULL when passing a parameter declared __attribute__((nonnull))

ENGINEERS AND DEVICESWORKING TOGETHER

Interesting problems getting AOSP to build with gcc● gcc now warns about bogus indentation (error with -Werror):

for (i=0;i<=jk;i++) {for(j=0,fw=0.0;j<=jx;j++) fw += x[j]*f[jx+i-j]; q[i] = fw;

}○ This may be nice to have in clang…

● gcc is more picky about potentially reaching the end of a non-void function without returning a value

● On the other hand, only clang thinks there’s something wrong withvoid doSomething(char a[10]) { if(strncmp(a, “1234567890”, sizeof(a))) something();}

ENGINEERS AND DEVICESWORKING TOGETHER

Current CI efforts● Nightly builds of clang master● On success, nightly builds of AOSP master with the clang master build

○ Need to apply a minor patch set to AOSP master: mostly about tuning compiler flags and bypassing newly added error diagnostics

● On success, boot-up test on LAVA Hikey● Building gcc based AOSP toolchains (with regular binutils updates) with every

TCWG gcc release

ENGINEERS AND DEVICESWORKING TOGETHER

Future plan: Building AOSP on aarch64● aarch64 machines are getting strong enough to replace x86 machines● Step 1: Build AOSP on regular Linux aarch64 hosts● Step 2: Build AOSP on AOSP -- why put all those 8+ cores on modern Android

devices to waste? Devices like Pixel C could be a good native development tool, and a startup in France seems to be working on a “plug your phone into this docking station to make it your desktop” type device.

ENGINEERS AND DEVICESWORKING TOGETHER

Future plan: LLD● LLD is a new linker coming out of the LLVM project, replacing the BFD linker

and the gold linker from traditional binutils● LLVM 4.0 comes with the first version of LLD that is usable

○ Can be used to build most components of a Linux system○ Patchset to make the kernel build with LLD is available, but needs some more work

● LLD should still be considered experimental, but may well be ready in time for O based releases○ Currently missing a couple of --fix-cortex-a*-* workarounds for processor errata known to affect

AOSP under some conditions

Questions?Ask now, or mail [email protected]

#BUD17For further information: www.linaro.org

BUD17 keynotes and videos on: connect.linaro.org

Thank You#BUD17

For further information: www.linaro.orgBUD17 keynotes and videos on: connect.linaro.org