Mirko Damiani
How to build a minimalGNU/Linux embedded system
. . . from scratch!
Better Embedded ConferenceFirenze, Sept. 24-25, 2012
2 / 35
Mirko Damiani – [email protected]
Terms of Interest● GNU/Linux
● GNU software● Linux kernel
● Minimal● Size and number of utilities● Complexity of scripts
● Embedded system● Custom boards● Low resources
3 / 35
Mirko Damiani – [email protected]
Prerequisites● Entry level
● GNU/Linux kernel● Hardware schematics● Operating systems● Computer architectures
● Intermediate level● GNU/Linux utilities● SW development
4 / 35
Mirko Damiani – [email protected]
Reference board● LeopardBoard 365
● DM365 SoC (Texas Instruments)● ARM processor
● Linux support● TI evaluation board (DaVinci)● LeopardBoard differs from TI evaluation board● Board Support Package available
5 / 35
Mirko Damiani – [email protected]
Overview● Board flashing
● Strategy
● System startup● Bootloader
● Kernel● How to load and start it
● Root filesystem● How to create it
6 / 35
Board flashing
● Common methods● Flashing strategy● Rootfs archive● Read-only rootfs● Flashing on the fly
7 / 35
Mirko Damiani – [email protected]
Common methods● Utilities to program the NAND flash
● SD/MMC card● USB (memory stick or protocol)● Serial port
● Custom boards may need a specificflashing strategy
● SD/MMC or USB may be missing● Flashing utilities might not work● Serial port is almost always present
8 / 35
Mirko Damiani – [email protected]
Flashing strategy● Let's suppose we are able to flash
at least the bootloader
● We load the entire system in RAM● Kernel is always loaded in RAM● Rootfs is passed as an initramfs archive● Kernel extracts its initramfs in RAM when it boots up
● System is flashed for permanent setup● Linux starts from NAND flash
9 / 35
Mirko Damiani – [email protected]
Flashing strategy● Small components
● Kernel is less than 4MB● Rootfs fits in 32MB (unpacked)
● No different configurations● The same system is used both for
flashing and production● Software development is easier
● Need more space?● Minimal rootfs is able to mount
other NAND partitions
PC(tftp server)
Board
NAND
10 / 35
Mirko Damiani – [email protected]
Rootfs archive● The same rootfs must be saved in 2 different formats
● Initramfs (gzipped cpio archive)● SquashFS image (read-only archive)
● Different kernel parameters, for instance:● Initramfs
rdinit=/sbin/init initrd=0x82000000,0x82a190● SquashFS
init=/sbin/init root=/dev/mtdblock3 rootfstype=squashfs
11 / 35
Mirko Damiani – [email protected]
Read-only rootfs● Still need to have some R/W folders
● /etc, /tmp, /var, /dev
● Move contents in RAM and remount from there● mount -n -t tmpfs none /mnt/rwfs -o size=4096k● mkdir -p /mnt/rwfs/etc● cp -a /etc/* /mnt/rwfs/etc● mount -n --bind /mnt/rwfs/etc /etc
● Now we don't have write errors anymore● Read-only filesystems are fast
12 / 35
Mirko Damiani – [email protected]
Flashing on the fly● Kernel and rootfs are downloaded via tftp server again● It's not convenient to save files in RAM,
since we might not have enough room
● Both files are directly flashed,without storing them in RAM
RAM NAND
NAND
13 / 35
System startup
● DM365 as reference● Boot overview● Boot loading stages● Debug at early stages
14 / 35
Mirko Damiani – [email protected]
Boot overview● Board initialization
● Low level, hardware related
● Bootloader execution● Persistent configuration (kernel params)
● Kernel loading● Operating system and devices setup
● System startup● /sbin/init and other scripts
bootloader
kernel
rootfs
15 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)Power on
RBL(1st stage)
UBL(2nd stage)
U-Boot(3rd stage)
● ROM boot loader (RBL)● ARM subsystem initialization
● User boot loader (UBL)● External devices initialization
● U-Boot● Kernel loading
16 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)Power on
RBL(1st stage)
UBL(2nd stage)
U-Boot(3rd stage)
NAND
UART
USB
Get UBL
Store in IRAM(32KB)
17 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)Power on
RBL(1st stage)
UBL(2nd stage)
U-Boot(3rd stage)
Get U-Boot
NAND
UART
USB
Store in RAM(128MB)
18 / 35
Mirko Damiani – [email protected]
Bootloader stages (DM365)Power on
RBL(1st stage)
UBL(2nd stage)
U-Boot(3rd stage)
Get Kernel
Run Linux
Scripts
Parameters
Environment
19 / 35
Mirko Damiani – [email protected]
RBL NAND loading (DM365)● Loading algorithm
● NAND memories are organized in sectors and pages
● RBL searches for a valid block● In case of read errors,
next block is searched fora valid UBL (up to #24)
● Robustness● Multiple UBL / U-Boot pairs
may be saved in NAND● Flashing utilities do this for you
20 / 35
Mirko Damiani – [email protected]
Debug at early stages● U-Boot also provides an UBL for DaVinci boards
● TI UBL may be replaced with U-Boot UBL● UBL sets the ARM processor frequency,
so i might want to change that values
● Very hard to debug● Commons methods (printf, debugger, gdb, valgrind...)
are not available● Boards simply don't boot,
without any evidence of errors
● A useful approach: exploit GPIO pins
21 / 35
Mirko Damiani – [email protected]
LeopardBoard GPIO● Access some registers (memory mapped I/O)
● Setup pin multiplexing● Select GPIO direction● Drive GPIO as output high or low
22 / 35
Kernel
● Board Support Packages● Configuration and build● Loading and parameters● Patches● NAND partitioning
23 / 35
Mirko Damiani – [email protected]
Board Support Packages● Kernel and drivers are patched to work with the
specified development board● We must understand how they works,
e.g. a camera driver would use video4linux● Custom boards may differ from development boards,
so additional SW patches are required
● Ad-hoc drivers are not easy to write from scratch, e.g a camera driver we would support only a specific sensor and very few operation modes
● Minimal drivers lead to an higher code maintenance
24 / 35
Mirko Damiani – [email protected]
Configuration and build● Compilation
● Download from www.kernel.org● Toolchain: Sourcery Codebench Lite
● Select the correct system and board, e.g.:● ARM system type● DaVinci 365 based system
● Disable all which is not needed● Vanilla
● We have a mainstream kernel● Easy to maintain (update)● Recent version
25 / 35
Mirko Damiani – [email protected]
Kernel loading● Loading is performed
by the bootloader● Arguments are passed
to the kernel● Kernel sets up its devices● When booting is completed,
/sbin/init is called(from within the rootfs)
● /sbin/init is responsible forbringing up the rest of thesystem (the userland)
args
bootloader
/sbin/initpid = 1
26 / 35
Mirko Damiani – [email protected]
Kernel patches● DaVinci RTC on LeopardBoard 365
● Driver is tested on DM360 EVM● LeopardBoard has a different schematic
● Mux pins and drive some GPIOs
27 / 35
Mirko Damiani – [email protected]
Kernel parameters● Serial console device and options
● console=ttyS0,115200n8
● LeopardBoard 365● MAC address changes at every reboot● We can save it on U-Boot environment,
and pass it to the Linux kernel– davinci_emac.macaddr=AA:BB:CC:DD:EE:FF– davinci_emac.macaddr=${ethaddr}
28 / 35
Mirko Damiani – [email protected]
NAND partitioning● Bootloader is
never touched
● Environment holdskernel arguments
● This layout may bestatically writtenin kernel sources,or passed througharguments
bootloader
environment
kernel
rootfs
30 blocks
256 KB
4 MB
32 MB
30 / 35
Mirko Damiani – [email protected]
Busybox● Provides almost a complete environment
for small systems● Tiny version of common utilities (fewer options)
● Single executable● Size optimization● Static or dynamic library linking● Symlinks to busybox executable
– e.g: /bin/ls -> /bin/busybox● Default configuration
● Static: 2MB● Dynamic: 936KB
31 / 35
Mirko Damiani – [email protected]
Busybox● Packages
● Coreutils, Shells, Editors, Network, …● Login files
– /etc/passwd, /etc/shadow, /etc/groups, …● System logs
– syslogd, klogd● ...
● Init program (/sbin/init)● Reads /etc/inittab● Runs /etc/rc.d/rcS● Other scripts, /etc/rc.d/init.d/network, ...
32 / 35
Mirko Damiani – [email protected]
Bootloader environment● Recall: boot env is stored into the 2nd partition● Example: U-Boot
● Provides some tools: fw_printenv, fw_setenv
● Generally speaking, it's possible toaccess them from userspace.
● Within our flashing procedure,IP address of tftp servermay be read from thebootloader environment
● System might not save at allkernel and rootfs into NAND,while being still configurable
192.168.10.101
33 / 35
Mirko Damiani – [email protected]
Building the root filesystem
foldersstructure
toolchainlibraries
busybox
kernelmodules
skeletonfiles
rootfs
cleanbinaries