An introductionto mGAR (Dublin 2011)

mGARProducing complex packages for Solaris the easy way

OpenCSW Technical WintercampDublin 2011

Dagobert Michelsendam@opencsw.org

1Samstag, 19. Februar 2011

find /usr/local/pkg -print > /tmp/filescat /tmp/files | pkgproto > /tmp/Prototypevim /tmp/Prototype

PKG=FOOtreeNAME=tree - A recursive directory listing programARCH=sparcVERSION=1.5.3,REV=2010.02.02CATEGORY=applicationVENDOR=http://mama.indstate.edu/users/ice/tree/EMAIL=joe@example.comPSTAMP=joe@darkstar-20100202140539CLASSES=noneHOTLINE=http://www.example.com/

pkgmk -o -r / -d /tmp -f Prototypepkgtrans . tree.pkg FOOtreegzip tree.pkg

Once upon a time

2Samstag, 19. Februar 2011

What is missing? What‘s wrong?

CompilationPackaging standardsChecking for errorsHardcoded prototypesHardcoded pkginfo

3Samstag, 19. Februar 2011

Enter mGAR

gmake package4Samstag, 19. Februar 2011

5

What is mGAR?

A system to build packages for SolarisOrigins are „GAR“LNX-BBC and GARNOMEm = modulated➜ multiple similar buildsFiles written in GNU Make

5Samstag, 19. Februar 2011

The minimal information to build a package:1. What should the package be named2. Where to get the sources3. What the dependencies are

The simplest case

6Samstag, 19. Februar 2011

NAME = gperfVERSION = 3.0.3CATEGORIES = devel

DESCRIPTION = A perfect hash function generator

MASTER_SITES = $(GNU_MIRROR)DISTFILES = $(NAME)-$(VERSION).tar.gz

CONFIGURE_ARGS = $(DIRPATHS)

TEST_TARGET = check

include gar/category.mk

Example: gperf

7Samstag, 19. Februar 2011

include gar/category.mk

Should be really named 'profiles'

Contains settings common to a class of builds

Generic

Perl Modules

Ruby Gems

Python Eggs

Categories

8Samstag, 19. Februar 2011

Typical customization needsCode needs to be patched to compile or run on SolarisCode doesn‘t honor information passed via ./configureFiles need to placed at specific locations in the package

9Samstag, 19. Februar 2011

Phases

prerequisites fetch checksum exract patch configure build test install merge package checkpkg

...and custom targets:pre-fetchpost-fetch

pre-extract(...)

10Samstag, 19. Februar 2011

„A Modulation is the process of varying one waveform in relation to another

waveform“For software builds that means:

„A Modulation is the process of varying a build in relation to certain parameters“

Modulations

11Samstag, 19. Februar 2011

PrerequisitesFetchChecksumExtractPatchConfigureBuildTestInstallMergePackage

Executedglobally

Executed insidemodulation

Executed globally

Phases and modulations

One modulation

12Samstag, 19. Februar 2011

What can be usedas modulator?

ISA for build architectureFor 32/64 bit versions and optimized binaries.See lame

Package versionUseful when multiple versions of a library should be includedSee automake

...anything that controls the buildStatic vs. dynamic linked versionsSee wget

Parameter sizeSee nrpe

Wide vs. narrow charactersSee ncurses

13Samstag, 19. Februar 2011

Modulationover ISA

Allows easy building of multiple ISAsBuilt-in special case of modulation

No merge needed also

Simplest case BUILD64 = 1

Advance dEXTRA_BUILD_ISAS = sparcv8plus+vis pentium_pro

14Samstag, 19. Februar 2011

Modulationover ISA requirements

Certain ISAs require specific platformsx64 requires Solaris 10, but integrated package for Solaris 8-10

Parallelizing modulation buildsMay require builds on multiple hosts

Build platform for each modulation can be specified in the package manifestHow does ISAEXEC relate to multi-ISA?

15Samstag, 19. Februar 2011

Cool trick:Use dynamically expanded variables

to set per-modulation

GARNAME = wget...EXTRA_MODULATORS = STATICMODULATIONS_STATIC = disable enable

EXTRA_CONFIGURE_STATIC-disable = --with-sslEXTRA_CONFIGURE_STATIC-enable = --disable-sharedEXTRA_CONFIGURE_STATIC-enable += --enable-staticEXTRA_CONFIGURE_STATIC-enable += --without-ssl...CONFIGURE_ARGS += $(EXTRA_CONFIGURE_STATIC-$(STATIC))

MERGE_SCRIPTS_isa-sparcv8-static-disable = copy-all MERGE_SCRIPTS_isa-sparcv8-static-enable = rename-wget copy-only MERGE_DIRS_isa-sparcv8-static-enable = $(bindir)

MERGE_SCRIPTS_isa-i386-static-disable = copy-all MERGE_SCRIPTS_isa-i386-static-enable = rename-wget copy-only MERGE_DIRS_isa-i386-static-enable = $(bindir)

merge-rename-wget: @mv $(DESTDIR)$(bindir)/wget $(DESTDIR)$(bindir)/wget.static...include gar/category.mk

Modulationover static/dynamic linking

Adds modulator withthis name

16Samstag, 19. Februar 2011

Modulationover parameter size

GARNAME = nrpe...EXTRA_MODULATORS = SIZEMODULATIONS_SIZE = 1k 8k...PATCHFILES_isa-sparcv8-size-8k = common.h.diffPATCHFILES_isa-i386-size-8k = common.h.diff...MERGE_SCRIPTS_isa-sparcv8-size-1k = copy-allMERGE_SCRIPTS_isa-sparcv8-size-8k = copy-nrpe-only

MERGE_SCRIPTS_isa-i386-size-1k = copy-allMERGE_SCRIPTS_isa-i386-size-8k = copy-nrpe-only

include gar/category.mk...merge-copy-nrpe-only: cp $(INSTALLISADIR)$(bindir)/nrpe_1k $(PKGROOT)$(bindir)/nrpe_8k cp $(INSTALLISADIR)$(LIBEXECDIR)/check_nrpe $(PKGROOT)$(LIBEXECDIR)/check_nrpe_8k @$(MAKECOOKIE)

Ok, I guess you understood how this works

Use patchfiles on aper-modulation basis

17Samstag, 19. Februar 2011

Modulationsand their names

The name of a modulation is <modulator1>-<value1>-<modulator2>-<value2>-...

Call each phase from a modulation manually with <phase>-<modulation>

Example: gmake build-isa-sparcv8

This results in a sub-make-invocation <phase>-modulated MODULATION=<MODULATION> <MODULATOR1>=<value1> ...

Example: gmake build-modulated MODULATION=isa-sparcv8 ISA=sparcv8

18Samstag, 19. Februar 2011

MergingWhy is this needed?

Optimized binaries with isaexec

Libraries may be located elsewhere in the package than they were configured

…/lib/32 -> .…/lib/64 -> sparcv9 | amd64

Pathes stored in pkgconfig-files

19Samstag, 19. Februar 2011

MergingWhy is this needed?

32 and 64 bit need different install locations to have sane pkgconfig-files

build-isa-sparcv8/opt/csw/bin/myappbuild-isa-sparcv9/opt/csw/bin/64/myapp

pkgroot/opt/csw/bin/myapppkgroot/opt/csw/bin/sparcv9/myapp

/opt/csw/bin/myapp -> isaexec/opt/csw/bin/sparcv8/myapp/opt/csw/bin/sparcv9/myapp

Merge

Package

Install

20Samstag, 19. Februar 2011

MergingBuiltin rules

Copy all Copy only someMERGE_DIRS_<mod>

Do not relocate

Do relocate

copy-all copy-only

copy-relocate copy-relocated-only

21Samstag, 19. Februar 2011

Mergingwith custom rules

Only merge specific filesGARNAME = nrpe...MERGE_SCRIPTS_isa-sparcv8-size-1k = copy-allMERGE_SCRIPTS_isa-sparcv8-size-8k = copy-nrpe-only

MERGE_SCRIPTS_isa-i386-size-1k = copy-allMERGE_SCRIPTS_isa-i386-size-8k = copy-nrpe-only

include gar/category.mk...merge-copy-nrpe-only: cp $(INSTALLISADIR)$(bindir)/nrpe_1k $(PKGROOT)$(bindir)/nrpe_8k cp $(INSTALLISADIR)$(LIBEXECDIR)/check_nrpe $(PKGROOT)$(LIBEXECDIR)/check_nrpe_8k @$(MAKECOOKIE)

This is the default buildJust copy everything over

We want to control exactly what is used from the additional modulation

Merging copies from the install location to PKGROOT

22Samstag, 19. Februar 2011

They mean more or less 'hooks'f cswinitsmf /opt/csw/etc/init.d/cswpuppetdi.<class> scripts handle the files

i.cswinitsmfr.cswinitsmf

GAR integration:INITSMF = $(prefix)/init\.d/cswpuppetd

Fully automated:Texinfo registrationConfiguration file preservationPlacement of README.CSW in package docs...

Class Action Scripts

23Samstag, 19. Februar 2011

checkpkg and overridesExamines a set of .pkg filesLooks for common issuesWritten mostly in Python, with lintian in mind

allows to override checks

24Samstag, 19. Februar 2011

Your package is ready!

25Samstag, 19. Februar 2011

Questions?

Bueller? Anybody?26Samstag, 19. Februar 2011