Chapter 4. Building installer images

Table of Contents

4.1. The standard D-I and CD images
4.1.1. Debian Installer images
4.1.2. CD and DVD images
4.2. Requirements for building
4.3. Build targets
4.4. The build system explained
4.4.1. config
4.4.2. pkg-lists
4.5. Result of the build
4.6. Library reduction
4.7. Using localudebs

This document only provides an introduction to building installer images using existing definitions. The README in installer/build in the SVN repository contains more detailed information about the build system and how to modify existing or define new images.

An image consists of:

Most d-i images are “ready for use”. The exception are the cdrom images which form only the base (kernel and initrd) for creating the actual CD or DVD images. The package used for creating the CD/DVD images is debian-cd.

On some architectures there is one CD image that is ready for use: the mini.iso. This image is produced as a by-product of the netboot target. A mini.iso image does not really support installing from CD as it retrieves all additional udebs and packages over the network.

It is important to distinguish between building images for release and building images for development/testing use.

A release build is done, as for other packages that are to be uploaded, from the installer directory using debian/rules. This will create a binary package (needed for uploading) containing some documentation, but the important bit is a tarball containing all installer images. After the upload this tarball needs BYHAND processing[7] by FTP-masters before the buildds will pick up the upload for other architectures. Nowadays the BYHAND processing for debian-installer (and also for tasksel) is automated.

Building images for development and testing is done from the installer/build directory[8] using fakeroot make <target>.

4.1. The standard D-I and CD images

An important difference between release and development builds is that release builds will use udebs for the same suite as the target system being installed, while development builds will by default install testing, but use udebs from unstable. This allows to mostly avoid the occasional breakage of the base system and tasks in unstable while using the most recent udebs.

4.1.1. Debian Installer images

  • daily D-I builds are built in unstable and use udebs from unstable
  • regular D-I uploads to the archive are built in unstable and use udebs from testing
  • D-I uploads to stable are built in stable and use udebs from stable + proposed-updates

The two config variables that determine this are:

USE_UDEBS_FROM

Defines the suite or codename from where udebs are taken. The default is set to unstable in ./config/common for SVN/trunk. For official uploads the value of this variable is overruled in debian/rules.

The value in ./config/common should be changed to the correct codename for RC release uploads (after branching) to ensure correct builds from the branch post-release.

DEBIAN_RELEASE

Defines the codename of the Debian release that should be installed by default. It gets included in /etc/default-release in most initrds and is read by various components during installation.

Because this is set to a codename, the value is correct for the whole lifetime of a Debian release and helps ensure the correct suite gets installed even when a release becomes “oldstable” or gets archived.

Some installation methods allow this value to be overruled by choosing a different suite during mirror selection or passing the suite= option boot parameter.

4.1.2. CD and DVD images

  • daily “sid_d-i” CD images are built using daily D-I builds, udebs from unstable and regular packages from testing
  • daily “squeeze_d-i” CD/DVD images are built using the current D-I release in testing, udebs from testing and regular packages from testing
  • weekly CD builds are built using the current D-I release in testing, udebs from testing and regular packages from testing
  • official CD/DVD builds use everything from stable and thus can only be built after the archive has already been updated to a new stable (point) release

The “sid_d-i” CDs are what's normally linked as the daily built images from the D-I project page; when preparing for a release those will link to “squeeze_d-i” CDs to facilitate pre-release testing.

4.2. Requirements for building

For both release and development builds the build dependencies as listed in installer/debian/control need to be satisfied.

To build installer images from SVN trunk, the build machine needs to be running unstable or you need to set up a sid chroot to build in. (To build images from one of the older release branches of the repository, the build machine needs to run or have a chroot for that release.)

During the build, the needed udebs will be retrieved from a mirror. By default this mirror is based on your /etc/apt/sources.list (see the generated file build/sources.list.udeb). To use a different source, create a file sources.list.udeb.local.

4.3. Build targets

To see which targets are available, run make. This will result in a list of some 130 targets, most of which are not really relevant. A more useful list can be obtained with make | grep ^build. The table below has the most often used targets for x86.

build_all Builds all images
build_cdrom_isolinux Builds the cdrom images
build_netboot Builds the netboot images and the mini.iso
reallyclean Completely cleans the build environment

The reallyclean target is often needed when changes are made between builds because otherwise udebs or information may be retrieved from temporary or cache directories and the changes will not take effect. The rebuild_* targets clean some of this, but not always enough.

4.4. The build system explained

The easiest way to start is with the purpose of the subdirectories in the installer/build directory.

  • util: contains helper scripts called from the Makefile

  • config: defines the available targets (per architecture)

  • pkg-lists: defines which udebs are included in an image (per image type)

  • boot: contains configuration files and make targets used to make images bootable

  • localudebs: allows to use (versions of) udebs not available on the mirror you use

Two files containing important configuration info are config/dir and config/common. However, normally there should be no need to modify any of the variables defined in these files.

Both the config and pkg-lists directories have a tree structure with general configuration defined in the root and more specific configuration defined in branches and leaves. Branches are defined in directories that have the same name as a config file on the higher level. The config directory contains Makefile snippets.

4.4.1. config

For example, the definition for i386 images starts with config/i386.cfg which, besides the current kernel versions, defines the media supported with the line:

MEDIUM_SUPPORTED = cdrom cdrom-xen netboot netboot-xen hd-media

These media correspond to the main targets for i386 and are further defined in config/i386. The hd-media.cfg file in that directory contains, amongst others, the following lines:

FLAVOUR_SUPPORTED = "" gtk
EXTRATARGETS = build_hd-media_gtk

Th first line defines that the netboot image has two flavors: the default one (for the newt frontend), and a second one for the gtk frontend. The latter is further defined in the config/i386/hd-media/gtk.cfg file. The second line ensures that the gtk image will always be built together with the newt image.

The files in config are processed recursively to dynamically generate the build targets, so in this example you get a hd-media, a hd-media_gtk target and targets for the other media.

The structure of the config files can get quite complex and it can be hard to keep track of the exact role of the different variables set in them.

4.4.2. pkg-lists

The list of udebs to be included in an image is built by the util/pkg-list script based on definitions in the pkg-lists directory. Again, processing can be quite complex. Let's take the netboot target for i386 as an example to explain it.

First the file pkg-lists/netboot/i386.cfg is considered and all udebs listed in it are added. Some example lines from that file:

console-keymaps-at
nic-modules-${kernel:Version}
mmc-modules-${kernel:Version} ?

The variable ${kernel:Version} is expanded to match the package name of the udeb based on the kernel version and flavor. If the name of a udeb is followed by a question mark it is skipped if the package is not available (without the question mark an error would be generated).

The pkg-list script will also look for the presence of files named common and local and thus pkg-lists/netboot/common is processed next. This file exists and lists a number of udebs that belong in any netboot image, independent of the architecture. This file contains two include directives which result in the specified files being processed next:

#include "base"
#include "kernel"

Thus, udebs listed in pkg-lists/base (containing udebs common to all images) and pkg-lists/kernel (included in all bootable images) are also processed.

The file pkg-lists/netboot/local does not normally exist as it is intended for the inclusion of non-standard udebs. It is also very useful for testing as it can be used to temporarily add udebs not normally included in an image without the need to modify the regular files.

Finally, the script will check for pkg-lists/local and pkg-lists/exclude. The latter exists and contains some udebs otherwise pulled in by dependencies, but that should not be included because of library reduction, which is covered in the next section. Note that the exclusion if not triggered by the file name, but rather by the dash after the name of the udebs.

All dependencies of udebs listed in pkg-lists will also be automatically included in the image.

To see how the package list is built for a particular image, set my $debug=1; in the util/pkg-list script.

4.5. Result of the build

If the build is successful, the images that were built can be found under the build/dest directory. Depending on the type of build you will also find manifest and log files there.

Before the image is created, its contents are assembled in the directory build/tmp/<target>. The tree subdirectory there contains the full contents of the initrd; other subdirectories are used for different purposes.

4.6. Library reduction

Library reduction (relinking a library leaving out unused symbols) is used as yet another method to minimize the size of initrds. The downside of library reduction is that this requires the dev and pic packages for the libraries to be reduced to be installed on the build system which also means that their version needs to match the version of the libraries in the udebs.

The size reduction is most significant for libc (40%) and libm (90%). Other libraries that are reduced include libresolv, libslang and libnewt. The reduction is done by calling mklibs from the main Makefile.

As only the executables that are included in an image are taken into account during the library reduction, we have provide for executables in components that are installed later as they would fail if they use symbols that have been taken out.

This is the reason that the udebs containing reduced libraries are excluded in pkg-lists/exclude which results in the udeb not being listed in the /var/lib/dpkg/status file in the intrd. If no udebs that are installed later depend on the library, all is well. If a udeb that does depend on it is installed later, anna (or rather udpkg) will see that the dependency is not satisfied, and will install the udeb so the unreduced library replaces the reduced version.

Note that library reduction is only done after unpacking udebs for inclusion in an image; the libraries included in udebs are never reduced.

4.7. Using localudebs

The localudebs directory allows to use a different version of udebs than is available from the mirror you use. This can be used to test a new version of a udeb or to run the installer with a debug version of a udeb. It can also be used to build an image with a custom udeb.

To use a local udeb, just copy it into the directory. A Packages file will be generated automatically. Your udeb should have a version equal to or greater than the udeb currently on the mirror you use.

Note that local udebs will only be included in the image if the udeb would be included in a normal build too. So it has to be selected by the pkg-list script. Create a pkg-lists/local or pkg-lists/<image>/local to add udebs to the image that would not normally be included.

Some things to keep in mind when using localudebs.

  • If you add an extra udeb, its dependencies will be included too. If those dependencies include virtual packages, the result is not always what you'd expect.

  • Adding extra udebs will increase the size of the initrd; some architectures have limits for initrd size.

  • If you use a sources.list.udeb.local, make sure to add as the first line:

    deb copy:<path-from-root-to>/installer/build/ localudebs/
    

  • Don't forget to clean up after you're finished.



[7] This entails unpacking the tarball into the correct location on the master mirror server and creating/updating the correct symlinks. See for example http://ftp.debian.org/debian/dists/sid/main/installer-i386/.

[8] This includes the daily built images available from http://www.debian.org/devel/debian-installer. These are generated and uploaded from machines run by d-i porters using the daily-build script.