http://www.sprezzatech.com/sprezzos.html
This repository builds and tests SprezzOS install media.
Components
----------
I. The scripts and miscellany contained in this repo.
II. The 's-i' installer program, forked from Debian's 'd-i'. It is checked out
as part of the 'build' script.
III. The udebs and debs that are part of SprezzOS, and not Debian. They are
build with the "world" and "udebs" targets in the toplevel, and copied by
hand to the SprezzOS APT repository.
IV. The udebs used by s-i to build the installer's initfs. These are
downloaded by the s-i build process, and copied from the localudebs/
directory in its build tree.
V. The udebs and debs used by simple-cdd to build the installer media's local
package repository. These are downloaded by simple-cdd, and explicitly
provided to it by the Makefile.
Dependencies
------------
Building requires GNU make, debootstrap, grub2-pc, grub2-efi, dh-di, simple-cdd
(which in turn requires debian-cd), wget, fuseiso9660 and probably a few other
things. Building with a decent level of parallelism (-j4) is recommended.
Testing adds a dependency on KVM.
Process
-------
The file "exclude-udebs-amd64" ought be merged into
/usr/share/debian-cd/data/unstable. It would be great if we could control this
from the build, but that doesn't appear easy to do :/.
By default, running "make" in this toplevel will build a SprezzOS installation
ISO. Running "make test" will build it, and then boot a KVM with it. Besides
the default target, "world", "kernel", and "udebs" exist to support building
userspace packages, kernel packages, and installer packages, respectively. This
proceeds as follows:
1. A chroot is created at "unstable" by invoking the script "build". "build"
uses debootstrap to set up the chroot, checks out "s-i" and its derivatives,
and checks out "sprezzos-world". Some files are copied into the chroot, both
by "build" and the Makefile. Among them is "innerbuild".
Step 1 is a dep of all build processes. For the default process, we proceed to
Step 2, as the Makefile chroots into "unstable/" and invokes "/innerbuild".
2. The "s-i" package is built in the chroot's context, generating a
debian-installer deb and a directory of initramfs images. The toplevel
symlink "dest" references these images, and is used by simple-cdd. Note
that the packages used in this step are those downloaded from an APT
repository, not packages built in the chroot.
3. The "makecd" script is invoked, building a Debian installer ISO in the
directory "images/".
4. The "grubify" script is invoked, building a SprezzOS installer ISO in the
toplevel.
This process requires that all component packages have already been built in a
satisfactory form, and are available on the referenced APT repositories. To
build these packages, run one of the helper scripts:
"world": Build normal, userspace packages. This involves running the Makefile
of the "sprezzos-world" repository within the chroot context.
"kernel": Build the Linux packages, and external module packages appropriate
for those Linux packages. This involves downloading the Linux
kernel and those modules' sources, and building them all.
"udebs": Build non-kernel packages for the installer. This involves patching
the checked-out Debian repositories within "s-i/packages", and
building the new udebs within the chroot context.
Building
--------
The Makefile ought, depending on the day of the week, go through the entire
(lengthy) CD build process without user interaction. During any serious
development, you'll likely find yourself using the following:
Run './update' to regenerate the tarball of upstream packages and fonts.
Run './build <builddir>' to generate the chroot at <builddir>. If you get an
error about signature verification failures, try running './update'.
Run './makecd' to generate the CD.
Run './runcd kvmdiskimg isoimage' to test the CD.
Notes
-----
Working with simple-cdd/debian-installer is not very pleasant. Building
SprezzOS involves:
- creating a clean environment for building, using debootstrap
- rebuilding the installer kernel's various udebs and debs
- building udebs of external modules for the installer kernel (SPL, ZFS)
- building udebs of partman extensions (partman-zfs)
- rebuilding the debian-installer udeb to use these udebs
- building a cd using these various packages, using simple-cdd
Tools you need to know beyond standard dev:
- the Debian packaging system (a package's debian/), fairly intimately
- debian-installer internals
- debhelper
- debconf
- the Debian kernel packaging system (kernel-wedge, etc)
- Partman, the installer's partitioning software
___________
|debootstrap|
-----------
This is pretty simple. We use the buildd variant with a bunch of extra
packages. We're specifying mklibs-copy in config/common in a workaround for
ZFS/SPL-related barfing during library size reduction. This and should go -- we
ought only include .ko's in the kernel udebs FIXME.
______
|kernel|
------
Please consult the Debian Kernel Maintainer's Handbook at
http://kernel-handbook.alioth.debian.org
and the Debian Kernel FAQ at
http://wiki.debian.org/KernelFAQ
and the Debian Wiki:
http://wiki.debian.org/HowToRebuildAnOfficialDebianKernelPackage
We are not using kernel-package's "make-kpkg" to build .debs, but instead a
version-tracked debian/ directory and dpkg-buildpackage. The result will be a
great many udebs and some debs.
Acquire the desired kernel via git or tarball. Acquire the debian packaging
from kernel-packaging in this tree. I've made changes including:
- removal of most patches
- removal of some module udebs (ext2, ext3, ext4, core):
- remove files from installer/ tree
- removal of some modules, but not entire udeb
- edit files in installer/ tree
- kernel configuration changes (FIXME i don't think i'm doing this correctly)
- involves files named 'config' in the config/ tree
- build configuration changes
- files named 'defines' in the config/ tree
Configure the kernel.
The control file is built by debian/bin/gencontrol.py, which calls kernel-wedge
after exporting KW_CONFIG_DIR appropriately for the architecture. gencontrol.py
appears to be getting ABI information from the changelog (?!?). Running
"kernel-wedge gen-control {ABIVERS}" by hand will properly rebuild the control
file, but running gencontrol.py again (say, via "debian/rules debian/control")
will kill it. I'm doing something incorrectly here FIXME.
We would run dpkg-buildpackage, but that invokes gencontrol.py. Instead, run
""debian/rules binary". FIXME
________________
|external modules|
----------------
You'll need access to the headers for the appropriate kernel. If you're using
the default kernel, just install its kernel-headers packages. Otherwise, you'll
need install the kernel-headers package generated in your kernel-building step.
Modify the debian/rules file to ensure configuration steps are using the
correct kernel locations. You need build a kernel module udeb for the installer
and then regular packages for the post-install environment.
_______
|partman|
-------
FIXME (considerations include: you won't show up in the partman config unless
your filesystem module tests pass; /var/log/partman is created once
partman starts running...)
_____
|udebs|
-----
.udeb files are generated by ensuring
XC-Package-Type: udeb
Section: debian-installer
is in the package's entry in the debian/control file.
If the udeb belongs in the main menu, it needs have a
XB-Installer-Menu-Item: 99999
line. 99999 places the entry at the bottom, and keeps it from being run as part
of a typical install. udebs ought carry as little baggage around as possible.
udebs need be placed in build/localudebs within a debian-installer source
checkout. dpkg-buildpackages will ensure the necessary Packages/Packages.gz
files are updated according to this directory's contents. They then need be
included in a processed build/pkg-list/ file. "local" is provided for our use.
${kernel:Version} will be substituted with the kernel version of the installer.
Dependencies needn't (and oughtn't) be listed in pkg-list/local.
________________
|debian-installer|
----------------
You'll need be able to build a debian-installer checkout. Check it out from
svn, or use "apt-get source debian-installer". dpkg-checkbuilddeps in a
debian-installer checkout will ensure you have the necessary packages
installed to rebuild it. The svn repo is:
svn://svn.debian.org/svn/d-i/trunk
cd trunk
scripts/git-setup
mr -p checkout
Edit build/config/common as necessary. If you want external apt sources, create
a new file sources.list.udeb.local. Ensure the first line is
deb copy:/debian-installer-xxxxx/build/ localudebs/
in order to use your localudebs.
__________
|simple-cdd|
----------
Once you've built your custom debian-installer, simple-cdd can be used to
generate installation media. You'll need to copy the contents of build/dest/
from within the installer manually, and there appears no way to do way to do
this better than:
mkdir -p tmp/mirror/dists/sid/main/installer-amd64/current/images/
cp -r dest/* tmp/mirror/dists/sid/main/installer-amd64/current/images/
where tmp/blahblahblah is a string extracted from reading simple-cdd source
code. This is all unfortunate.
Create a profiles/ directory, and populate it. Read the source for examples;
it's all too depressing to talk about.
_____________
|grubification|
-------------
simple-cdd sets up an ISO9660 image booting with ISOLINUX. For a number of
reasons, we want to use grub2 instead:
- better graphics support
- more flexible
- unified pre- and post-install bootloader environment
- EFI support
- one braindamaged bootloader is quite enough, thanks
The 'grubify()' function in makecd mounts simple-cdd's output using FUSE and
fuseiso9660, adds some files used during bootloading, and runs 'grub-mkrescue'
to make a new ISO using GRUB2 as its bootloader.