http://www.sprezzatech.com/sprezzos.html This repository builds and tests SprezzOS install media. Components ---------- I. The scripts and miscellany contained in this repo. II. The 's-i' installer program, forked from Debian's 'd-i'. It is checked out as part of the 'build' script. III. The udebs and debs that are part of SprezzOS, and not Debian. They are build with the "world" and "udebs" targets in the toplevel, and copied by hand to the SprezzOS APT repository. IV. The udebs used by s-i to build the installer's initfs. These are downloaded by the s-i build process, and copied from the localudebs/ directory in its build tree. V. The udebs and debs used by simple-cdd to build the installer media's local package repository. These are downloaded by simple-cdd, and explicitly provided to it by the Makefile. Dependencies ------------ Building requires GNU make, debootstrap, grub2-pc, grub2-efi, dh-di, simple-cdd (which in turn requires debian-cd), wget, fuseiso9660 and probably a few other things. Building with a decent level of parallelism (-j4) is recommended. Testing adds a dependency on KVM. Process ------- The file "exclude-udebs-amd64" ought be merged into /usr/share/debian-cd/data/unstable. It would be great if we could control this from the build, but that doesn't appear easy to do :/. By default, running "make" in this toplevel will build a SprezzOS installation ISO. Running "make test" will build it, and then boot a KVM with it. Besides the default target, "world", "kernel", and "udebs" exist to support building userspace packages, kernel packages, and installer packages, respectively. This proceeds as follows: 1. A chroot is created at "unstable" by invoking the script "build". "build" uses debootstrap to set up the chroot, checks out "s-i" and its derivatives, and checks out "sprezzos-world". Some files are copied into the chroot, both by "build" and the Makefile. Among them is "innerbuild". Step 1 is a dep of all build processes. For the default process, we proceed to Step 2, as the Makefile chroots into "unstable/" and invokes "/innerbuild". 2. The "s-i" package is built in the chroot's context, generating a debian-installer deb and a directory of initramfs images. The toplevel symlink "dest" references these images, and is used by simple-cdd. Note that the packages used in this step are those downloaded from an APT repository, not packages built in the chroot. 3. The "makecd" script is invoked, building a Debian installer ISO in the directory "images/". 4. The "grubify" script is invoked, building a SprezzOS installer ISO in the toplevel. This process requires that all component packages have already been built in a satisfactory form, and are available on the referenced APT repositories. To build these packages, run one of the helper scripts: "world": Build normal, userspace packages. This involves running the Makefile of the "sprezzos-world" repository within the chroot context. "kernel": Build the Linux packages, and external module packages appropriate for those Linux packages. This involves downloading the Linux kernel and those modules' sources, and building them all. "udebs": Build non-kernel packages for the installer. This involves patching the checked-out Debian repositories within "s-i/packages", and building the new udebs within the chroot context. Building -------- The Makefile ought, depending on the day of the week, go through the entire (lengthy) CD build process without user interaction. During any serious development, you'll likely find yourself using the following: Run './update' to regenerate the tarball of upstream packages and fonts. Run './build ' to generate the chroot at . If you get an error about signature verification failures, try running './update'. Run './makecd' to generate the CD. Run './runcd kvmdiskimg isoimage' to test the CD. Notes ----- Working with simple-cdd/debian-installer is not very pleasant. Building SprezzOS involves: - creating a clean environment for building, using debootstrap - rebuilding the installer kernel's various udebs and debs - building udebs of external modules for the installer kernel (SPL, ZFS) - building udebs of partman extensions (partman-zfs) - rebuilding the debian-installer udeb to use these udebs - building a cd using these various packages, using simple-cdd Tools you need to know beyond standard dev: - the Debian packaging system (a package's debian/), fairly intimately - debian-installer internals - debhelper - debconf - the Debian kernel packaging system (kernel-wedge, etc) - Partman, the installer's partitioning software ___________ |debootstrap| ----------- This is pretty simple. We use the buildd variant with a bunch of extra packages. We're specifying mklibs-copy in config/common in a workaround for ZFS/SPL-related barfing during library size reduction. This and should go -- we ought only include .ko's in the kernel udebs FIXME. ______ |kernel| ------ Please consult the Debian Kernel Maintainer's Handbook at http://kernel-handbook.alioth.debian.org and the Debian Kernel FAQ at http://wiki.debian.org/KernelFAQ and the Debian Wiki: http://wiki.debian.org/HowToRebuildAnOfficialDebianKernelPackage We are not using kernel-package's "make-kpkg" to build .debs, but instead a version-tracked debian/ directory and dpkg-buildpackage. The result will be a great many udebs and some debs. Acquire the desired kernel via git or tarball. Acquire the debian packaging from kernel-packaging in this tree. I've made changes including: - removal of most patches - removal of some module udebs (ext2, ext3, ext4, core): - remove files from installer/ tree - removal of some modules, but not entire udeb - edit files in installer/ tree - kernel configuration changes (FIXME i don't think i'm doing this correctly) - involves files named 'config' in the config/ tree - build configuration changes - files named 'defines' in the config/ tree Configure the kernel. The control file is built by debian/bin/gencontrol.py, which calls kernel-wedge after exporting KW_CONFIG_DIR appropriately for the architecture. gencontrol.py appears to be getting ABI information from the changelog (?!?). Running "kernel-wedge gen-control {ABIVERS}" by hand will properly rebuild the control file, but running gencontrol.py again (say, via "debian/rules debian/control") will kill it. I'm doing something incorrectly here FIXME. We would run dpkg-buildpackage, but that invokes gencontrol.py. Instead, run ""debian/rules binary". FIXME ________________ |external modules| ---------------- You'll need access to the headers for the appropriate kernel. If you're using the default kernel, just install its kernel-headers packages. Otherwise, you'll need install the kernel-headers package generated in your kernel-building step. Modify the debian/rules file to ensure configuration steps are using the correct kernel locations. You need build a kernel module udeb for the installer and then regular packages for the post-install environment. _______ |partman| ------- FIXME (considerations include: you won't show up in the partman config unless your filesystem module tests pass; /var/log/partman is created once partman starts running...) _____ |udebs| ----- .udeb files are generated by ensuring XC-Package-Type: udeb Section: debian-installer is in the package's entry in the debian/control file. If the udeb belongs in the main menu, it needs have a XB-Installer-Menu-Item: 99999 line. 99999 places the entry at the bottom, and keeps it from being run as part of a typical install. udebs ought carry as little baggage around as possible. udebs need be placed in build/localudebs within a debian-installer source checkout. dpkg-buildpackages will ensure the necessary Packages/Packages.gz files are updated according to this directory's contents. They then need be included in a processed build/pkg-list/ file. "local" is provided for our use. ${kernel:Version} will be substituted with the kernel version of the installer. Dependencies needn't (and oughtn't) be listed in pkg-list/local. ________________ |debian-installer| ---------------- You'll need be able to build a debian-installer checkout. Check it out from svn, or use "apt-get source debian-installer". dpkg-checkbuilddeps in a debian-installer checkout will ensure you have the necessary packages installed to rebuild it. The svn repo is: svn://svn.debian.org/svn/d-i/trunk cd trunk scripts/git-setup mr -p checkout Edit build/config/common as necessary. If you want external apt sources, create a new file sources.list.udeb.local. Ensure the first line is deb copy:/debian-installer-xxxxx/build/ localudebs/ in order to use your localudebs. __________ |simple-cdd| ---------- Once you've built your custom debian-installer, simple-cdd can be used to generate installation media. You'll need to copy the contents of build/dest/ from within the installer manually, and there appears no way to do way to do this better than: mkdir -p tmp/mirror/dists/sid/main/installer-amd64/current/images/ cp -r dest/* tmp/mirror/dists/sid/main/installer-amd64/current/images/ where tmp/blahblahblah is a string extracted from reading simple-cdd source code. This is all unfortunate. Create a profiles/ directory, and populate it. Read the source for examples; it's all too depressing to talk about. _____________ |grubification| ------------- simple-cdd sets up an ISO9660 image booting with ISOLINUX. For a number of reasons, we want to use grub2 instead: - better graphics support - more flexible - unified pre- and post-install bootloader environment - EFI support - one braindamaged bootloader is quite enough, thanks The 'grubify()' function in makecd mounts simple-cdd's output using FUSE and fuseiso9660, adds some files used during bootloading, and runs 'grub-mkrescue' to make a new ISO using GRUB2 as its bootloader.