dh-kpatches implementation

The two Parts

The dh-kpatches system is made of two parts. The first one takes patch descriptions, and produces kernel-patch packages; the second one is (currently) contained in those patch packages, and handles application of the patches to a kernel source tree, as well as their removal.

The application/removal scripts are (currently) generated from templates by the dh_installkpatches script for each kernel-patch package.

Currently, the patch descriptions are completely parsed by dh_installkpatches, and the apply/unpatch scripts are specific to each package, and can only act on the patches dh_installkpatches taught them about.

Packaging-time work

The dh_installkpatches helper

This is the debhelper-based script that will cause our packages to install patches in the One True Way. Its base structure is that of a standard debhelper script.

<dh_installkpatches.in>=
#!/usr/bin/perl -w
#
# dh_installkpatches $Revision: 1.17.1.4.1.23 $
#
# Reads debian/$package.kpatches[.foo], installs all files necessary
# to have make-kpkg use the kernel patches described in there.
#
# (c) 2000-2003 Yann Dirson <dirson@debian.org>
# Some parts based on code from Adam Di Carlo and Joey Hess

use strict;
use Debian::Debhelper::Dh_Lib;
init();

<definitions for the core kpatch system>

PACKAGE: foreach my $package (@{$dh{DOPACKAGES}}) {
  my $tmp = tmpdir($package);
  my $ext = pkgext($package);

  # There are two filename formats, the usual
  # plus an extended format (debian/package.*).

  opendir(DEB,"debian/") || error("can't read debian directory: $!");
  # If this is the main package, we need to handle unprefixed filenames.
  # For all packages, we must support both the usual filename format plus
  # that format with a period an something appended.
  my $regexp="\Q$package\E\.";
  if ($package eq $dh{MAINPACKAGE}) {
    $regexp="(|$regexp)";
  }
  my @files = grep { /^${regexp}kpatches(\..*)?$/ } readdir(DEB);
  closedir(DEB);

  # next package if there are no patches in there
  next PACKAGE if $#files < 0;

  <process binary package $package>
}

Patch processing

For each binary package, all kpatches files are processed to generate one apply script with a single diff operation (and its corresponding unpatch script).

<process binary package $package>= (<-U)
<assert kpatch:depends is referenced in control file>

foreach my $file (@files) {
  my %patchinfo = read_control_file ("debian/$file");

  #   use Data::Dumper;
  #   print Dumper (%patchinfo);

  my $patchid = $patchinfo{general}->{'patch-id'};

  # transformation of the ID to be acceptable as part of an envvar's name
  $patchinfo{general}->{'clean-patch-id'} = $patchinfo{general}->{'patch-id'};
  $patchinfo{general}->{'clean-patch-id'} =~ s/-/_/g;

  # protect pipes and dquotes for sed command-line
  $patchinfo{general}->{'patch-name'} =~ s,([|\"]),\\$1,g;

  <generate apply/unpatch scripts for given $package and kpatches $file>
}

<set kpatch:Depends substvar>

kpatch:depends substvar

A number of packages are needed by kernel-patch packages:

bash 2.x or better
because apply scripts use arrays
patch
because we (currently) only apply patches made of diff files
grep-dctrl
is used to register the version of the kernel-patch package into the kernel-image package
<definitions for the core kpatch system>= (<-U) [D->]
my $pkgdeps = "bash (>= 2.0), patch, grep-dctrl";

The substvar code was derived from similar functionnality in dh_perl v3.4.1. For idempotency, we first remove anything this program might have previously added to the substvars file.

<set kpatch:Depends substvar>= (<-U)
if (-e "debian/${ext}substvars") {
  complex_doit("grep -v ^kpatch:Depends= debian/${ext}substvars > debian/${ext}substvars.new || true");
  doit("mv", "debian/${ext}substvars.new","debian/${ext}substvars");
}

complex_doit("echo 'kpatch:Depends=$pkgdeps' >> debian/${ext}substvars");

We also make sure the package uses our substvar, and abort if not.

<assert kpatch:depends is referenced in control file>= (<-U)
die 'debian/control must make package ' . $package . ' depend on ${kpatch:Depends}'
  if system ("dpkg-gencontrol -p$package -Pdebian -O -T/dev/null -Vkpatch:Depends=KPATCHISUSED |"
           . "grep -q '^Depends: .*KPATCHISUSED'") != 0;

Kernel-build-time work

For each v0 kpatches file, we produce an apply and an unpatch script, necessary to work properly with make-kpkg. Those are currently produced from templates.

Apply script

<apply.tmpl>=
#! /bin/bash
set -e

<Assert this is a kernel tree>
<Apply.tmpl constants>

DEPENDS=(#DEPENDS#)

KVERSIONS=(#KVERSIONS#)
PATCHFILES=(#PATCHFILES#)
DEBPATCHFILES=(#DEBPATCHFILES#)
STRIPLEVELS=(#STRIPLEVELS#)

<Make apply idempotent>
<Get current kernel version>
<Select alternative to apply, as $IDX>

echo >&2 "START applying patch \"#PATCHNAME#\""

<Handle dependencies>

<Assert patch applies, using dry-run>
<Apply the patch>

<Store necessary info for unpatching>
<Create script to have this patch registered in doc-dir>

echo >&2 "END applying patch \"#PATCHNAME#\""
<Get current kernel version>= (<-U)
VERSION=$(grep ^VERSION Makefile 2>/dev/null | \
        sed -e 's/[^0-9]*\([0-9]*\)/\1/')
PATCHLEVEL=$( grep ^PATCHLEVEL Makefile 2>/dev/null | \
        sed -e 's/[^0-9]*\([0-9]*\)/\1/')
SUBLEVEL=$(grep ^SUBLEVEL Makefile 2>/dev/null | \
        sed -e 's/[^0-9]*\([0-9]*\)/\1/')
EXTRAVERSION=$(grep ^EXTRAVERSION Makefile 2>/dev/null | \
        sed -e 's/EXTRAVERSION =[       ]*\([^  ]*\)$/\1/')
KERNELRELEASE=${VERSION}.${PATCHLEVEL}.${SUBLEVEL}${EXTRAVERSION}
<Select alternative to apply, as $IDX>= (<-U)
IDX=

declare -i i=${#PATCHFILES[*]}-1
while [ $i -ge 0 ]
do
    v=${KVERSIONS[$i]}
    if [ -n "$KPATCH_#CLEANPATCHID#" -a "$v" = "$KPATCH_#CLEANPATCHID#" \
         -o "$v" = "$KERNELRELEASE" -o "$v" = all ]
    then
        IDX=$i
    fi
    i=i-1
done

<Assert we have an alternative to apply>
<Record parameters for applying alternative $IDX>

If the KPATCH_* mechanism was used, we have a special error message listing the requested version.

<Assert we have an alternative to apply>= (<-U)
if [ -n "$KPATCH_#CLEANPATCHID#" -a ${KVERSIONS[$IDX]} != "$KPATCH_#CLEANPATCHID#" ]
then
    echo >&2 "Requested kernel version \`$KPATCH_#CLEANPATCHID#' not found for patch #PATCHID#"
    exit 1
elif [ -z "$IDX" ]
then
    echo >&2 "No \"#PATCHNAME#\" patch found for kernel version $KERNELRELEASE"
    exit 1
fi

Of special notice is the "debian" pseudo-flavour. Before the kernel-patch-debian era, we had to check for README.Debian. Now we must check for version.Debian.

<Record parameters for applying alternative $IDX>= (<-U)
KVERSION=${KVERSIONS[$IDX]}
STRIPLEVEL=${STRIPLEVELS[$IDX]}

if [ "${DEBPATCHFILES[$IDX]}" != '' -a \
    \( -r version.Debian -o -r README.Debian \) ]
then
    PATCHFILE=${DEBPATCHFILES[$IDX]}
else
    PATCHFILE=${PATCHFILES[$IDX]}
fi
<Handle dependencies>= (<-U)
<Check for dependencies>
<Apply dependencies>
<Check for dependencies>= (<-U)
NEEDED_DEPS=
for dep in ${DEPENDS[$IDX]}
do
    if [ -x ${TOPPATCHDIR}/${ARCHITECTURE}/${VERSION}.${PATCHLEVEL}.${SUBLEVEL}/apply/$dep ]
    then
        NEEDED_DEPS="${ARCHITECTURE}/${VERSION}.${PATCHLEVEL}.${SUBLEVEL}/apply/$dep $NEEDED_DEPS"
    elif [ -x ${TOPPATCHDIR}/all/${VERSION}.${PATCHLEVEL}.${SUBLEVEL}/apply/$dep ]
    then
        NEEDED_DEPS="all/${VERSION}.${PATCHLEVEL}.${SUBLEVEL}/apply/$dep $NEEDED_DEPS"
    elif [ -x ${TOPPATCHDIR}/${ARCHITECTURE}/apply/$dep ]
    then
        NEEDED_DEPS="${ARCHITECTURE}/apply/$dep $NEEDED_DEPS"
    elif [ -x ${TOPPATCHDIR}/all/apply/$dep ]
    then
        NEEDED_DEPS="all/apply/$dep $NEEDED_DEPS"
    else
        echo >&2 "ERROR: Patch dependency \`$dep' not found - aborting"
        echo >&2 "END applying patch \"#PATCHNAME#\""
        exit 1
    fi
done
<Apply dependencies>= (<-U)
if [ "$NEEDED_DEPS" ]
then
    echo >&2 "Ensuring the following patches are applied first: $NEEDED_DEPS"
    for apply in ${NEEDED_DEPS}
    do
        ${TOPPATCHDIR}/$apply

        # check something was applied
        if [ ! -f debian/APPLIED_${ARCHITECTURE}_$dep -a \
             ! -f debian/APPLIED_all_$dep ]
        then
            echo >&2 "ERROR: patch dependency did not left a patch stamp (version mismatch ?) - aborting"
            echo >&2 "END applying patch \"#PATCHNAME#\""
            exit 1
        fi
    done
    UNPATCHDEPS=$(echo ${NEEDED_DEPS} | sed s,/apply/,/unpatch/,g)
fi
<Assert patch applies, using dry-run>= (<-U)
echo >&2 "Testing whether \"#PATCHNAME#\" patch for $KVERSION applies (dry run):"
if ! [ -r $PATCHFILE ]
then
    echo >&2 "\"#PATCHNAME#\" patch for $KVERSION not found"
    exit 1
elif ! $DECOMPRESSOR $PATCHFILE |
        patch --force --dry-run $PATCH_OPTIONS -p$STRIPLEVEL
then
    echo >&2 "\"#PATCHNAME#\" patch for $KVERSION does not apply cleanly"
    exit 1
fi

We do not use --force on second run, there should be no need for it. If something requires interaction, it is likely there is a bug somewhere, better catch it.

After applying the diff, we remove any empty files, so that files "removed" by the diff are really removed. We make an exception for ./debian/ contents, and for files named APPLIED*, or else some (buggy) stamp files may be caught too.

<Apply the patch>= (<-U)
if ! $DECOMPRESSOR $PATCHFILE |
        patch $PATCH_OPTIONS -p$STRIPLEVEL
then
    # This should never happen, thanks to the dry-run
    echo >&2 "ASSERTION FAILED - \"#PATCHNAME#\" patch for $KVERSION failed"
    echo >&2 "END applying patch \"#PATCHNAME#\""
    exit 1
fi
echo >&2 "\"#PATCHNAME#\" patch for $KVERSION succeeded"

<Remove empty files>

All information necessary for unpatching is stored in a debian/APPLIED_* file, for use by the unpatch script.

<Store necessary info for unpatching>= (<-U)
mkdir -p debian
cat > 'debian/APPLIED_#PATCHARCH#_#PATCHID#' <<EOF
PATCHFILE='$PATCHFILE'
STRIPLEVEL='$STRIPLEVEL'
DEPENDS='$UNPATCHDEPS'
EOF
<Create script to have this patch registered in doc-dir>= (<-U)
mkdir -p debian/image.d
PKGNAME=`dpkg -S $PATCHFILE | cut -d: -f1`
PKGVERS=`grep-dctrl -n -P $PKGNAME -s Version -X /var/lib/dpkg/status`
cat > 'debian/image.d/register-#PATCHID#' <<EOF
#!/bin/sh

# This scripts documents the "#PATCHNAME#" kernel patch into the
# kernel-image package, as being applied to the kernel.

docdir=\${IMAGE_TOP}/usr/share/doc/kernel-image-\${version}

mkdir -p \${docdir}

(
    printf '#PATCHNAME# (#PATCHID#)${KPATCH_#CLEANPATCHID#:+ for kernel ${KPATCH_#CLEANPATCHID#}},'
    echo ' from package $PKGNAME, version $PKGVERS'
) >> \${docdir}/applied-patches
EOF
chmod +x 'debian/image.d/register-#PATCHID#'

Generated by GNU enscript 1.6.4.
<Apply.tmpl constants>= (<-U)
TOPPATCHDIR=/usr/src/kernel-patches
ARCHITECTURE=`dpkg --print-installation-architecture`
DECOMPRESSOR="zcat -f"
PATCH_OPTIONS="--ignore-whitespace --silent"
# This is informational only, used by lskpatches
DHPKPATCHES_VERSION=#DHKPVERS#

Generated by GNU enscript 1.6.4.

We consider it a success if the patch is already applied.

FIXME: we should make sure that the currently-applied patch is the one that would have been applied.

<Make apply idempotent>= (<-U)
[ -f debian/APPLIED_${ARCHITECTURE}_#PATCHID# -o \
  -f debian/APPLIED_all_#PATCHID# ] && exit 0

Generated by GNU enscript 1.6.4.

Unpatch script

<unpatch.tmpl>=
#! /bin/bash
set -e

# This "unpatch" script was automatically generated by
# dh_installkpatches from unpatch.tmpl $Revision: 1.17.1.4.1.23 $

<Assert this is a kernel tree>

ARCHITECTURE=`dpkg --print-installation-architecture`
PATCHID=#PATCHID#
PATCHARCH=#PATCHARCH#

TOPPATCHDIR=/usr/src/kernel-patches
DECOMPRESSOR="zcat -f"

STAMP=debian/APPLIED_${PATCHARCH}_$PATCHID

PATCH_OPTIONS="--ignore-whitespace --silent"

[ -f $STAMP ] || exit 0

<Assert stamp file is not empty>
<Read contents from stamp file>
<Assert diff file is still there>
<Do not unpatch if there are still dependants>

<Remove the patch>

<Assert stamp file is not empty>= (<-U)
if [ ! -s $STAMP ]
then
    cat >&2 <<EOF
ERROR: "#PATCHNAME#" patch was applied using old mechanism.
The relevant patch file may even not be on your system any more.
I strongly suggest you remove this kernel tree and unpack a clean one.
EOF
    exit 1
fi

<Read contents from stamp file>= (<-U)
if [ $(echo $(wc -l < $STAMP) ) != 1 ]
then
    . $STAMP
else
    # old format
    APPLY_INFO=$(cat $STAMP)
    PATCHFILE=$(echo ${APPLY_INFO} | cut -d# -f1)
    STRIPLEVEL=$(echo ${APPLY_INFO} | cut -d# -f2)
fi

<Assert diff file is still there>= (<-U)
if [ ! -r $PATCHFILE ]
then
    cat >&2 <<EOF
ERROR: applied "#PATCHNAME#" patch file could not be found.
Presumably the package containing it was removed or upgraded.
I strongly suggest you remove this kernel tree and unpack a clean one.
EOF
    exit 1
fi

<Remove those patches we depended on>= (U->)
if [ -n "$DEPENDS" ]
then
    echo >&2 "Also un-applying the following patches: $DEPENDS"
    for dep in $DEPENDS
    do
        if [ -x ${TOPPATCHDIR}/$dep ]
        then
            ${TOPPATCHDIR}/$dep
        else
            echo >&2 "Could not unpatch dependency: $dep - stopping here."
            echo >&2 "END unpatching patch \"#PATCHNAME#\""
            exit 1
        fi
    done
fi

<Do not unpatch if there are still dependants>= (<-U)
if grep -l "^DEPENDS=.*[' ]#PATCHARCH#/unpatch/#PATCHID#[' ]" debian/APPLIED_* >/dev/null 2>/dev/null
then
    # There are patches depending on this one, to be removed before.
    # the dep will be removed by the last of those other patches.
    echo >&2 "NOT unpatching \"#PATCHNAME#\" since other patches still rely on it"
    exit 0
fi

<Remove the patch>= (<-U)
echo >&2 "START unpatching patch \"#PATCHNAME#\""

$DECOMPRESSOR $PATCHFILE | patch -R -p$STRIPLEVEL $PATCH_OPTIONS
rm -f debian/APPLIED_${PATCHARCH}_$PATCHID debian/image.d/register-#PATCHID#

<Remove those patches we depended on>
<Remove empty files>
rmdir -p debian/image.d || true

echo >&2 "END unpatching patch \"#PATCHNAME#\""

Common stuff

<Assert this is a kernel tree>= (<-U <-U)
if ! [ -d kernel -a -d Documentation ]
then
    echo >&2 "Not in kernel top level directory. Exiting"
    exit 1
fi
<Remove empty files>= (<-U <-U)
echo >&2 "Removing empty files:"
# make an exception for ./debian, or else the stamp files will go too.
find . -path ./debian -prune -o \
       -type f -size 0 ! -name 'APPLIED*' -exec rm {} \; -print
echo >&2 "Done."

Reading a kpatches control file

When reading a kpatches control file, we store its contents according a data model suitable for the most recent file-format, and will map data extracted from old file-formats into the modern one.

Generic (v1) data structures

The v1 data structures are a hash of general fields taken from the initial stance (eg. PatchName), a hash of default values also taken from the initial stance, and a sequence of alternative versions of the patch, each of which containing a sequence of operations to apply to the kernel tree:

<initialise %patchinfo with skeleton structure>= (U->)
my %patchinfo = ('general' => {},
                 'defaults' => {},
                 'alternatives' => []);

We provide a single entry-point to the controlfile reader, which will act in different ways according to the format-version.

If we have a v0 header, we migrate the relevant contents of the v0 header into the default-values section ot the %patchinfo structure, before going on reading.

<definitions for the core kpatch system>+= (<-U) [<-D->]
sub read_control_file {
  my ($file) = @_;

  <initialise %patchinfo with skeleton structure>

  open (IN, $file) or die "cannot open $file: $!";

  # read control-file header
  read_control_file_section ($patchinfo{general});

  if ((!defined $patchinfo{general}->{'kpatch-format-version'}) or
      ($patchinfo{general}->{'kpatch-format-version'} == 0)) {
    $patchinfo{defaults} = control_file_v0_header_to_v1_defaults ($patchinfo{general});
    <read v0 file content>

    # } elsif ($patchinfo{general}->{'kpatch-format-version'} == 1) {
    # Revision 1
    # Eh, not yet :)
  } else {
    die "Unsupported Kpatch-format-version: \`" .
      $patchinfo{general}->{'kpatch-format-version'} . "'";
  }

  close IN;

  validate (\%patchinfo);
  return %patchinfo;
}

Once a %patchinfo structure has been filled, we can run some checks on it, to make as sure as possible that it is correct.

FIXME: this should do much more tests !

<definitions for the core kpatch system>+= (<-U) [<-D->]
sub validate {
  my ($patchinfo) = @_;

  die "Patch-Id can only contain alphanumerics, hyphens, and underscores"
    if $patchinfo->{general}->{'patch-id'} =~ /[^\w-]/;

  foreach my $alternative (@{$patchinfo->{alternatives}}) {
    foreach my $operation (@{$alternative->{operations}}) {
      die "Diff file does not exist: " . $operation->{'diff-file'}
        if ($operation->{format} eq 'diff') and ! -r $operation->{'diff-file'};
      die "Diff file changes EXTRAVERSION: " . $operation->{'diff-file'}
        if 0 == system ('grep -q "^-EXTRAVERSION\>" ' . $operation->{'diff-file'});
    }
  }
}

Reading a control-file section is something quite generic. Maybe there is a generic API for doing that now ?

This function slightly adapted from code in install-docs from Adam Di Carlo, and this is the main reason why it is not split into chunks.

<definitions for the core kpatch system>+= (<-U) [<-D->]
sub read_control_file_section {
  my ($pfields) = @_;

  my $alreadyreadsomething = 0;
  my ($key,$value);
  while (<IN>) {
    chomp;

    # empty line?
    if (/^\s*$/o) {
      if ($alreadyreadsomething) {
        last;
      } else {
        next;
      }
    }

    $alreadyreadsomething = 1;

    if (/^(\S+)\s*:\s*(.*)$/) {
      # first line of a new field

      ($key,$value) = (lc $1,$2);
      #print STDERR "$key -> $value\n";
      if (exists $pfields->{$key}) {
        warn "warning: $key: overwriting previous setting of control field";
      }
      $pfields->{$key} = $value;

    } elsif (/^\s+(\S.*)$/) {
      # additional line in a multi-line field
      $value = $1;
      defined($key) or die "syntax error in control file: no field specified";
      #print STDERR "$key -> $value (continued)\n";
      $pfields->{$key} .= "\n$value";

    } else {
      die "syntax error in control file: $_";
    }
  }

  return $alreadyreadsomething;
}

Handling v0 definitions

A v0 kpatches file is just a list of alternatives, each composed of only one diff operation. We have to convert them into proper structures so that the v1-based processing works as expected.

<read v0 file content>= (<-U)
my $cfs = {};
while (read_control_file_section ($cfs)) {
  push (@{$patchinfo{alternatives}},
        control_file_v0_section_to_alternative($cfs));
  $cfs = {};
}

The function that does the conversion simply maps known fields into their final location.

<definitions for the core kpatch system>+= (<-U) [<-D->]
sub control_file_v0_section_to_alternative {
  my ($cfs) = @_;

  # FIXME: should also process general section, and convert default
  # values - probably in a similar function

  return {
          conditions => {
                         'kernel-version' => $cfs->{'kernel-version'},
                         'architecture' => $cfs->{architecture},
                        },
          depends => $cfs->{depends},
          defaults => {},
          operations => [
                         {
                          'format' => 'diff',
                          'diff-file' => $cfs->{'patch-file'},
                          'debian-diff-file' => $cfs->{'debian-patch-file'},
                          'path-strip-level' => $cfs->{'path-strip-level'},
                         }
                        ],
         };
}

The v0 header differs from the v1 header, in that "default" entries are implicit, and for a small well-defined set of fields. When reading a v0 header, we have to explicit them, and strip them from the general fields.

<definitions for the core kpatch system>+= (<-U) [<-D->]
sub control_file_v0_header_to_v1_defaults {
  my ($header) = @_;
  my %defaults;

  foreach my $key (keys %{$header}) {
    if (! grep { $key eq $_ } ('patch-name', 'patch-id')) {
      $defaults{$key} = $header->{$key};
      delete $header->{$key};
    }
  }

  return \%defaults;
}

Fields of kpatch files

There are 2 types of fields, denoted by constants defined the fields library. Some are mandatory, some are optional. The optional ones can all be given a default value, but mandatory ones cannot use this inheritance mechanism -- hence the $FIELD_INHERITS name.

Due to this inheritance mechanism, the final value of a field has to be computed according to given rules for inheritance and defaulting.

From text fields to data structures

Generic alternative fields

We record all archs mentionned in the kpatches file, so that we know for which archs we have to generate the apply/unpatch scripts.

<compute and split 'architecture' field>= (U->)
{
 my @archfield =
 split (/, */, field_value ($patchinfo{general}, $alternative->{conditions},
                            'architecture', $FIELD_INHERITS, 'all',
                            \%patchinfo));
 $alternative->{conditions}->{architecture} = \@archfield;

 foreach my $arch (@archfield) {
   push @archs, $arch unless grep { $_ eq $arch } @archs;
 }
}

The kernel version can be specified either as a single version or as a range.

FIXME: how a is value "all" handled ? It seems filtered out by the test below...

<compute 'kernel-version' value or range>= (U->)
{
  my $kversion = field_value ($patchinfo{general}, $alternative->{conditions},
                              'kernel-version', $FIELD_MANDATORY);
  # parse "2.4.5 - 2.4.7" and "2.5.4 -" syntaxes

  my @kv = split (/\s+/, $kversion);
  if ($#kv > 0) {
    # FIXME: validity check is really too strict, but we need a
    # good kversion comparison algorithm to attempt any better
    # (ie. "-pre" and "-test" at least are special)
    $kv[0] =~ m/^(\d+\.\d+)\.(\d+)$/ or die "Malformed kernel version: `$kv[0]'";

    my ($branch, $first) = ($1, $2);
    die "Malformed kernel-version range \`$kversion'"
      unless ($kv[1] eq '-') && ($#kv <= 2);
    if ($#kv == 1) {
      die "Unbounded ranges not supported yet: \`$kversion'";
      $kversion = [ $branch, $first ];
    } else {
      $kv[2] =~ m/^(\d+\.\d+)\.(\d+)$/ or die "Malformed kernel version: `$kv[2]'";
      die "Cross-branch ranges are not allowed: `$kversion'"
        unless $1 == $branch;
      die "Reverse-ordered range: `$kversion'" if $2 < $first;
      $kversion = [ $branch, $first, $2 ];
    }
  } else {
    $kv[0] =~ m/^(\d+\.\d+)\.(\d+)/ or die "Malformed kernel version: `$kv[0]'";
  }

  $alternative->{conditions}->{'kernel-version'} = $kversion;
}

Depends is a comma- and space-separated list, and will be used in the bash apply script to initialize an array, so must be made space-separated only.

<compute 'depends' field>= (U->)
$alternative->{depends} = field_value ($patchinfo{general}, $alternative,
                                       'depends', $FIELD_INHERITS, "",
                                       \%patchinfo);
$alternative->{depends} =~ s/, */ /g;

Per-operation fields

This is simplest of all fields, nothing special is done apart from memorizing it.

<compute 'path-strip-level' field for $op>= (U->)
$op->{'path-strip-level'} = field_value ($patchinfo{general}, $op,
                                         'path-strip-level', $FIELD_INHERITS, 1,
                                         $alternative, \%patchinfo);

Since the diff files may be compressed, we are only recording here their location in the source tree, and we are delaying the recording of their final filename until they are installed and maybe compressed. This will be done as we <install the diff files>.

<compute (debian-)diff-file fields for $op>= (U->)
$op->{'diff-file'} = field_value ($patchinfo{general}, $op,
                                  'diff-file', $FIELD_MANDATORY);
$op->{'debian-diff-file'} = field_value ($patchinfo{general}, $op,
                                         'debian-diff-file', $FIELD_INHERITS);

Field-handling library

Field inheritance

<definitions for the core kpatch system>+= (<-U) [<-D->]
my $FIELD_MANDATORY = 0;
my $FIELD_INHERITS = 1;

$inherits should have one of the values defined above. $default can be undef when there is no default value for the field.

<definitions for the core kpatch system>+= (<-U) [<-D->]
sub field_value {
  my ($general, $hash, $name, $inherits, $default, @defaultlists) = @_;

  my $value = $hash->{$name};
  if ($inherits == $FIELD_MANDATORY) {
    die "Patchfile info lacks $name field" unless defined $value;
  }
  # first go through explicit default values
  foreach my $defaultlist (@defaultlists) {
    $value = $defaultlist->{defaults}->{$name} unless defined $value;
  }
  # then use hardcoded default as a fallback
  if (defined $default) {
    $value = $default unless defined $value;
  }

  return $value;
}

Recording a patchfile field for later use

We use hashes to store sequences of values for each arch. These sequences of values will be directly used to produce the apply/unpatch scripts from their templates.

<definitions for the core kpatch system>+= (<-U) [<-D]
# records a field value for a given patchfile on a given arch
sub record_patchfile_field {
  my ($hash, $arch, $value) = @_;

  if (defined $hash->{$arch}) {
    $hash->{$arch} .= " $value";
  } else {
    $hash->{$arch} = "$value";
  }
}

Patch processing

In a single apply script, all alternatives are handled, and the one to use will be determined at runtime.

We currently support only one operation per alternative (v0 format), and a great deal of this restriction lies in this part, so I will not spend to much time structuring this part, until things are properly implemented.

To get proper v1 support we will put everything including kpatches file parsing into a generic apply script, and have patch packages only symlink to this one.

<generate apply/unpatch scripts for given $package and kpatches $file>= (<-U)
my %kversions=();
my %patchfiles=();
my %debpatchfiles=();
my %striplevels=();
my %depends=();
my @archs=();

# put the right files in the right places
foreach my $alternative (@{$patchinfo{alternatives}}) {
  my $op = $alternative->{operations}->[0];
  <compute values for $alternative>
  <install the diff files>
  <record the $alternative and single $op>
}
<output apply/unpatch scripts>

For each alternative, we compute the value to use for each field, possibly using a default value if none was provided, and in some cases splitting into an array. The final value is stored in the place of the original one.

Diff files thus identified are then installed, and entries for the diff files are then recorded for use when generating the apply/unpatch scripts for each of the named architectures.

There is no particular ordering required between the computation of the various fields, but to help moving towards v1 format, we group them by first handling those specific to the alternative itself, and then those specific to the single diff operation in this alternative.

<compute values for $alternative>= (<-U)
<compute 'depends' field>
<compute and split 'architecture' field>
<compute 'kernel-version' value or range>

<compute 'path-strip-level' field for $op>
<compute (debian-)diff-file fields for $op>

All diff files are now located under a single /usr/src/kernel-patches/diffs/ tree.

<install the diff files>= (<-U)
my $srcdir = "/usr/src/kernel-patches/diffs/$patchid";
doit ("mkdir",  "-p", "$tmp$srcdir") unless -d "$tmp$srcdir";

$op->{'installed-diff-file'} = "$srcdir/" . basename($op->{'diff-file'});
doit ("cp", $op->{'diff-file'}, "$tmp$op->{'installed-diff-file'}");
doit ("gzip", "-9f", "$tmp$op->{'installed-diff-file'}");
$op->{'installed-diff-file'} = "$op->{'installed-diff-file'}.gz"
  if -r "$tmp$op->{'installed-diff-file'}.gz";

if (defined $op->{'debian-diff-file'}) {
  $op->{'installed-debian-diff-file'} = "$srcdir/" . basename($op->{'debian-diff-file'});
  doit ("cp", $op->{'debian-diff-file'}, "$tmp$op->{'installed-debian-diff-file'}");
  doit ("gzip", "-9f", "$tmp$op->{'installed-debian-diff-file'}");
  $op->{'installed-debian-diff-file'} = "$op->{'installed-debian-diff-file'}.gz"
    if -r "$tmp$op->{'installed-debian-diff-file'}.gz";
} else {
  $op->{'installed-debian-diff-file'} = '';
}

Version ranges are currently emulated, by duplicating the entry for each integer kernel revision in the range.

This is a temporary behaviour, which does not allow pre-release kernel versions to be seen as part of the range. Adequate version-comparison function and data-structures will be used instead, when time permits.

<record the $alternative and single $op>= (<-U)
foreach my $arch (@{$alternative->{conditions}->{architecture}}) {
  if ((ref $alternative->{conditions}->{'kernel-version'}) eq 'ARRAY') {
    for (my $v = $alternative->{conditions}->{'kernel-version'}->[1];
         $v <= $alternative->{conditions}->{'kernel-version'}->[2];
         $v++) {
      record_patchfile_field (\%kversions, $arch, 
                              $alternative->{conditions}->{'kernel-version'}->[0] . '.' . $v);
      record_patchfile_field (\%striplevels, $arch, $op->{'path-strip-level'});
      record_patchfile_field (\%depends, $arch, '"' . $alternative->{depends} . '"');
      record_patchfile_field (\%patchfiles, $arch, '"' . $op->{'installed-diff-file'} . '"');
      record_patchfile_field (\%debpatchfiles, $arch, '"' . $op->{'installed-debian-diff-file'} . '"');
    }
  } else {
    record_patchfile_field (\%kversions, $arch, 
                            $alternative->{conditions}->{'kernel-version'});
    record_patchfile_field (\%striplevels, $arch, $op->{'path-strip-level'});
    record_patchfile_field (\%depends, $arch, '"' . $alternative->{depends} . '"');
    record_patchfile_field (\%patchfiles, $arch, '"' . $op->{'installed-diff-file'} . '"');
    record_patchfile_field (\%debpatchfiles, $arch, '"' . $op->{'installed-debian-diff-file'} . '"');
  }
}

apply and unpatch are currently generated from templates, by substituting in there the values computed from the patch definition expressed in the kpatches file.

<output apply/unpatch scripts>= (<-U)
foreach my $arch (@archs) {
  my $pdir = "/usr/src/kernel-patches/$arch";
  foreach my $script ('apply', 'unpatch') {
    doit ("mkdir", "-p", "$tmp$pdir/$script");
    complex_doit ('sed < @TMPLDIR@/' . "$script.tmpl >$tmp$pdir/$script/$patchid" .
                  ' -e \'s/#PATCHID#/' . $patchinfo{general}->{'patch-id'} . '/g\'' .
                  ' -e \'s/#CLEANPATCHID#/' . $patchinfo{general}->{'clean-patch-id'} . '/g\'' .
                  ' -e \'s|#PATCHNAME#|' . $patchinfo{general}->{'patch-name'} . '|g\'' .
                  " -e 's/#DHKPVERS#/@DHKPVERS@/g'" .
                  " -e 's,#TMPLDIR#,@TMPLDIR@,g'" .
                  " -e 's/#DEPENDS#/$depends{$arch}/g'" .
                  " -e 's/#KVERSIONS#/$kversions{$arch}/g'" .
                  " -e 's|#PATCHFILES#|$patchfiles{$arch}|g'" .
                  " -e 's|#DEBPATCHFILES#|$debpatchfiles{$arch}|g'" .
                  " -e 's/#PATCHARCH#/$arch/g'" .
                  " -e 's/#STRIPLEVELS#/$striplevels{$arch}/g'"
                 );
    doit ("chmod", "0755", "$tmp$pdir/$script/$patchid");

    doit ("mkdir", "-p", "$tmp/usr/share/doc/$package");
    doit ('cp',
          '@TMPLDIR@/README-kernelpatch.Debian',
          "$tmp/usr/share/doc/$package/");
  }
}

Index of chunks

*