Chapter 3. Debian package management

Table of Contents

3.1. Basics of Debian package management
3.1.1. Package configuration
3.1.2. Basic precautions
3.1.3. Life with eternal upgrades
3.1.4. Debian archive basics
3.1.5. Package dependencies
3.1.6. The event flow of the Debian package management
3.1.7. Basics of using aptitude
3.1.7.1. Key bindings of aptitude
3.1.7.2. Package views under aptitude
3.1.7.3. The command line use of aptitude
3.1.7.4. The aptitude regex formula
3.1.7.5. Search method options with aptitude
3.1.8. Examples for using aptitude
3.1.8.1. List packages with regex matching package name
3.1.8.2. Browse with the regex matching
3.1.8.3. Tidy auto/manual install status
3.1.8.4. Purge removed packages for good
3.1.9. System wide upgrade with aptitude
3.1.10. Aptitude advantages
3.1.11. Package activity logs
3.1.12. Package maintenance tools (low-level)
3.1.13. Search on the package meta data
3.1.14. The advanced system maintenance
3.1.15. Verify installed package files
3.1.16. Safeguard for package problems
3.1.17. What to do when you face some package problems
3.2. Debian package management internals
3.2.1. Archive meta data
3.2.2. Top level Release file and authenticity
3.2.3. Archive level Release files
3.2.4. Fetching of the meta data for the package
3.2.4.1. The package state for APT
3.2.4.2. The package state for aptitude
3.2.5. The local copies of the fetched packages
3.2.6. The Debian package file name
3.2.7. The dpkg command
3.2.8. The update-alternative command
3.2.9. The dpkg-statoverride command
3.2.10. The dpkg-divert command
3.3. Recovery from a broken system
3.3.1. Incompatibility with old user configuration
3.3.2. Different packages with overlapped files
3.3.3. Fixing broken package script
3.3.4. Rescue using the dpkg command
3.3.5. Recover package selection data
3.4. Tips for the package management
3.4.1. How to pick Debian packages
3.4.2. Packages from mixed source of archives
3.4.3. Tweaking candidate version
3.4.4. Volatile and Backports.org
3.4.5. Emergency downgrading
3.4.6. Who uploaded the package?
3.4.7. The equivs package
3.4.8. Port a package to the stable system
3.4.9. Small public package archive
3.4.10. Record/copy system configuration
3.4.11. Convert or install an alien binary package
3.4.12. Extract package without dpkg
3.4.13. More readings for the package management

Debian is a volunteer organization which builds consistent distributions of pre-compiled binary packages of free software and distributes them from its archive.

The Debian archive is offered by many remote mirror sites for access through HTTP and FTP methods. It is also available as CD-ROM/DVD.

The Debian package management system, when used properly, offers the user to install consistent sets of binary packages to the system from the archive. Currently, there are 23720 packages available for the amd64 architecture.

The Debian package management system has a rich history and many choices for the front end user program and back end archive access method to be used. Currently, we recommend aptitude as the main front end program for the Debian package management activity.

Table 3.1.  List of the Debian package management tools

package

popcon

size

description

aptitude

V:21, I:98

8976

terminal-based package manager (current standard, front-end for apt)

apt

V:85, I:99

5180

Advanced Packaging Tool, front-end for dpkg providing "http", "ftp", and "file" archive access methods (apt-get/apt-cache commands included)

tasksel

V:5, I:93

884

tool for selecting tasks for installation on Debian system (front-end for apt)

dselect

V:7, I:89

2176

terminal-based package manager (previous standard, front-end for apt and other old access methods)

dpkg

V:86, I:99

6920

package maintenance system for Debian

dpkg-ftp

V:0.12, I:0.6

136

older ftp method for dselect

synaptic

V:20, I:53

5844

graphical package manager (Gnome front-end for apt)

gnome-apt

V:0.2, I:1.2

604

graphical package manager (Gnome front-end for apt)

kpackage

V:5, I:15

1496

graphical package manager (KDE front-end for apt)

apt-utils

V:51, I:99

456

APT utility programs: apt-extracttemplates(1), apt-ftparchive(1), and apt-sortpkgs(1)

apt-listchanges

V:3, I:6

241

package change history notification tool

apt-listbugs

V:1.7, I:2

428

lists critical bugs before each APT installation

apt-file

V:1.9, I:8

140

APT package searching utility -- command-line interface

apt-rdepends

V:0.18, I:0.9

92

recursively lists package dependencies


[Note] Note

The annoying bug #411123 for the mixed use of aptitude and apt-get commands has been resolved. If this kept you from using aptitude, please reconsider.

3.1. Basics of Debian package management

3.1.1. Package configuration

Here are some key points for package configuration on the Debian system:

  • The manual configuration by the system administrator is respected. In other words, the package configuration system makes no intrusive configuration for the sake of convenience.

  • Each package comes with its own configuration script with standardized user interface called debconf(7) to help initial installation process of the package.

  • Debian Developers try their best to make your upgrade experience flawless with package configuration scripts.

  • Full functionalities of packaged software are available to the system administrator. But ones with security risks are disabled in the default installation.

  • If you manually activate a service with some security risks, you are responsible for the risk containment.

  • Esoteric configuration may be manually enabled by the system administrator. This may creates interference with popular generic helper programs for the system configuration.

3.1.2. Basic precautions

[Warning] Warning

Do not install packages from random mixture of suites. It will likely break the package consistency which requires deep system management knowledge, such as compiler ABI, library version, interpreter features, etc.

The newbie Debian system administrator should stay with the stable release of Debian while applying only security updates. I mean that some of the valid actions are better avoided, as a precaution, until you understand the Debian system very well:

  • Do not include testing or unstable in /etc/apt/sources.list,

  • Do not mix standard Debian with other non-Debian archives such as Ubuntu in /etc/apt/sources.list,

  • Do not create /etc/apt/preferences,

  • Do not change default behavior of package management tools through configuration files without knowing their full impacts,

  • Do not install random packages by "dpkg -i <random_package>",

  • Do not ever install random packages by "dpkg --force-all -i <random_package>",

  • Do not erase or alter files in /var/lib/dpkg/, or

  • Do not overwrite system files by installing software programs directly compiled from source. (Install them into /usr/local or /opt.)

The non-compatible effects caused by above actions to the Debian package management system may leave your system unusable.

The serious Debian system administrator who runs mission critical servers, should use extra precautions:

  • Do not install any packages including security updates from Debian without thoroughly testing them with your particular configuration under safe conditions. (Although Debian has been offering an extremely stable system for a long time, you as the system administrator are responsible for your system in the end.)

3.1.3. Life with eternal upgrades

Despite my warnings above, I know many readers of this document wish to run the "testing" or "unstable" suites of Debian as their main Desktop system since they work very well for self-administered Desktop environments. Because they are updated frequently, they offer the latest features.

[Caution] Caution

For your production server, the "stable" suite with the security updates is recommended. The same can be said for desktop PCs on which you can spend limited administration efforts, e.g. for your mother's PC.

It takes no more than simply setting the distribution string in the /etc/apt/sources.list to the suite: "testing" or "unstable"; or the codename: "lenny" or "sid". This will let you live the life of eternal upgrades.

The use of "testing" or "unstable" is a lot of fun but comes with some risks. Even though the "unstable" suite of Debian system looks very stable for most of the times, there have been some package problems on the "testing" and "unstable" suite of Debian system and a few of them were not so trivial to resolve. It may be quite painful for you. Sometimes, you may have a broken package or missing functionality for a few weeks.

Here are some ideas to ensure quick and easy recovery from bugs in Debian packages:

  • make the system dual bootable by installing the "stable" suite of Debian system to another partition.

  • make the installation CD handy for the rescue boot.

  • consider installing apt-listbugs to check the Debian Bug Tracking System (BTS) information before the upgrade.

  • learn the package system infrastructure enough to work around the problem.

  • create a chroot or similar environment and run the latest system in it in advance. (optional)

(If you can not do any one of these precautionary actions, you are probably not ready for the "testing" and "unstable" suites.)

Enlightenment with the following will save a person from the eternal karmic struggle of upgrade hell and let him reach Debian nirvana.

3.1.4. Debian archive basics

Let's look into the Debian archive from a system user's perspective. For the typical HTTP access, the archive is specified in the /etc/apt/sources.list file as, e.g. for the current "stable" == "etch" system:

deb http://ftp.us.debian.org/debian/ etch main contrib non-free
deb-src http://ftp.us.debian.org/debian/ etch main contrib non-free

deb http://security.debian.org/ etch/updates main contrib
deb-src http://security.debian.org/ etch/updates main contrib

Here, I tend to use codename "etch" instead of suite name "stable" to avoid surprises when the next "stable" is released.

The meaning of this is described in "man 5 sources.list" and key points are:

  • The "deb" line defines for the binary packages.

  • The "deb-src" line defines for the source packages.

  • The 1st argument is the root URL of the Debian archive.

  • The 2nd argument is the distribution: either the suite name or the codename.

  • The 3rd and following arguments are the list of valid archive component names of the Debian archive.

The "deb-src" lines can safely be omitted (or commented out by placing "#" at the start of the line) if it is just for aptitude which does not access source related meta data. It will speed up the updates of the archive meta data. The URL can be "http://", "ftp://", "file://", ....

[Tip] Tip

If "sid" is used in the above example instead of "etch", the "deb: http://security.debian.org/ ..." line for security updates in the /etc/apt/sources.list is not required. Security updates are only available for "stable" and "testing" (i.e., "etch" and "lenny").

Here are the lists of URL of the Debian archive sites and suite or codename used in the configuration file:

Table 3.2.  Lists of Debian archive sites.

archive URL

suite (codename)

purpose

http://ftp.us.debian.org/debian/

stable (etch)

stable (etch) release

http://ftp.us.debian.org/debian/

testing (lenny)

testing (lenny) release

http://ftp.us.debian.org/debian/

unstable (sid)

unstable (sid) release

http://ftp.us.debian.org/debian/

experimental

experimental pre-release (optional, only for developer)

http://ftp.us.debian.org/debian/

stable-proposed-updates

Updates for the next stable point release (optional)

http://security.debian.org/

stable/updates

Security updates for stable release (important)

http://security.debian.org/

testing/updates

Security updates for testing release (important)

http://volatile.debian.org/debian-volatile/

volatile

Compatible updates for spam filter and IM clients, etc.

http://volatile.debian.org/debian-volatile/

volatile-sloppy

Non-compatible updates for spam filter, IM clients, etc.

http://backports.org/debian/

etch-backports

Newer backported packages for etch. (non-official, optional)


[Tip] Tip

For the Debian system with the "stable" and "testing" suites, it is a good idea to include lines with http://security.debian.org/ in the /etc/apt/sources.list to enable access to the archive for security updates as in the example above.

[Caution] Caution

Only pure stable release with security updates provides the best stability. Running mostly stable release mixed with some packages from testing or unstable release is riskier than running pure unstable release. If you really need the latest version of some programs under stable release, please use packages from the debian-volatile project and http://backports.org/ (see: Section 3.4.4, “Volatile and Backports.org”) services. These services must be used with extra care.

[Caution] Caution

You should basically list only one of "stable", "testing", or "stable" suites in the "deb" line. If you list any combination of "stable", "testing", and "stable" suites in the "deb" line, APT programs slow down while only the latest archive is effective. Multiple listing makes sense for these when the /etc/apt/preferences file is used with clear objectives (see: Section 3.4.3, “Tweaking candidate version”).

Each Debian archive consists of 3 components. Components are alternatively called categories in "Debian Policy" or areas in "Debian Social Contract". The component is grouped by the compliance to "The Debian Free Software Guidelines " (DFSG):

Table 3.3.  The lists of Debian archive components.

component

number of packages

criteria

main

23071

The package is fully compliant to DSFG and does not depend the non-free package.

contrib

293

The package is compliant to the DSFG but depends on the non-free package.

non-free

356

The package is not compliant to the DSFG but distributable and useful.


Here the number of packages in the above is for the amd64 architecture. Strictly speaking, only the main component archive shall be considered the Debian system.

The Debian archive organization can be studied best by pointing your browser to the each archive URL appended with dists or pool.

The distribution is referred by two ways, the suite or codename. The word distribution is alternatively used as the synonym to the suite in many documentation. The relationship between the suite and the codename can be summarized as:

Table 3.4.  The relationship between suite and codename.

Timing

suite = "stable"

suite ="testing"

suite ="unstable"

current (as of 2008-08-31 13:52:02 UTC)

codename = "etch"

codename = "lenny"

codename = "sid"

after the "lenny" release

codename = "lenny"

codename = "lenny+1"

codename = "sid"


The history of codenames are described in Debian FAQ: 6.3.1 Which other codenames have been used in the past?

In the stricter Debian archive terminology, the word "section" is specifically used for the categorization of packages by the application area. (Although, the word "main section" may sometimes be used to describe the Debian archive section which provides the main component.)

Every time a new upload is done by the Debian developer (DD) to the "unstable" archive (via incoming processing), DD is required to ensure uploaded packages to be compatible with the latest set of packages in the latest "unstable" archive.

If DD breaks this compatibility intentionally for important library upgrade etc, there is usually announcement to the debian-devel mailing list etc.

Before a set of packages are moved by the Debian archive maintenance script from the "unstable" archive to the "testing" archive, the archive maintenance script not only checks the maturity (about 10 days old) and the status of the RC bug reports for the packages but also tries to ensure them to be compatible with the latest set of packages in the "testing" archive. This process makes the "testing" archive very current and usable.

Through the gradual archive freeze process led by the release team, the "testing" archive will be matured to make it completely consistent and bug free with some manual interventions. Then the new "stable" release is created by assigning the codename for the old "testing" archive to the new "stable" archive and creating the new codename for the new "testing" archive. The initial contents of the new "testing" archive is exactly the same as that of the newly released "stable" archive.

Both the "unstable" and the "testing" archives may suffer temporary glitches due to:

  • broken package upload to the archive (mostly for "unstable"),

  • delay of accepting the new packages to the archive (mostly for "unstable"),

  • archive synchronization timing issue (both for "testing" and "unstable"),

  • manual intervention to the archive such as package removal (more for "testing"), etc.

So if you ever decide to use these archives, you should be able to fix or work around these kinds of glitches.

[Tip] Tip

When tracking the "testing" archive, problem caused by a removed package is usually worked around by installing corresponding package from the "unstable" archive which is uploaded for bug fix.

See Debian Policy Manual for definition of:

3.1.5. Package dependencies

The Debian system offers a consistent set binary packages through its versioned binary dependency declaration mechanism through the control file fields. Here is a bit over simplified definition for them.

Table 3.5.  List of package dependencies.

dependency

meaning

Depends

This declares an absolute dependency and all of the packages listed in this field must be installed at the same time or in advance.

Pre-Depends

This is like Depends, except that it requires completed installation of the listed packages in advance.

Recommends

This declares a strong, but not absolute, dependency. Most users would not want the package unless all of the packages listed in this field are installed.

Suggests

This declares a weak dependency. Many users of this package may benefit from installing packages listed in this field but can have reasonable functions without them.

Enhances

This declares a week dependency like Suggests but works in the opposite direction.

Conflicts

This declares an absolute incompatibility. All of the packages listed in this field must be removed to install this package.

Replaces

This is declared when files installed by this package replace files in the listed packages.

Provides

This is declared when this package provide all of the files and functionality in the listed packages.


[Note] Note

Please note that defining, Provides, Conflicts and Replaces simultaneously to an virtual package is the sane configuration. This ensures that only one real package providing this virtual package can be installed at any one time.

The official definition including source dependency can be found in the Policy Manual: Chapter 7 - Declaring relationships between packages.

3.1.6. The event flow of the Debian package management

The simplified event flow of the upgrade and the install are:

  1. The local copy of the package archive information is updated (fetch data -> reconstruct meta data).

  2. APT system makes decision on candidate version which is usually the latest available version. (See Section 3.4.3, “Tweaking candidate version”} for exception.)

  3. The system administrator makes choice such as upgrade of the entire system or install several new packages with candidate version.

  4. Selected binary packages are processed as (fetch -> unpack -> preinst -> install -> postinst).

The package removal process has 2 distinct stages and the simplified event flow of them are:

  • remove : remove all installed files except configuration files as (prerm -> remove -> postrm).

  • purge : purge all installed files completely including configuration files as (prerm -> purge -> postrm).

Here, I intentionally skipped technical details for the sake of big picture.

3.1.7. Basics of using aptitude

Aptitude is the current preferred package management tool for the Debian system. It can be used as the commandline alternative to apt-get/apt-cache and also as the full screen menu-driven tool alternative to dselect.

The selection of a package in aptitude not only pulls in packages which are defined in its "Depends:" list but also defined in the "Recommends:" list if the menu "F10 -> Options -> Dependency handling" is set accordingly. These auto installed packages are removed automatically if they are no longer needed under aptitude.

[Note] Note

Before the "lenny" release, apt-get and other standard APT tools did not offer the autoremove functionality.

For the package management, you start aptitude interactive mode from the console shell prompt as:

$ sudo aptitude -u
Password:

This will update the local copy of the archive information and display the package list in the full screen with menu. Aptitude places its configuration at $HOME/.aptitude/config.

[Tip] Tip

If you want to use root's configuration instead of user's one, use "sudo -H aptitude ..." instead of "sudo aptitude ..." in the above expression.

3.1.7.1. Key bindings of aptitude

Notable key strokes to browse status of packages and to set "planned action" on them in this full screen mode are:

Table 3.6.  List of key bindings for aptitude.

key

key binding

F10 or in a terminal window Ctrl-t

Menu

?

Display help for keystroke (more complete listing)

F10 -> Help -> User's Manual

Display User's Manual

u

Update package archive information

+

Mark the package for the upgrade or the install

-

Mark the package for the remove (keep conffiles)

_

Mark the package for the purge (remove conffiles)

=

Place the package on hold

U

Mark all upgradable packages (function as dist-upgrade)

g

Start downloading and installing selected packages

q

Quit current screen and save changes

x

Quit current screen and discard changes

Enter

View information about a package

C

View a package's changelog

l

Change the limit for the displayed packages

/

Search for the first match

\

Repeat the last search


The file name specification of the command line and the menu prompt after pressing "l" and "/" take the aptitude regex as described below. For the input to the menu prompt and argument to "aptitude search" command, "~n" is prepended to match the package name with the pattern if the input does not start with "~" character.

[Tip] Tip

You need to press "U" to get all the installed packages upgraded to the candidate version in the visual interface. Otherwise only the selected packages and certain packages with versioned dependency to them are upgraded to the candidate version.

3.1.7.2. Package views under aptitude

In the interactive full screen mode of aptitude(8), packages in the package list are displayed like this by default:

idA   libsmbclient                             -2220kB 3.0.25a-1  3.0.25a-2

Here, this line means from the left as:

  • The "current state" flag (the first letter)

  • The "planned action" flag (the second letter)

  • The "automatic" flag (the third letter)

  • The package name

  • The change in disk space usage attributed to "planned action".

  • The current version of the package.

  • The candidate version of the package.

[Tip] Tip

The full list of flags are given at the bottom of Help screen shown by pressing "?".

The candidate version is chosen according to the current local policy and preferences (see apt_preferences(5)).

Several types of package views are available under the menu "Views":

Table 3.7.  Views for aptitude.

view

categorization

status

Package View

See Table 3.8, “The categorization of standard aptitude views. ”. (default)

Good

Audit Recommendations

Packages which are recommended by some installed packages but not yet installed are listed.

Good.

Flat Package List

Packages are listed without categorization (for use with regex).

Good

Debtags Browser

Packages are categorized according to their debtags entries.

Very usable

Categorical Browser

Packages are categorized according to their category.

Deprecated (Use debtags!)


The standard "Package View" categorizes packages somewhat like dselect with few extra features. For switching distribution to a newer one can be achieved basically by

Table 3.8. The categorization of standard aptitude views.

category

organization

"Upgradable Packages"

Organized as section --> component --> package

"New Packages"

, ,

"Installed Packages"

, ,

"Not Installed Packages"

, ,

"Obsolete and Locally Created Packages"

, ,

"Virtual Packages"

You can pick a particular package from a set of packages with the same function.

"Tasks"

You can cherry pick particular packages from a set of packages of a task.


3.1.7.3. The command line use of aptitude

If you are migrating from the apt-get/apt-cache usage, here is the rough guide for the aptitude equivalents:

Table 3.9.  The comparative commands for aptitude and apt-get/apt-cache.

New aptitude syntax

Old apt-get/apt-cache syntax

aptitude update <package>

apt-get update <package>

aptitude install <package>

apt-get install <package>

aptitude upgrade <package>

apt-get upgrade <package>

aptitude safe-upgrade <package>

N/A

aptitude full-upgrade <package>

apt-get dist-upgrade <package>

aptitude remove <package>

apt-get remove <package>

aptitude purge <package>

apt-get purge <package>

aptitude clean

apt-get clean

aptitude autoclean

apt-get autoclean

aptitude show <package>

apt-cache show <package>

aptitude search <regex>

apt-cache search <regex>

aptitude why <regex>

N/A (See below)

aptitude why-not <regex>

N/A


You may lose the advantage of aptitude keeping track of which packages you have deliberately installed if other package tools are used. Although it is now safe to mix different package tools on the Debian system, it is best to continue using aptitude as much as possible.

The difference between upgrade and dist-upgrade only appears when new versions of packages stand in different dependency relationships from old versions of those packages. The "aptitude upgrade" command will never install new packages nor remove installed packages.

The "aptitude why <regex>" can list more information by "aptitude -v why <regex>". Similar information can be obtained by "apt-cache rdepends <package>".

When aptitude command is started in the commandline mode and faces some issues such as package conflicts, you can switch to the full screen mode by pressing "e"-key later at the prompt.

You may provide command options right after "aptitude".

Table 3.10.  Notable command options for "aptitude".

command option

effects

-s

simulate the result of the command.

-d

download only but no install/upgrade.

-D

show brief explanations before the automatic installations and removals.


See aptitude(8) and the "User's Manual" /usr/share/doc/aptitude/README for more.

3.1.7.4. The aptitude regex formula

The aptitude regex formula is mutt-like extended ERE (see: Section 2.6.2, “Regular expressions”) and the meanings of the aptitude specific special match rule extensions are as below:

Table 3.11.  List of the aptitude regex formula.

meaning of the extended match rule

regex formula

match on package name

~n<regex_name>

match on description

~d<regex_description>

match on task name

~t<regex_task>

match on debtag

~G<regex_debtag>

match on maintainer

~m<regex_maintainer>

match on package section

~s<regex_section>

match on package version

~V<regex_version>

match archive

~A{sarge,etch,sid}

match origin

~O{debian,...}

match priority

~p{extra,important,optional,required,standard}

match essential packages

~E

match virtual packages

~v

match new packages

~N

match with pending action

~a{install,upgrade,downgrade,remove,purge,hold,keep}

match installed packages

~i

match installed packages with A-mark (auto installed package)

~M

match installed packages without A-mark (administrator selected package)

~i!~M

match installed and upgradable packages

~U

match removed but not purged packages

~c

match removed, purged or can-be-removed packages

~g

match with broken relation

~B<type>

match broken depends/predepends/conflict packages

~b

match packages whose control files define relation <type> to the <term> package

~D[<type>:]<term>

match packages whose control files define broken relation <type> to the <term> package

~DB[<type>:]<term>

match packages to which the <term> package defines relation <type>

~R[[<type>:]<term>

match packages to which the <term> package defines broken relation <type>

~RB[<type>:]<term>

match packages to which some other installed packages depend on

~R~i

match packages to which no other installed packages depend on

!~R~i

match packages to which some other installed packages depend or recommend on

~R~i|~Rrecommends:~i

match <term> package with filtered version

~S filter <term>

match all packages (true)

~T

match no packages (false)

~F


Here,

  • regex part is the same ERE as the one used in typical Unix-like text tools using "^", ".*", "$" etc. as in egrep, awk and perl.

  • relation <type> is one of (depends, predepends, recommends, suggests, conflicts, replaces, provides).

  • the default relation type is "depends".

[Tip] Tip

When <regex_pattern> is a null string, place "~T" immediately after the command.

Short cuts:

  • "~P<term>" == "~Dprovides:<term>"

  • "~C<term>" == "~Dconflicts:<term>"

  • "...~W term" == "(...|term)"

Users familiar with mutt will pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in the "User's Manual" /usr/share/doc/aptitude/README.

3.1.7.5. Search method options with aptitude

Aptitude offers several options for you to search packages using its regex formula:

  • "aptitude search '<aptitude_regex>'" to list their installation status, package name and short description.

  • "aptitude show '<package_name>'" to list their installation detailed description.

  • limit view to matching packages: Type "l" in the full screen mode.

  • search the first found package: type "/" in the full screen mode. "n" for find-next, "\" for backward search.

Here, the string for <package_name> is treated as the exact string match to the package name unless it is started explicitly with "~" to be the regex formula.

3.1.8. Examples for using aptitude

Here are few examples of using aptitude.

3.1.8.1. List packages with regex matching package name

The following command lists packages with regex matching names.

$ aptitude search '~n(pam|nss).*ldap'
p libnss-ldap - NSS module for using LDAP as a naming service
p libpam-ldap - Pluggable Authentication Module allowing LDAP interfaces

This is quite handy for you to find the exact name of a package.

3.1.8.2. Browse with the regex matching

The regex "~dipv6" in the "New Flat Package List" view with "l" prompt, highlights packages with the matching description and let you browse their information interactively.

3.1.8.3. Tidy auto/manual install status

Here is how I tidy auto/manual install status for packages (after using non-aptitude package installer etc.):

  • Start aptitude in interactive mode as root.

  • Type "u", "U", "f" and "g" to update and upgrade package list and packages.

  • Type "l" to enter the package tree limit as "~i(~R~i|~Rrecommends:~i)" and type "M" over "Installed Packages" as auto installed.

  • Type "l" to enter the package tree limit as "~prequired|~pimportant|~pstandard|~E" and type "m" over "Installed Packages" as manual installed.

  • Type "l" to enter the package tree limit as "~i!~M" and remove unused package by typing "-" over each of them after exposing them by typing "[" over "Installed Packages".

  • Type "l" to enter the package tree limit as "~i" and type "m" over "Tasks" as manual installed.

  • Exit aptitude.

  • Start "apt-get -s autoremove|less" as root to check what are not used.

  • Restart aptitude in interactive mode and mark needed packages as "m".

  • Restart "apt-get -s autoremove|less" as root to recheck REMOVED contain only expected packages.

  • Start "apt-get autoremove|less" as root to autoremove unused packages.

The "m" action over "Tasks" is an optional one to prevent mass package removal situation in future.

3.1.8.4. Purge removed packages for good

You can purge all remaining configuration files of removed packages:

# aptitude search '~c'
  • check results

# aptitude purge '~c'

You may want to do the similar in the interactive mode so you can have a bit more fine grained control over the action to exclude "Obsolete and Locally Created Packages" to be purged.

3.1.9. System wide upgrade with aptitude

You can perform system wide upgrade to a newer release by changing contents of the /etc/apt/sources.list file pointing to a new release and running the "aptitude update; aptitude dist-upgrade" command.

To upgrade from "stable" to "testing" or "unstable", you replace "etch" in the /etc/apt/sources.list example of Section 3.1.4, “Debian archive basics” with "lenny" or "sid".

In reality, you may face some complications due to some package transition issues, mostly due to package dependencies. The larger the difference of the upgrade, the more likely you face larger troubles. For the transition from the old "stable" archive to the new "stable" after its release, you can read its new Release Notes and follow the exact procedure described in it to minimize troubles.

When you decide to move from "stable" to "testing" before its formal release, there are no Release Notes to help you. The difference between "stable" and "testing" could have grown quite large after the previous "stable" release and makes upgrade situation complicated.

You should make some precautionary moves while gathering latest information from mailing list and using common senses:

  • read previous "Release Notes".

  • back up entire system (especially data and configuration information).

  • have bootable media handy for broken bootloader.

  • inform users on the system well in advance.

  • record upgrade activity with the script(1) command.

  • apply "unmarkauto" to essential packages, e.g., "aptitude unmarkauto vim", to prevent removal.

  • minimize installed packages to reduce chance of package conflicts, e.g., remove desktop task packages.

  • remove the /etc/apt/preferences file. (disable apt-pinning)

  • try to upgrade step wise: "oldstable" --> "stable" --> "testing" --> "unstable".

  • update the /etc/apt/sources.list file to point to new archive only and run "aptitude update".

  • install, optionally, new core packages first, e.g., "aptitude install perl".

  • run the "aptitude dist-upgrade -s" command to assess impact.

  • run the "aptitude dist-upgrade" command.

[Caution] Caution

It is not wise to skip major Debian release when upgrading between "stable" releases.

[Caution] Caution

In previous "Release Notes", GCC, Linux Kernel, initrd-tools, Glibc, Perl, APT tool chain, etc. have required some special attention for system wide upgrade.

For daily upgrade in "unstable", see Section 3.1.16, “Safeguard for package problems”.

3.1.10. Aptitude advantages

Aptitude has advantages over other APT based packaging systems (apt-get, apt-cache, synaptic, ...):

  • aptitude removes unused auto installed packages automatically using its own extra layer of package state file (/var/lib/aptitude/pkgstates). (For new lenny, other APT does the same.)

  • aptitude makes it easy to resolve package conflicts and to add recommended packages.

  • aptitude makes it easy to keep track of obsolete software by listing under "Obsolete and Locally Created Packages".

  • aptitude gives a log of its history in /var/log/aptitude.

  • aptitude offers access to all versions of the package if available.

  • aptitude includes a fairly powerful regex based system for searching particular packages and limiting the package display.

  • aptitude in the full screen mode has su functionality embedded and can be run from normal user until you really need administrative privileges.

For the old etch release version, synaptic also gives you the history log; apt-get did not but you can rely on the log of dpkg.

Anyway, aptitude is nice for interactive console use.

3.1.11. Package activity logs

You can check package activity history in the log files.

Table 3.12.  The log files for package activities.

file

content

/var/log/dpkg.log

Log of dpkg level activity for all package activities.

/var/log/aptitude

Log of aptitude activity for itself.


3.1.12. Package maintenance tools (low-level)

Here are list of low-level package maintenance tasks for which aptitude is too high-level, plus the suggested tool.

Table 3.13.  List of safe low level system maintenance commands.

action

command

list status of an installed package for the bug report.

COLUMNS=120 dpkg -l <package_name_pattern>

list the contents of an installed package.

dpkg -L <package_name>

list the manpages for an installed package.

dpkg -L <package_name> | egrep '/usr/share/man/man.*/.+'

list installed packages which have matching file name.

dpkg -S <file_name_pattern>

list packages in archive which have matching file name.

apt-file search <file_name_pattern>

list the contents of matching packages in archive.

apt-file list <package_name_pattern>

reconfigure the exact package .

dpkg-reconfigure <package_name>

reconfigure the exact package with the most detailed question.

dpkg-reconfigure -p=low <package_name>

reconfigure packages from the full screen menu.

configure-debian

audit system for partially installed packages.

dpkg --audit

configures all partially installed packages.

dpkg --configure -a

show available version, priority, and archive information of a binary package.

apt-cache policy <binary_package_name>

show available version, archive information of a package.

apt-cache madison <package_name>

show source package information of a binary package.

apt-cache showsrc <binary_package_name>


Please note:

  • Unlike aptitude which uses regex (see: Section 2.6.2, “Regular expressions”), these commands in the above use pattern like shell glob (see: Section 2.5.3, “Shell glob”).

  • apt-file commands require apt-file package and run "apt-file update" in advance.

  • configure-debian command requires configure-debian package and runs dpkg-reconfigure as its backend.

  • dpkg-reconfigure command runs package scripts using debconf as its backend.

3.1.13. Search on the package meta data

The grep-dctrl(1), grep-status(1), and grep-available(1) commands can be used to search any file which has the general format of a Debian package control file.

The "dpkg -S <file_name_pattern>" can be used search package names which contain files with the matching name installed by dpkg. But this overlooks files created by the maintainer scripts.

If you need to make more elaborate search on the dpkg meta data, you need to run "grep -e regex_pattern *" command in the /var/lib/dpkg/info/ directory. This will let you identify:

  • the package name which installs, creates or modifies particular file which match pattern.

  • the package name which asks the installation query words which match pattern.

If you wish to look up package dependency recursively, you should use apt-rdepends(8).

3.1.14. The advanced system maintenance

Here are list of tasks for the advanced system maintenance such as creating and installing the backported and customized packages on the target system itself. For this kind of tasks, aptitude is not much of help and other lower level package and porting tools should be carefully used by the expert.

Table 3.14.  List of advanced low level system maintenance commands.

install required packages to build package.

apt-get build-dep <package_name>

download a source. (from standard archive)

apt-get source <package_name>

download a source and rebuild package(s) locally.

apt-get -b source <package_name>

download a source packages. (from other archive)

dget <URL for dsc file>

build a source tree from a set of source packages (*.tar.gz *.diff.gz).

dpkg-source -x <package_name>_<version>-<debian_version>.dsc

build package(s) from a local source tree.

debuild binary

build a kernel package from a kernel source tree.

make-kpkg kernel_image

build a kernel package from a kernel source tree with initramfs enabled.

make-kpkg --initrd kernel_image

install a local package to the system.

dpkg -i <package_name>_<version>-<debian_version>_<arch>.deb

install local package(s) to the system.

debi <package_name>_<version>-<debian_version>_<arch>.dsc

save dpkg level package selection state information

dpkg --get-selection '*' >selection.txt

set dpkg level package selection state information

dpkg --set-selection <selection.txt


Please note:

  • "apt-get build-dep", "apt-get source" and "apt-cache showsrc" commands require deb-src entry in /etc/apt/sources.list.

  • dget, debuild, and debi commands require devscripts package.

  • make-kpkg command requires kernel-package package.

  • All system configuration and installation commands require to be run from root.

[Tip] Tip

See Section 13.9, “Making Debian package” for general packaging.

3.1.15. Verify installed package files

The installation of debsums package enables verification of installed package files against MD5sum values in the Packages file with debsums(1) command. See: Section 11.3.5, “The MD5 sum” for how MD5sum works.

3.1.16. Safeguard for package problems

Many user prefer to follow the unstable release of the Debian system for its new features and packages. This makes the system more prone to be hit by the critical package bugs.

The installation of the apt-listbugs package will provide safeguard to the critical bugs by checking Debian BTS automatically for critical bugs when upgrading with APT system.

The installation of the apt-listchanges package will provide important news in NEWS.Debian when upgrading with APT system.

3.1.17. What to do when you face some package problems

You should read the fine official documentation. The first document to read is the Debian specific /usr/share/doc/<package_name>/README.Debian. Other documentation in /usr/share/doc/<package_name>/ should be consulted too. If you set shell as previously discussed, type:

$ cd <package_name>
$ pager README.Debian
$ mc

You may need to install the corresponding documentation package named with "-doc" suffix for detailed information.

If you are experiencing problems with a specific package, make sure to check out these sites first:

Table 3.15.  List of key web site to resolving problems with a specific package.

site

command

Home page of the Debian bug tracking system (BTS)

$ sensible-browser http://bugs.debian.org/

The bug report of a known package name.

$ sensible-browser http://bugs.debian.org/<bug_number>

The bug report of known bug number.

$ sensible-browser http://bugs.debian.org/<package_name>


Search Google with search words including "site:debian.org".

When you file a bug report, please use reportbug command.

Downgrading from a later release of a package to an earlier one is not officially supported in Debian (see: Section 3.4.3, “Tweaking candidate version”). However, you may find that you have to downgrade a specific package in order to re-install a version of a package that works when a new version malfunctions. You may find these previous package files locally in /var/cache/apt/archives/ or remotely at http://snapshot.debian.net/. You need to make sure the consistency are kept.

3.2. Debian package management internals

Let's learn how the Debian package management system works internally. This should help you to create your own solution to some package problems.

3.2.1. Archive meta data

The meta data files are stored under the dist/ on each Debian mirror sites, e.g., ftp://ftp.us.debian.org/debian/. Its archive structure can be browsed by the web browser. There are 6 types of key meta data:

Table 3.16.  The content of the Debian archive meta data.

file

location

content

Release

top of distribution

archive description and integrity information

Release.gpg

top of distribution

signature file for Release signed with the archive key

Contents-<architecture>

top of distribution

list of all files for all the packages in the pertinent archive

Release

top of each distribution/component/architecture combination

archive description

Packages

top of each distribution/component/binary-architecture combination

concatenated debian/control for binary packages

Sources

top of each distribution/component/source combination

concatenated debian/control for source packages


In the recent archive, these meta data are stored as the compressed and differential files to reduce network traffic.

3.2.2. Top level Release file and authenticity

Each suites of the Debian archive has a top level Release file, e.g., ftp://ftp.us.debian.org/debian/dists/unstable/Release:

Origin: Debian
Label: Debian
Suite: unstable
Codename: sid
Date: Sat, 26 Jan 2008 20:13:58 UTC
Architectures: alpha amd64 arm hppa hurd-i386 i386 ia64 m68k mips mipsel powerpc s390 sparc
Components: main contrib non-free
Description: Debian x.y Unstable - Not Released
MD5Sum:
 e9f11bc50b12af7927d6583de0a3bd06 22788722 main/binary-alpha/Packages
 43524d07f7fa21b10f472c426db66168  6561398 main/binary-alpha/Packages.gz
...
[Note] Note

Here, you can find my rationale to use the "suite", "codeneme", and "components" in Section 3.1.4, “Debian archive basics”. The "distribution" is used when referring to both "suite" and "codeneme".

The integrity of the top level Release file is verified by cryptographic infrastructure called the secure apt.

  • The cryptographic signature file Release.gpg is created from the authentic top level Release file and the secret Debian archive key.

  • The public Debian archive signing key can be seeded into /etc/apt/trusted.gpg:

  • The secure APT system verifies the integrity of the downloaded top level Release file cryptographically by this Release.gpg file and the public Debian archive key in /etc/apt/trusted.gpg.

The integrity of all the Packages and Sources files are verified by using MD5sum values in its top level Release file. The integrity of all package files are verified by using MD5sum values in the Packages and Sources files. See debsums(1) and Section 3.1.15, “Verify installed package files”.

Since the cryptographic signature verification is very CPU intensive process than the MD5sum value calculation, use of MD5sum value for each package while using cryptographic signature for the top level Release file provides the good security with the performance (see: Section 11.3, “Data security infrastructure”).

3.2.3. Archive level Release files

[Tip] Tip

The archive level Release files are used for the rule of apt_preferences(5).

There are archive level Release files for all archive locations specified by "deb:" line in /etc/apt/sources.list, such as "ftp://ftp.us.debian.org/debian/dists/unstable/main/binary-amd64/Release" or "ftp://ftp.us.debian.org/debian/dists/sid/main/binary-amd64/Release":

Archive: unstable
Component: main
Origin: Debian
Label: Debian
Architecture: amd64
[Caution] Caution

For Archive stanza, suite names ("stable", "testing", "unstable", ...) are used in the Debian archive while codenames ("dapper", "feisty", "gutsy", "hardy", "intrepid", ...) are used in the Ubuntu archive.

For some archives, such as "experimental", "volatile-sloppy", and "etch-backports", which contain packages which should not be installed automatically, there is an extra line, e.g., "ftp://ftp.us.debian.org/debian/dists/experimental/main/binary-amd64/Release":

Archive: experimental
Component: main
Origin: Debian
Label: Debian
NotAutomatic: yes
Architecture: amd64

Please note that for normal archives without "NotAutomatic: yes", the default Pin-Priority value is 500, while for special archives with "NotAutomatic: yes", the default Pin-Priority value is 1 (see apt_preferences(5) and Section 3.4.3, “Tweaking candidate version”).

3.2.4. Fetching of the meta data for the package

When APT tools, such as aptitude, apt-get, synaptic, apt-file, auto-apt..., are used, we need to update the local copies of the meta data containing the Debian archive information. These local copies have file names corresponding to the specified distribution component and architecture names in the /etc/apt/sources.list (see: Section 3.1.4, “Debian archive basics”) under the /var/lib/apt/lists directory as:

  • ftp.us.debian.org_debian_dists_<distribution>_Release

  • ftp.us.debian.org_debian_dists_<distribution>_Release.gpg

  • ftp.us.debian.org_debian_dists_<distribution>_<section>_binary-<architecture>_Packages

  • ftp.us.debian.org_debian_dists_<distribution>_<section>_source_Sources

  • /var/cache/apt/apt-file/ftp.us.debian.org_debian_dists_<distribution>_Contents-<architecture>.gz (for apt-file)

First 4 are shared by all the pertinent APT commands and updated from command line by "apt-get update" and "aptitude update". The Packages meta data are updated if there is the deb line in /etc/apt/sources.list. The Sources meta data are updated if there is the deb-src line in /etc/apt/sources.list.

The Packages and Sources meta data contain "Filename:" stanza pointing to the file location of the binary and source packages. Currently, they are located under the pool/ directory tree for the improved transition over the releases.

The local copies of Packages meta data can be interactively searched with the help of aptitude. The specialized search command grep-dctrl(1) can search the local copies of Packages and Sources meta data.

The local copyies of Contents-<architecture> files can be updated by "apt-file update" and location is different from other 4 files. See apt-file(1). (The auto-apt uses different location for local caching of Contents-<architecture>.gz as default.)

3.2.4.1. The package state for APT

In addition to the remotely fetched meta data, the APT tool after "lenny" stores its locally generated installation state information in the /var/lib/apt/extended_states which is used only by all APT tools to track all auto installed packages.

3.2.4.2. The package state for aptitude

In addition to the remotely fetched meta data, the aptitude command stores its locally generated installation state information in the /var/lib/aptitude/pkgstates which is used only by it.

3.2.5. The local copies of the fetched packages

All the remotely fetched packages via APT mechanism are stored in the /var/cache/apt/packages until they are cleaned.

3.2.6. The Debian package file name

The Debian package files has particular name structures:

Table 3.17.  The name structures of the Debian packages.

entity

name structure

The binary package (a.k.a deb)

<package-name>_<epoch>:<upstream-version>-<debian.version>-<architecture>.deb

The binary package for the debian-installer (a.k.a udeb)

<package-name>_<epoch>:<upstream-version>-<debian.version>-<architecture>.udeb

The source package (upstream source)

<package-name>_<epoch>:<upstream-version>-<debian.version>.tar.gz

The source package (Debian changes)

<package-name>_<epoch>:<upstream-version>-<debian.version>.diff.gz

The source package (description)

<package-name>_<epoch>:<upstream-version>-<debian.version>.dsc


where,

Table 3.18.  The usable characters for each component in the Debian package names.

component

usable characters (regex)

required

<package-name>

[a-z,A-Z,0-9,.,+,-]+

required

<epoch>:

[0-9]+:

optional

<upstream-version>

[a-z,A-Z,0-9,.,+,-,:]+

required

<debian.version>

[a-z,A-Z,0-9,.,+,~]+

optional


[Note] Note

You can check package version order by dpkg(1), e.g., "dpkg --compare-versions 7.0 gt 7.~pre1 ; echo $?" .

[Note] Note

The debian-installer (d-i) uses udeb as the file extension for its binary package instead of normal deb. An udeb package is a stripped down deb package which removes few non-essential contents such as documentation to save space while relaxing the package policy requirements. Both deb and udeb package share the same package structure. The "u" stands for micro.

3.2.7. The dpkg command

The dpkg is the lowest level tool for the Debian package management. This is very powerful and this need to be used with care.

The fetched package is processed by dpkg in the following order:

  1. unpack the deb file ("ar -x" equivalent)

  2. preinst using debconf

  3. install the package content to the system ("tar -x" equivalent)

  4. postinst using debconf

The debconf system provides standardized user interaction with i18n and l17n supports.

Here dpkg creates following files under /var/lib/dpkg/info/ directory : While installing package called <package_name>, dpkg creates several files and execute scripts.

Table 3.19.  The notable files for dpkg.

file

contents

/var/lib/dpkg/info/<package_name>.conffiles

list of user modifiable files.

/var/lib/dpkg/info/<package_name>.list

list of files and directories installed by the package.

/var/lib/dpkg/info/<package_name>.md5sums

list of MD5 hash values for files installed by the package.

/var/lib/dpkg/info/<package_name>.preinst

package script run before the package installation.

/var/lib/dpkg/info/<package_name>.postinst

package script run after the package installation.

/var/lib/dpkg/info/<package_name>.prerm

package script run before the package removal.

/var/lib/dpkg/info/<package_name>.postrm

package script run after the package removal.

/var/lib/dpkg/info/<package_name>.config

package script for debconf system.

/var/lib/dpkg/alternatives/<package_name>

the alternative information used by the update-alternatives command.

/var/lib/dpkg/available

the availability information for all the package.

/var/lib/dpkg/diversions

the diversions information used by the dpkg-divert command.

/var/lib/dpkg/status

the status information for all the packages.

/var/lib/dpkg/status-old

the first-generation backup of the var/lib/dpkg/status file.

/var/backups/dpkg.status*

the second-generation backup and older ones of the var/lib/dpkg/status file.


The last file status is also used by the tools such as "dpkg", "select update" and "apt-get -u dselect-upgrade".

The specialized search command grep-dctrl(1) can search the local copies of status and available meta data.

[Tip] Tip

In the debian-installer environment, the udpkg command is used to open udeb packages. The udpkg is a stripped down version of dpkg command.

3.2.8. The update-alternative command

The Debian system has mechanism to install somewhat overlapping programs peacefully using update-alternatives(8). For example, to make the command vi select to run vim while installing both vim and nvi:

$ ls -l $(type -p vi)
lrwxrwxrwx 1 root root 20 2007-03-24 19:05 /usr/bin/vi -> /etc/alternatives/vi
$ sudo update-alternatives --display vi
...
$ sudo update-alternatives --config vi
  Selection    Command
-----------------------------------------------
      1        /usr/bin/vim
*+    2        /usr/bin/nvi

Enter to keep the default[*], or type selection number: 1

The Debian alternatives system keeps its selection as symlinks in /etc/alternatives/. The selection process uses corresponding file in /var/lib/dpkg/alternatives/.

3.2.9. The dpkg-statoverride command

Stat overrides provided by the dpkg-statoverride(8) command are a way to tell dpkg(1) to use a different owner or mode for a file when a package is installed. If "--update" is specified and file exists, it is immediately set to the new owner and mode.

[Note] Note

I use the word file here, but in reality this can be any filesystem object that dpkg handles, including directories, devices, etc.

[Caution] Caution

The direct alteration of owner or mode for a file owned by the package using chmod or chown commands by the system administrator will be reset by the next upgrade of the package.

3.2.10. The dpkg-divert command

File diversions provided by the dpkg-divert(8) command are a way of forcing dpkg(1) not to install a file into its default location, but to a diverted location. The use of dpkg-divert is meant for the package maintenance scripts. Its use by the system administrator is deprecated.

3.3. Recovery from a broken system

When running "unstable" system, the administrator is expected to recover from broken package management situation.

[Caution] Caution

Some methods described here are high risk actions. You have been warned!

3.3.1. Incompatibility with old user configuration

If a desktop GUI program experienced instability after significant upstream version upgrade, you should suspect interferences with old local configuration files created by it. If it is stable under newly created user account, this hypothesis is confirmed. (This is a bug of packaging and usually avoided by the packager.)

To recover stability, you should move corresponding local configuration files and restart the GUI program. You may need to read old configuration file contents to recover configuration information later. (Do not erase them too quickly.)

3.3.2. Different packages with overlapped files

The archive level package management systems, such as aptitude(8) or apt-get(1), will not even try to install packages with overlapped files using package dependencies (see Section 3.1.5, “Package dependencies”).

Errors by the package maintainer or deployment of inconsistently mixed source of archives (see Section 3.4.2, “Packages from mixed source of archives”) by the system administrator may create situation with incorrectly defined package dependencies. When you install a package with overlapped files using aptitude(8) or apt-get(1) under such situation, dpkg(1) which unpacks package ensures to return error to the calling program without overwriting existing files.

[Caution] Caution

The use of third party packages introduces significant system risks via maintainer scripts which are run with root privilege and can do anything to your system. The dpkg(1) command only protects against overwriting by the unpacking.

You can work around such broken installation by removing the old offending package, <old-package>, first:

$ sudo dpkg -P <old-package>

3.3.3. Fixing broken package script

When a command in the package script returns error for some reason and the script exits with error, the package management system aborts their action and ends up with partially installed packages. When a package contains bugs in its removal scripts, the package may become impossible to remove and quite nasty.

For the package script problem of "<package_name>", you should look for:

  • /var/lib/dpkg/info/<package_name>.preinst

  • /var/lib/dpkg/info/<package_name>.postinst

  • /var/lib/dpkg/info/<package_name>.prerm

  • /var/lib/dpkg/info/<package_name>.postrm

You edit the offending part of the script from the root:

  • to prepend with ": #" or,

  • to append with "|true".

Then configures all partially installed packages by:

# dpkg --configure -a

3.3.4. Rescue using the dpkg command

Since dpkg is very low level package tool, it can function under the very bad situation such as unbootable system without network connection. Let's assume foo package was broken and needs to be replaced.

You may still find cached copies of older version of foo package in /var/cache/apt/archives/ which is bug free or you may be able to download a bug free foo package to other functioning machine. If you can boot the system, you may install it by:

# dpkg -i /path/to/foo_<old_version>_<arch>.deb
[Tip] Tip

If system breakage is minor, you may alternatively downgrade the whole system as Section 3.4.5, “Emergency downgrading” using the higher level APT system.

If your system is unbootable and if you are booting from the rescue CD while mounting the system to /target, you may then alternatively install it by:

# dpkg --root /target -i /path/to/foo_<old_version>_<arch>.deb

This second example should work even if the dpkg command on the system is broken.

If attempting to install a package this way fails due to some dependency violations and you really need to do this as the last resort, you can override dependency using dpkg's --ignore-depends, --force-depends and other options. If you do this, you need to make serious effort to restore proper dependency later. See dpkg(8) for details.

3.3.5. Recover package selection data

If /var/lib/dpkg/status becomes corrupt for any reason, the Debian system loses package selection data and suffers severely. Look for the old /var/lib/dpkg/status file at /var/lib/dpkg/status-old or /var/backups/dpkg.status.*.

Keeping /var/backups/ in a separate partition may be a good idea since this directory contains lots of important system data.

If everything is gone, I recommend to make fresh re-install after making backup of the system in such serious breakage though. You can still recover some information from directories in /usr/share/doc/ to guide your new installation.

... reinstall minimal (desktop) system
... place old system at /path/to/old/system/
# cd /path/to/old/system/usr/share/doc
# ls -1 >~/ls1.txt
# cd /usr/share/doc
# ls -1 >>~/ls1.txt
# cd
# sort ls1.txt | uniq | less

Then you will be presented with missing package names. (There may be some non-package names such as "texmf".)

3.4. Tips for the package management

3.4.1. How to pick Debian packages

You can seek packages which satisfy your needs with aptitude from the package description or from the list under "Tasks".

When you encounter more than 2 similar packages and wonder which one to install without "trial and error" efforts, you can use some common sense. I consider following points are good indications of preferred packages.

  • essential: yes > no

  • component: main > contrib > non-free

  • priorities: required > important > standard > optional > extra

  • task: package listed in task such as desktop

  • package selected by the dependency package (e.g., python2.4 by python)

  • popcon: higher in the vote and install number

  • changelog: regular updates by the maintainer

  • BTS: No RC bugs (no critical, no grave, and no serious bugs)

  • BTS: responsive maintainer to bug reports

  • BTS: higher number of the recently fixed bugs

  • BTS: lower number of remaining non-wishlist bugs

Debian being a volunteer project with distributed development model, its archive contains many packages with different focus and quality. You must make your own decision what to do with them.

3.4.2. Packages from mixed source of archives

[Caution] Caution

Installing packages from mixed source of archives is not supported by the official Debian distribution except for officially supported particular combinations of archives such as "stable" with security updates and volatile updates.

Here is an example of operations to include specific newer upstream version packages found in "unstable" while tracking "testing" for single occasion:

  • change the /etc/apt/sources.list file temporarily to single "unstable" entry

  • run "aptitude update"

  • run "aptitude install <package-name>"

  • recover the original /etc/apt/sources.list file for "testing"

  • run "aptitude update"

You do not create the /etc/apt/preferences file nor need to worry about apt-pinning with this manual approach. But this is very cumbersome.

[Caution] Caution

When using mixed source of archives, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. If package incompatibility exists, you may break system. You must be able to judge these technical requirements. The use of mixed source of random archives is completely optional operation and its use is not something I encourage you to use.

General rules for installing packages from different archives are:

  • non-binary packages ("Architecture: all") are safer to install.

    • documentation packages: no special requirements

    • interpreter program packages: compatible interpreter must be available

  • completely statically linked binary packages are safe to install.

  • binary packages (non "Architecture: all") usually face many road blocks and unsafe to install.

    • library version compatibility (including "libc")

    • related utility program version compatibility

    • Kernel ABI compatibility

    • C++ ABI compatibility

    • ...

[Note] Note

Except to avoid broken package for a short term, installing binary packages from officially unsupported archives is generally bad idea. This is true even if you use apt-pinning (see Section 3.4.3, “Tweaking candidate version”). You should consider chroot or similar techniques (see Section 10.6, “The chroot”) to run programs from different archives.

3.4.3. Tweaking candidate version

Without the /etc/apt/preferences file, APT system choses the latest available version as the candidate version using the version string. This is the normal state and most recommended usage of APT system. All officially supported combinations of archives do not require the /etc/apt/preferences file since some archives which should not be used as the automatic source of upgrades are marked as NotAutomatic and dealt properly.

[Tip] Tip

The version string comparison rule can be verified with, e.g., "dpkg --compare-versions ver1.1 gt ver1.1~1; echo $?" (see dpkg(1)).

When you install packages from mixed source of archives (see Section 3.4.2, “Packages from mixed source of archives”) regularly, you can automate these complicated operations by creating the /etc/apt/preferences file with proper entries and tweaking the package selection rule for candidate version as described in apt_preferences(5). This is called apt-pinning.

[Warning] Warning

Use of apt-pinning by a novice user is sure call for major troubles. You must avoid using apt-pinning except when you absolutely need it.

[Caution] Caution

When using apt-pinning, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. The apt-pinning is completely optional operation and its use is not something I encourage you to use.

[Caution] Caution

Archive level Release files (see Section 3.2.3, “Archive level Release files”) are used for the rule of apt_preferences(5). Thus apt-pinning works only with "suite" name for normal Debian archives and security Debian archives. (This is different from Ubuntu archives). For example, you can do "Pin: release a=unstable" but can not do "Pin: release a=sid" in the /etc/apt/preferences file.

[Note] Note

Even if you do not create the /etc/apt/preferences file, you can do fairly complex system operations (see Section 3.3.4, “Rescue using the dpkg command” and Section 3.4.2, “Packages from mixed source of archives”) without apt-pinning.

Here is a simplified explanation of apt-pinning technique. Each package has its Pin-Priority value based on the entries in the /etc/apt/preferences file or the default values.

Table 3.20.  List of essential default Pin-Priority values.

Pin-Priority

description

990

default value for package from the target release archive

500

default value for package from the normal archive

100

default value for package from the installed package

1

default value for package from the NotAutomatic archive


APT system normally choses highest Pin-Priority upgrading package from the available resources defined in the /etc/apt/sources.list file as the candidate version. This version restriction for upgrading is dropped to enable downgrading if the Pin-Priority of package is larger than 1000 (see Section 3.4.5, “Emergency downgrading”).

The target release can be set:

  • by /etc/apt/apt.conf, e.g., "APT::Default-Release "stable";" line in it, or

  • by "-t" option argument, e.g., "apt-get install -t testing some-package".

The archive level Release file (see Section 3.2.3, “Archive level Release files”) of NotAutomatic archive contains "NotAutomatic: yes".

[Caution] Caution

Although cryptic to read, you should test Pin-Priority situation of your /etc/apt/preferences file using the output of "apt-cache policy <package>".

The Pin-Priority values of <package> from multiple sources are shown by the output of "apt-cache policy <package>":

  • a line started with "Package pin:" lists the package version of pin if association just with <package> is defined, e.g., "Package pin: 0.190",

  • no line with "Package pin:" exists if no association just with <package> is defined,

  • the Pin-Priority value associated just with <package> is listed right side of all version strings, e.g., "0.181 700",

  • 0 is listed right side of all version strings if no association just with <package> is defined, e.g., "0.181 0", and

  • the Pin-Priority values of archives (defined as "Package: *" in the /etc/apt/preferences file) are listed left side of all archive paths, e.g., "200 http://backports.org etch-backports/main Packages".

Here is an example of apt-pinning technique to include specific newer upstream version packages found in "unstable" regularly upgraded while tracking "testing". You list all required archives in the /etc/apt/sources.list file as:

deb http://ftp.us.debian.org/debian/ testing main contrib non-free
deb http://ftp.us.debian.org/debian/ unstable main contrib non-free
deb http://security.debian.org/ testing/updates main contrib

and set the /etc/apt/preferences file as:

Package: *
Pin: release a=testing
Pin-Priority: 500

Package: *
Pin: release a=unstable
Pin-Priority: 200

When you wish to install a package named "<package-name>" with its dependencies from "unstable" archive under this configuration, you issue the following command which switches target release with "-t" option (Pin-Priority of "unstable" becomes 990.):

$ sudo aptitude install -t unstable <package-name>

With this configuration, usual execution of "aptitude upgrade" and "aptitude dist-upgrade" will upgrade packages which were installed from "testing" archive using current "testing" archive and packages which were installed from "unstable" archive using current "unstable" archive.

[Caution] Caution

Be careful not to remove "testing" entry from the /etc/apt/sources.list file. Without "testing" entry in it, APT system will upgrade packages using newer "unstable" archive.

[Tip] Tip

I usually edit the /etc/apt/sources.list file to comment out "unstable" archive entry right after above operation. This avoids slow update process of having too many entries in the /etc/apt/sources.list file although this prevents upgrading packages which were installed from "unstable" archive using current "unstable" archive.

[Tip] Tip

If "Pin-Priority: 20" is used instead of "Pin-Priority: 200" for the /etc/apt/preferences file, already installed packages having Pin-Priority value of 100 will never be upgraded by "unstable" archive even if "testing" entry in the /etc/apt/sources.list file is removed.

If you wish to track particular packages in "unstable" automatically without initial "-t unstable" installation, you must create the /etc/apt/preferences file and explicitly lists all those packages at the top of it as:

Package: <package-1>
Pin: release a=unstable
Pin-Priority: 700

Package: <package-2>
Pin: release a=unstable
Pin-Priority: 700

...

These will set Pin-Priority value for each specific package. For example, in order to track the latest "unstable" version of this "Debian Reference" in English, you should have following entries in the /etc/apt/preferences file:

Package: debian-reference-en
Pin: release a=unstable
Pin-Priority: 700

Package: debian-reference-common
Pin: release a=unstable
Pin-Priority: 700
[Tip] Tip

This apt-pinning technique is valid even when you are tracking "stable" archive. Documentation packages have been always safe to install from "unstable" archive in my experience. (Reminder: This multi-archive tracking is not officially supported feature.)

Here is another example of apt-pinning technique to include specific newer upstream version packages found in "experimental" while tracking "unstable". You list all required archives in the /etc/apt/sources.list file as:

deb http://ftp.us.debian.org/debian/ unstable main contrib non-free
deb http://ftp.us.debian.org/debian/ experimental main contrib non-free
deb http://security.debian.org/ testing/updates main contrib

The default Pin-Priority value for "experimental" archive is always 1 since it is NotAutomatic archive (see Section 3.2.3, “Archive level Release files”). There is no need to set Pin-Priority value in the /etc/apt/preferences file to use experimental archive unless you wish to track particular packages automatically.

3.4.4. Volatile and Backports.org

There are The debian-volatile Project and http://backports.org archives which provide updgrade packages for "stable".

[Warning] Warning

Do not use all packages available in the NotAutomatic archives such as "etch-backports" and "volatile-sloppy". Use only selected packages which fits your needs.

[Caution] Caution

http://backports.org is a non-Debian archive, although its packages are signed by Debian developers.

[Caution] Caution

Archive level Release files (see Section 3.2.3, “Archive level Release files”) are used for the rule of apt_preferences(5). Thus apt-pinning works only with "code" name for volatile Debian archives. This is different from other Debian archives. For example, you can do "Pin: release a=etch" but can not do "Pin: release a=stable" in the /etc/apt/preferences file for volatile Debian archives.

Here is an example of apt-pinning technique to include specific newer upstream version packages found in "etch-backports" while tracking "etch" and "volatile". You list all required archives in the /etc/apt/sources.list file as:

deb http://ftp.us.debian.org/debian/ etch main contrib non-free
deb http://security.debian.org/ etch/updates main contrib
deb http://volatile.debian.org/debian-volatile/ etch/volatile main contrib non-free
deb http://volatile.debian.org/debian-volatile/ etch/volatile-sloppy main contrib non-free
deb http://backports.org/debian/ etch-backports main contrib non-free

The default Pin-Priority value for http://backports.org archive and "volatile-sloppy" are always 1 since they are NotAutomatic archive (see Section 3.2.3, “Archive level Release files”). There is no need to set Pin-Priority value explicitly in the /etc/apt/preferences file just to use for http://backports.org and "volatile-sloppy" archive unless you wish to track packages automatically for next upgrading.

So whenever you wish to install a package named "<package-name>" with its dependency from "etch-backports" archive, you use following command while switching target release with "-t" option:

$ sudo aptitude install -t etch-backports <package-name>

If you wish to upgrade particular packages, you must create the /etc/apt/preferences file and explicitly lists all packages in it as:

Package: <package-1>
Pin: release o=Backports.org archive
Pin-Priority: 700

Package: <package-2>
Pin: release o=volatile.debian.org
Pin-Priority: 700

...

Alternatively, with the /etc/apt/preferences file as:

Package: *
Pin: release a=stable , o=Debian
Pin-Priority: 500

Package: *
Pin: release a=etch, o=volatile.debian.org
Pin-Priority: 500

Package: *
Pin: release a=etch-backports, o=Backports.org archive
Pin-Priority: 200

Package: *
Pin: release a=etch-sloppy, o=volatile.debian.org
Pin-Priority: 200

execution of "aptitude upgrade" and "aptitude dist-upgrade" will upgrade packages which were installed from "stable" archive using current "stable" archive and packages which were installed from other archives using current corresponding archive for all archives in the /etc/apt/sources.list file.

3.4.5. Emergency downgrading

[Caution] Caution

Downgrading is not officially supported by the Debian by design. It should be done only as a part of emergency recovery process. Despite of this situation, it is known to work well in many incidents. For critical systems, You should backup all important data on the system after the recovery operation and re-install the new system from the scratch.

You may be lucky to downgrade from newer archive to older archive to recover from broken system upgrade. This is lazy alternative to tedious actions of many "dpkg -i <broken-package>_<old-version>.deb" commands (see Section 3.3.4, “Rescue using the dpkg command”).

For downgrading system tracking "unstable" to "testing", change the /etc/apt/sources.list file from:

deb http://ftp.us.debian.org/debian/ sid main contrib non-free

to:

deb http://ftp.us.debian.org/debian/ lenny main contrib

and set the /etc/apt/preferences file as:

Package: *
Pin: release a=testing
Pin-Priority: 1010

Then run "aptitude dist-upgrade" to force downgrading of packages across the system. You should remove this special /etc/apt/preferences file after the downgrading.

3.4.6. Who uploaded the package?

Although the maintainer name listed in /var/lib/dpkg/available and /usr/share/doc/package_name/changelog provide some information on "who is behind the packaging activity", the actual uploader of the package is somewhat obscure. The who-uploads(1) in devscripts package identifies the actual uploader of Debian source packages.

3.4.7. The equivs package

If you are to compile a program from source to replace the Debian package, it is best to make it into a real local debianized package (*.deb) and use private archive.

If you chose to compile a program from source and to install them under /usr/local instead, you may need to use equivs as a last resort to satisfy the missing package dependency.

Package: equivs
Priority: extra
Section: admin
Description: Circumventing Debian package dependencies
 This is a dummy package which can be used to create Debian
 packages, which only contain dependency information.

3.4.8. Port a package to the stable system

For partial upgrades of the "stable" system, rebuilding a package within its environment using the source package is desirable. This avoids massive package upgrades due to their dependencies. First, add the following entries to the /etc/apt/sources.list of a "stable" system:

deb-src http://http.us.debian.org/debian unstable  main contrib non-free

Then get the required packages and the source to be downloaded and compiled by:

# aptitude update
# aptitude dist-upgrade
# aptitude install fakeroot devscripts build-essential
# apt-get build-dep foo
# apt-get source -b foo

3.4.9. Small public package archive

Here is an example for creating a small public package archive compatible with the modern secure APT system (see Section 3.2.2, “Top level Release file and authenticity”). Let's assume few things:

  • Account name: foo

  • Host name: www.example.com

  • Required packages: apt-utils, gnupg, and other packages.

  • URL: http://www.example.com/~foo/ displays /home/foo/public_html/index.html

  • Architecture of packages: amd64

One time setup of APT archive on your server system:

  • Create Foo's archive key on server system:

$ ssh foo@www.example.com
$ gpg --gen-key
...
$ gpg -K
...
sec   1024D/3A3CB5A6 2008-08-14
uid                  Foo Bar (ARCHIVE KEY) <foo@www.example.com>
ssb   2048g/6856F4A7 2008-08-14
$ gpg --export -a 3A3CB5A6 >foo.public.key
  • New Foo's archive key is 3A3CB5A6

  • Publish foo.public.key file.

  • Create Foo's archive skeleton:

$ umask 022
$ mkdir -p ~/public_html/debian/pool/main
$ mkdir -p ~/public_html/debian/dists/unstable/main/binary-amd64
$ mkdir -p ~/public_html/debian/dists/unstable/main/source
$ cd ~/public_html/debian
$ cat > dists/unstable/main/binary-amd64/Release << EOF
Archive: unstable
Version: 4.0
Component: main
Origin: Foo
Label: Foo
Architecture: amd64
EOF
$ cat > dists/unstable/main/source/Release << EOF
Archive: unstable
Version: 4.0
Component: main
Origin: Foo
Label: Foo
Architecture: source
EOF
$ cat >aptftp.conf <<EOF
APT::FTPArchive::Release {
  Origin "Foo";
  Label "Foo";
  Suite "unstable";
  Codename "sid";
  Architectures "amd64";
  Components "main";
  Description "Foo's public archive";
};
EOF
$ cat >aptgenerate.conf <<EOF
Dir::ArchiveDir ".";
Dir::CacheDir ".";
TreeDefault::Directory "pool/";
TreeDefault::SrcDirectory "pool/";
Default::Packages::Extensions ".deb";
Default::Packages::Compress ". gzip bzip2";
Default::Sources::Compress "gzip bzip2";
Default::Contents::Compress "gzip bzip2";

BinDirectory "dists/unstable/main/binary-amd64" {
  Packages "dists/unstable/main/binary-amd64/Packages";
  Contents "dists/unstable/Contents-amd64";
  SrcPackages "dists/unstable/main/source/Sources";
};

Tree "dists/unstable" {
  Sections "main";
  Architectures "amd64 source";
};
EOF

Repetitive update of APT archive contents on your server system:

  • Place all package files into ~foo/public_html/debian/pool/main/ by executing "dupload -t foo changes_file" in client while having ~/.dupload.conf containing:

$cfg{'foo'} = {
  fqdn => "www.example.com",
  method => "scpb",
  incoming => "/home/foo/public_html/debian/pool/main",
  # The dinstall on ftp-master sends emails itself
  dinstall_runs => 1,
};

$cfg{'foo'}{postupload}{'changes'} = "
  echo 'cd public_html/debian ;
  apt-ftparchive generate -c=aptftp.conf aptgenerate.conf;
  apt-ftparchive release -c=aptftp.conf dists/unstable >dists/unstable/Release ;
  rm -f dists/unstable/Release.gpg ;
  gpg -u 3A3CB5A6 -bao dists/unstable/Release.gpg dists/unstable/Release'| 
  ssh foo@www.example.com  2>/dev/null ;
  echo 'Package archive created!'";

The postupload hook script initiated by dupload(1) creates updated archive files for each upload.

You can add this small public archive to the apt-line of your client system:

$ sudo bash
# echo "deb http://www.example.com/~foo/debian/ unstable main" \
   >> /etc/apt/sources.list
# apt-key add foo.public.key
[Tip] Tip

If the archive is located on the local file system, you can use "deb file:///home/foo/debian/ ..." instead.

3.4.10. Record/copy system configuration

To make a local copy of the package and debconf selection states:

# dpkg --get-selections '*' > selection.dpkg
# debconf-get-selections    > selection.debconf

Here, '*' makes selection.dpkg to include package entries for "purge" too.

You can transfer these 2 files to another computer, and install there with:

# dselect update
# debconf-set-selections < myselection.debconf
# dpkg --set-selections  < myselection.dpkg
# apt-get -u dselect-upgrade    # or dselect install

If you are thinking about managing many cluster of servers with practically the same configuration, you should consider to use specialized package such as fai to manage the whole system.

3.4.11. Convert or install an alien binary package

The alien command enables the conversion of binary packages provided in Red Hat rpm, Stampede slp, Slackware tgz, and Solaris pkg file formats into a Debian deb package. If you want to use a package from another Linux distribution than the one you have installed on your system, you can use alien to convert it to your preferred package format and install it. alien also supports LSB packages.

The original package needs to be statically linked or its library dependency needs to be satisfied manually. So use this command with great care.

3.4.12. Extract package without dpkg

The current .deb package contents can be extracted without using the dpkg command on any unix-like environment using standard ar and tar commands.

# ar x /path/to/dpkg_<version>_<arch>.deb
# ls
total 24
-rw-r--r-- 1 osamu osamu  1320 2007-05-07 00:11 control.tar.gz
-rw-r--r-- 1 osamu osamu 12837 2007-05-07 00:11 data.tar.gz
-rw-r--r-- 1 osamu osamu     4 2007-05-07 00:11 debian-binary
# mkdir control
# mkdir data
# tar xvzf control.tar.gz -C control
# tar xvzf data.tar.gz -C data

You can also browse package content using the mc command.

3.4.13. More readings for the package management

You should read:

  • manpages for aptitude(8), "dpkg", "man 8 tasksel", "man 8 apt-get", "man 8 apt-config", "man 8 apt-key", "man 5 sources.list", "man 5 apt.conf", and "man 5 apt_preferences";

  • "/usr/share/doc/apt-doc/guide.html/index.html" and "/usr/share/doc/apt-doc/offline.html/index.html" from the apt-doc package;

  • Read /usr/share/doc/aptitude/html/en/index.html from the aptitude-doc-en package.

The official and detailed secondary information on the Debian archive are given by:

The tutorial for building of a Debian package for the common Debian user is given by: