Copyright © 2007-2009 Osamu Aoki
Abstract
This book is free; you may redistribute it and/or modify it under the terms of the GNU General Public License of any version compliant to the Debian Free Software Guidelines (DFSG).
Table of Contents
List of Tables
ls -l
" output
chmod
(1) commands.
$HOME
" values.
aptitude
(8).
pam_unix
(8).
/etc/passwd
".
/etc/network/interfaces
"
net-tools
commands to new iproute2
commands.
ls -l
" command for lenny
.
/etc/fstab
" entry.
cvs
(1)).
bsdmainutils
and coreutils
packages.
This Debian Reference (version 2) (2009-06-26 15:13:16 UTC) is intended to provide a broad overview of Debian system administration as a post-installation user guide.
The target reader is someone who is willing to learn shell scripts but who is not ready to read all the C sources to figure out how the GNU/Linux system works.
All warranties are disclaimed. All trademarks are property of their respective trademark owners.
The Debian system itself is a moving target. This makes its documentation difficult to be current and correct. Although the current unstable version of Debian system was used as the basis for writing this, some contents may be already outdated by the time you read this.
Please treat this document as the secondary reference. This document does not replace any authoritative guides. The author and contributors do not take responsibility for consequences of errors, omissions or ambiguity in this document.
The Debian Project is an association of individuals who have made common cause to create a free operating system. It's distribution is characterized by:
unstable
and testing
archives.
Free Software pieces in Debian come from GNU, Linux, BSD, X, ISC, Apache, Ghostscript, Common Unix Printing System , Samba, GNOME, KDE, Mozilla, OpenOffice.org, Vim, TeX, LaTeX, DocBook, Perl, Python, Tcl, Java, Ruby, PHP, Berkeley DB, MySQL, PostgreSQL, Exim, Postfix, Mutt, FreeBSD, OpenBSD, Plan 9 and many more independent free software projects. Debian integrates this diversity of Free Software into one system.
Following guiding rules were followed while compiling this document:
I tried to elucidate hierarchical aspects and lower levels of the system.
You are required to seek help from (in approximate order of importance, starting with the most important sources):
/usr/share/doc/<package_name>
" directory,
dpkg -L <package_name> |grep '/man/man.*/'
",
dpkg -L <package_name> |grep '/info/'
",
![]() |
Note |
---|---|
For detailed documentation, you may need to install the corresponding documentation package named with " |
This document provides information through the following simplified presentation style with bash
(1) shell command examples and bullets:
# <command in root account> $ <command in user account>
These shell prompts distinguish account used and correspond to set environment variables as: "PS1='\$'
" and "PS2=' '
". These values are chosen for the sake of readability of this document and are not typical on actual installed system.
![]() |
Note |
---|---|
See the meaning of the " |
A command snippet quoted in a text paragraph is referred by the typewriter font between double quotation marks, such as "aptitude safe-upgrade
".
A text data from a configuration file quoted in a text paragraph is referred by the typewriter font between double quotation marks, such as "deb-src
".
A command is referred by its name in the typewriter font optionally followed by its manpage section number in parenthesis, such as bash
(1). You are encouraged to obtain information by typing:
$ man 1 bash
A manpage is referred by its name in the typewriter font followed by its manpage section number in parenthesis, such as sources.list
(5). You are encouraged to obtain information by typing:
$ man 5 sources.list
An info page is referred by its command snippet in the typewriter font between double quotation marks, such as "info make
". You are encouraged to obtain information by typing:
$ info make
A filename is referred by the typewriter font between double quotation marks, such as "/etc/passwd
". For configuration files, you are encouraged to obtain information by typing:
$ sensible-pager "/etc/passwd"
A directory name is referred by the typewriter font between double quotation marks, such as "/etc/init.d/
". You are encouraged to explore its contents by typing:
$ mc "/etc/init.d/"
A package name is referred by its name in the typewriter font, such as vim
. You are encouraged to obtain information by typing:
$ dpkg -L vim $ apt-cache show vim $ aptitude show vim
A documentation may indicate its location by the filename in the typewriter font between double quotation marks, such as "/usr/share/doc/sysv-rc/README.runlevels.gz
" and "/usr/share/doc/base-passwd/users-and-groups.html
"; or by its URL, such as http://www.debian.org. You are encouraged to read the documentation by typing:
$ zcat "/usr/share/doc/sysv-rc/README.runlevels.gz" | sensible-pager $ sensible-browser "/usr/share/doc/base-passwd/users-and-groups.html" $ sensible-browse "http://www.debian.org"
An environment variable is referred by its name with leading "$
" in the typewriter font between double quotation marks, such as "$TERM
". You are encouraged to obtain its current value by typing:
$ echo "$TERM"
The popcon data is presented as the objective measure for the popularity of each package. It was downloaded on 2009-06-21 15:05:53 UTC and contains the total submission of 84969 reports over 96434 binary packages and 19 architectures.
![]() |
Note |
---|---|
Please note that the |
The popcon number preceded with "V:" for "votes" is calculated by "100 * (the popcon submissions for the package executed recently on the PC)/(the total popcon submissions)".
The popcon number preceded with "I:" for "installs" is calculated by "100 * (the popcon submissions for the package installed on the PC)/(the total popcon submissions)".
![]() |
Note |
---|---|
The popcon figures should not be considered as absolute measures of the importance of packages. There are many factors which can skew statistics. For example, some system participating popcon may have mounted directories such as " |
The package size data is also presented as the objective measure for each package. It is based on the "Installed-Size:
" reported by "apt-cache show
" or "aptitude show
" command (currently on amd64
architecture for the unstable
release). The reported size is in KiB (Kibibyte = unit for 1024 bytes).
![]() |
Note |
---|---|
A package with a small numerical package size may indicate that the package in the |
Here are some interesting quotes from the Debian mailing list which may help enlighten new users:
<miquels at cistron.nl>
<tollef at add.no>
I think learning a computer system is like learning a new foreign language. Although tutorial books and documentation are helpful, you have to practice it yourself. In order to help you get started smoothly, I will elaborate a few basic points.
The powerful design of Debian GNU/Linux comes from the Unix operating system, i.e., a multiuser, multitasking operating system. You must learn to take advantage of the power of these features and similarities between Unix and GNU/Linux.
Don't shy away from Unix oriented texts and don't rely solely on GNU/Linux texts, as this will rob you of much useful information.
"Rute User's Tutorial and Exposition", in the Debian non-free archive as the rutebook
package (popcon: I:0.2), provides a good online resource to the generic system administration.
![]() |
Note |
---|---|
If you have been using any Unix-like system for a while with command line tools, you probably know everything I explain here. Please use this as a reality check and refresher. |
Upon starting the system, you are presented with the character based login screen if you did not install X Window System with the display manager such as gdm
. Suppose your hostname is foo
, the login prompt looks like:
foo login:
If you did install a GUI environment such as GNOME or KDE, then you can get to a login prompt by Ctrl-Alt-F1, and you can return to the GUI environment via Alt-F7 (see Section 1.1.6, “Virtual consoles” below for more).
At the login prompt, you type your username, e.g. penguin
, and press the Enter-key, then type your password and press the Enter-key again.
![]() |
Note |
---|---|
Following the Unix tradition, the username and password of the Debian system are case sensitive. The username is usually chosen only from the lowercase. The first user account is usually created during the installation. Additional user accounts can be created with |
The system starts with the greeting message stored in "/etc/motd
" (Message Of The Day) and with the command prompt as:
Debian GNU/Linux lenny/sid foo tty1 foo login: penguin Password: Last login: Sun Apr 22 09:29:34 2007 on tty1 Linux snoopy 2.6.20-1-amd64 #1 SMP Sun Apr 15 20:25:49 UTC 2007 x86_64 The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. foo:~$
Here, the main part of the greeting message can be customized by editing the "/etc/motd.tail
" file. The first line is generated from the system information using "uname -snrvm
".
Now you are in the shell. The shell interprets your commands.
If you installed X Window System with a display manager such as GNOME's gdm
by selecting "Desktop environment" task during the installation, you will be presented with the graphical login screen upon starting your system. You type your username and your password to login to the non-privileged user account. Use tab to navigate between username and password, or use the mouse and primary click.
You can gain the shell prompt under X by starting a x-terminal-emulator
program such as gnome-terminal
(1), rxvt
(1) or xterm
(1). Under the GNOME Desktop environment, clicking "Applications" → "Accessories" → "Terminal" does the trick.
You can also see the section below Section 1.1.6, “Virtual consoles”.
Under some other Desktop systems (like fluxbox
), there may be no obvious starting point for the menu. If this happens, just try (right) clicking the center of the screen and hope for a menu to pop-up.
The root account is also called superuser or privileged user. From this account, you can perform the following system administration activities:
This unlimited power of root account requires you to be considerate and responsible when using it.
![]() |
Warning |
---|---|
Never share the root password with others. |
![]() |
Note |
---|---|
File permissions of a file (including hardware devices such as CD-ROM etc. which are just another file for the Debian system) may render it unusable or inaccessible by non-root users. Although the use of root account is a quick way to test this kind of situation, its resolution should be done through proper setting of file permissions and user's group membership (see Section 1.2.3, “Filesystem permissions”). |
Here are a few basic methods to gain the root shell prompt by using the root password:
root
.
su -l
". (This does not preserve the environment of the current user)
su
". (This preserves some of the environment of the current user)
When your desktop menu does not start GUI system administration tools automatically with the appropriate privilege, you can start them from the root shell prompt of the X terminal emulator, such as gnome-terminal
(1), rxvt
(1), or xterm
(1). See Section 1.1.4, “The root shell prompt” and Section 7.8.4, “Running X clients as root”.
![]() |
Warning |
---|---|
Never start the X display/session manager under the root account by typing in |
![]() |
Warning |
---|---|
Never run untrusted remote GUI program under X window when critical information is displayed since it may eavesdrop your X screen. |
In the default Debian system, there are six switchable VT100-like character consoles available to start the command shell directly on the Linux host. Unless you are in a GUI environment, you can switch between the virtual consoles by pressing the Left-Alt-key
and one of the F1
— F6
keys simultaneously. Each character console allows independent login to the account and offers the multiuser environment. This multiuser environment is a great Unix feature, and very addictive.
If you are under the X Window System, you gain access to the character console 1 by pressing Ctrl-Alt-F1
key, i.e., the left-Ctrl-key
, the left-Alt-key
, and the F1-key
are pressed together. You can get back to the X Window System, normally running on the virtual console 7, by pressing Alt-F7
.
You can alternatively change to another virtual console, e.g. to the console 1, by the command:
# chvt 1
You type Ctrl-D
, i.e., the left-Ctrl-key
and the d-key
pressed together, at the command prompt to close the shell activity. If you are at the character console, you will return to the login prompt with this. Even though these control characters are referred as "control D" with the upper case, you do not need to press the Shift-key. The short hand expression, ^D
, is also used for Ctrl-D
. Alternately, you can type "exit".
If you are at x-terminal-emulator
(1), you can close x-terminal-emulator
window with this.
Just like any other modern OS where the file operation involves caching data in memory for improved performance, the Debian system needs the proper shutdown procedure before power can safely be turned off. This is to maintain the integrity of files, by forcing all changes in memory to be written to disk. If the software power control is available, the shutdown procedure automatically turns off power of the system. (Otherwise, you may have to press power button for few seconds after the shutdown procedure.)
Under the normal multiuser mode, use following from the root command prompt to shutdown the system:
# shutdown -h now
Under the single-user mode, use following from the root command prompt to shutdown the system:
# poweroff -i -f
Alternatively, you may type Ctrl-Alt-Delete
(The left-Ctrl-key
, the left-Alt-Key
, and the Delete
are pressed together) to shutdown if "/etc/inittab
" contains "ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -h now
" in it. See inittab
(5) for details.
When the screen goes berserk after doing some funny things such as "cat <some-binary-file>
", type "reset
" at the command prompt. You may not be able to see the command echoed as you type. You may also issue "clear
" to clean up the screen.
Although even the minimal installation of the Debian system without any desktop environment tasks provides the basic Unix functionality, it is a good idea to install few additional commandline and curses based character terminal packages such as mc
and vim
with aptitude
(8) for beginners to get started. From the shell prompt as root:
# aptitude update ... # aptitude install mc vim sudo ...
If you already had these packages installed, nothing will be installed.
Table 1.1. List of interesting text-mode program packages.
package | popcon | size | description |
---|---|---|---|
mc
|
V:12, I:27 | 6364 | A text-mode full-screen file manager |
sudo
|
V:44, I:74 | 592 | A program to allow limited root privileges to users |
vim
|
V:14, I:30 | 1740 | Unix text editor Vi IMproved, a programmers text editor (standard version) |
vim-tiny
|
V:18, I:90 | 828 | Unix text editor Vi IMproved, a programmers text editor (compact version) |
emacs21
|
V:3, I:7 | 8176 | GNU project Emacs, the Lisp based extensible text editor (version 21) |
emacs22
|
V:4, I:7 | 11032 | GNU project Emacs, the Lisp based extensible text editor (version 22) |
w3m
|
V:23, I:85 | 1968 | Text-mode WWW browsers |
gpm
|
V:3, I:5 | 564 | The Unix style cut-and-paste on the text console (daemon) |
It may be a good idea to read some informative documentations.
Table 1.2. List of informative documentation packages.
package | popcon | size | description |
---|---|---|---|
doc-debian
|
I:83 | 376 | Debian Project documentation, (Debian FAQ) and other documents |
debian-policy
|
I:2 | 2740 | Debian Policy Manual and related documents |
developers-reference
|
I:1.2 | 1348 | Guidelines and information for Debian developers |
maint-guide
|
I:0.9 | 644 | Debian New Maintainers' Guide |
debian-history
|
I:0.4 | 2544 | History of the Debian Project |
debian-faq
|
I:45 | 1190 | Debian FAQ |
doc-linux-text
|
I:83 | 8616 | Linux HOWTOs and FAQ (text) |
doc-linux-html
|
I:1.0 | 62564 | Linux HOWTOs and FAQ (html) |
sysadmin-guide
|
I:0.3 | 964 | The Linux System Administrators' Guide |
rutebook
|
I:0.2 | 8264 | Linux: Rute User's Tutorial and Exposition (non-free) |
You can install some of these packages by issuing the following command from the root shell prompt:
# aptitude install package_name
If you do not want to use your main user account for the following training activities, you can create a training user account, e.g. fish
. Type at root shell prompt:
# adduser fish
This will create a new account named as fish
. After your practice, you can remove this user account and its home directory by:
# deluser --remove-home fish
For the typical single user workstation such as the desktop Debian system on the laptop PC, it is common to deploy simple configuration of sudo
(8) as follows to let the non-privileged user, e.g. penguin
, to gain administrative privilege just with his user password (not with the root password).
# echo "penguin ALL=(ALL) ALL" >> /etc/sudoers
This trick should only be used for the single user workstation which you administer and where you are the only user.
![]() |
Warning |
---|---|
Do not set up accounts of regular users on multiuser workstation like this because it would be very bad for system security. |
![]() |
Caution |
---|---|
The password and the account of the |
![]() |
Caution |
---|---|
Administrative privilege in this context belongs to someone authorized to perform the system administration task on the workstation. Never give some manager in the Admin department of your company or your boss such privilege unless they are authorized and capable. |
![]() |
Note |
---|---|
For providing access privilege to limited devices and limited files, you should consider to use group to provide limited access instead of using the |
![]() |
Note |
---|---|
With more thoughtful and careful configuration, |
Now you are ready to play with the Debian system without risks as long as you use the non-privileged user account.
This is because the Debian system is, even after the default installation, configured with proper file permissions which prevent non-privileged users from damaging the system. Of course, there may still be some holes which can be exploited but those who worry about these issues should not be reading this section but should be reading Securing Debian Manual.
We will learn the Debian system as a Unix-like system with:
In GNU/Linux and other Unix-like operating systems, files are organized into directories. All files and directories are arranged in one big tree rooted at "/
". It's called a tree because if you draw the file system, it looks like a tree (upside down).
These files and directories can be spread out over several devices. mount
(8) serves to attach the file system found on some device to the big file tree. Conversely, umount
(8) will detach it again. On recent Linux kernels, mount
(8) with some options can bind part of a file tree somewhere else or can mount filesystem as shared, private, slave, or unbindable. Supported mount options for each filesystem are available in "/share/doc/linux-doc-2.6.*/Documentation/filesystems/
".
Directories on Unix systems are called folders on some other systems. Please also note that there is no concept for drive such as "A:
" on any Unix system. There is one file system, and everything is included. This is a huge advantage compared to Windows.
Here are Unix file basics:
MYFILE
" and "MyFile
" are different files.
/
". Don't confuse this with the home directory for the root user: "/root
".
/
". The root directory is an exception; its name is "/
" (pronounced "slash" or "the root directory") and it cannot be renamed.
/
" directory, and there's a "/
" between each directory or file in the filename. The first "/
" is the top level directory, and the other "/
"'s separate successive subdirectories, until we reach the last entry which is the name of the actual file. The words used here can be confusing. Take the following fully-qualified filename as an example: "/usr/share/keytables/us.map.gz
". However, people will also refer to its basename "us.map.gz
" alone as a filename.
/etc/
" and "/usr/
". These subdirectories in turn branch into still more subdirectories, such as "/etc/init.d/
" and "/usr/local/
". The whole thing viewed collectively is called the directory tree. You can think of an absolute filename as a route from the base of the tree ("/
") to the end of some branch (a file). You will also hear people talk about the directory tree as if it were a family tree: thus subdirectories have parents, and a path shows the complete ancestry of a file. There are also relative paths that begin somewhere other than the root directory. You should remember that the directory "../
" refers to the parent directory. This terminology also applies to other directory like structures, such as hierarchical data structures.
C:\
". (However, directory entries do exist that refer to physical devices as a part of the normal filesystem. See Section 1.2.2, “Filesystem internals”.)
![]() |
Note |
---|---|
While you can use almost any letters or symbols in a file name, in practice it is a bad idea to do so. It is better to avoid any characters that often have special meanings on the command line, including spaces, tabs, newlines, and other special characters: |
![]() |
Note |
---|---|
The word "root" can mean either "root user" or "root directory". The context of their usage should make it clear. |
![]() |
Note |
---|---|
The word path is used not only for fully-qualified filename as above but also for the command search path. The intended meaning is usually clear from the context. |
The detailed best practices for the file hierarchy are described in the Filesystem Hierarchy Standard ("/usr/share/doc/debian-policy/fhs/fhs-2.3.txt.gz
" and hier
(7)). You should remember the following facts as the starter:
Table 1.3. List of usage of key directories.
directory | usage |
---|---|
/
|
A simple "/ " represents the root directory.
|
/etc/
|
This is the place for the system wide configuration files. |
/var/log/
|
This is the place for the system log files. |
/home/
|
This is the directory which contains all the home directories for all non-privileged users. |
Following the Unix tradition, the Debian GNU/Linux system provides the filesystem under which physical data on harddisks and other storage devices reside, and the interaction with the hardware devices such as console screens and remote serial consoles are represented in an unified manner under "/dev/
".
Each file, directory, named pipe (a way two programs can share data), or physical device on a Debian GNU/Linux system has a data structure called an inode which describes its associated attributes such as the user who owns it (owner), the group that it belongs to, the time last accessed, etc. If you are really interested, see "/usr/include/linux/fs.h
" for the exact definition of "struct inode
" in the Debian GNU/Linux system. The idea of representing just about everything in the file system was a Unix innovation, and modern Linux kernels have developed this idea ever further. Now, even information about processes running in the computer can be found in the file system.
This abstract and unified representation of physical entities and internal processes is very powerful since this allows us to use the same command for the same kind of operation on many totally different devices. It is even possible to change the way the kernel works by writing data to special files that are linked to running processes.
![]() |
Tip |
---|---|
If you need to identify the correspondence between the file tree and the physical entity, execute |
Filesystem permissions of Unix-like system are defined for three categories of affected users:
For the file, each corresponding permission allows:
For the directory, each corresponding permission allows:
Here, the execute permission on a directory means not only to allow reading of files in that directory but also to allow viewing their attributes, such as the size and the modification time.
ls
(1) is used to display permission information (and more) for files and directories. When it is invoked with the "-l
" option, it displays the following information in the order given:
Table 1.4. List of the first character of "ls -l
" output
character | meaning |
---|---|
-
|
normal file |
d
|
directory |
l
|
symlink |
c
|
character device node |
b
|
block device node |
p
|
named pipe |
s
|
socket |
chown
(1) is used from the root account to change the owner of the file. chgrp
(1) is used from the file's owner or root account to change the group of the file. chmod
(1) is used from the file's owner or root account to change file and directory access permissions. Basic syntax to manipulate a foo
file is:
# chown <newowner> foo # chgrp <newgroup> foo # chmod [ugoa][+-=][rwxXst][,...] foo
For example, in order to make a directory tree to be owned by a user foo
and shared by a group bar
, issue following commands from the root account:
# cd /some/location/ # chown -R foo:bar . # chmod -R ug+rwX,o=rX .
There are three more special permission bits:
Here the output of "ls -l
" for these bits is capitalized if execution bits hidden by these outputs are unset.
Setting set user ID on an executable file allows a user to execute the executable file with the owner ID of the file (for example root). Similarly, setting set group ID on an executable file allows a user to execute the executable file with the group ID of the file (for example root). Because these settings can cause security risks, enabling them requires extra caution.
Setting set group ID on a directory enables the BSD-like file creation scheme where all files created in the directory belong to the group of the directory.
Setting the sticky bit on a directory prevents a file in the directory from being removed by a user who is not the owner of the file. In order to secure contents of a file in world-writable directories such as "/tmp
" or in group-writable directories, one must not only reset the write permission for the file but also set the sticky bit on the directory. Otherwise, the file can be removed and a new file can be created with the same name by any user who has write access to the directory.
Here are a few interesting examples of file permissions.
$ ls -l /etc/passwd /etc/shadow /dev/ppp /usr/sbin/exim4 crw------- 1 root root 108, 0 2007-04-29 07:00 /dev/ppp -rw-r--r-- 1 root root 1427 2007-04-16 00:19 /etc/passwd -rw-r----- 1 root shadow 943 2007-04-16 00:19 /etc/shadow -rwsr-xr-x 1 root root 700056 2007-04-22 05:29 /usr/sbin/exim4 $ ls -ld /tmp /var/tmp /usr/local /var/mail /usr/src drwxrwxrwt 10 root root 4096 2007-04-29 07:59 /tmp drwxrwsr-x 10 root staff 4096 2007-03-24 18:48 /usr/local drwxrwsr-x 4 root src 4096 2007-04-27 00:31 /usr/src drwxrwsr-x 2 root mail 4096 2007-03-28 23:33 /var/mail drwxrwxrwt 2 root root 4096 2007-04-29 07:11 /var/tmp
There is an alternative numeric mode to describe file permissions with chmod
(1). This numeric mode uses 3 to 4 digit wide octal (radix=8) numbers.
Table 1.5. The numeric mode for file permissions in chmod
(1) commands.
digit | meaning |
---|---|
1st optional digit | sum of set user ID (=4), set group ID (=2), and sticky bit (=1) |
2nd digit | sum of read (=4), write (=2), and execute (=1) permissions for user |
3rd digit | ditto for group |
4th digit | ditto for other |
This sounds complicated but it is actually quite simple. If you look at the first few (2-10) columns from "ls -l
" command output and read it as a binary (radix=2) representation of file permissions ("-" being "0" and "rwx" being "1"), the last 3 digit of the numeric mode value should make sense as an octal (radix=8) representation of file permissions to you. For example, try:
$ touch foo bar $ chmod u=rw,go=r foo $ chmod 644 bar $ ls -l foo bar -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:22 bar -rw-r--r-- 1 penguin penguin 12 2007-04-29 08:22 foo
![]() |
Tip |
---|---|
If you need to access information displayed by " |
What permissions are applied to a newly created file or directory is restricted by the umask
shell builtin command. See dash
(1), bash
(1), and builtins
(7).
(file permissions) = (requested file permissions) & ~(umask value)
Table 1.6. The umask value examples.
umask | usage | file permissions created | directory permissions created |
---|---|---|---|
0022
|
writable only by the user |
-rw-r--r--
|
-rwxr-xr-x
|
0002
|
writable by the group |
-rw-rw-r--
|
-rwxrwxr-x
|
The Debian system uses a user private group (UPG) scheme as its default. A UPG is created whenever a new user is added to the system. A UPG has the same name as the user for which it was created and that user is the only member of the UPG. UPG scheme makes it is safe to set umask to 0002
since every user has their own private group. (In some Unix variants, it is quite common to setup all normal users belonging to a single users
group and is good idea to set umask to 0022
for security in such cases.)
In order to make group permissions to be applied to a particular user, that user needs to be made a member of the group using "sudo vigr
".
![]() |
Note |
---|---|
Alternatively, you may dynamically add users to groups during the authentication process by adding " |
The hardware devices are just another kind of file on the Debian system. If you have problems accessing devices such as CD-ROM and USB memory stick from a user account, you should make that user a member of the relevant group.
Some notable system-provided groups allow their members to access particular files and devices without root
privilege.
Table 1.7. List of notable system-provided groups for file access.
group | accessible files and devices |
---|---|
dialout
|
Full and direct access to serial ports ("/dev/ttyS[0-3] ").
|
dip
|
Limited access to serial ports for Dialup IP connection to trusted peers. |
cdrom
|
CD-ROM, DVD+/-RW drives. |
audio
|
An audio device. |
video
|
A video device. |
scanner
|
Scanner(s). |
adm
|
System monitoring logs. |
staff
|
Some directories for junior administrative work: "/usr/local ", "/home ".
|
![]() |
Tip |
---|---|
You need to belong to the |
Some notable system-provided groups allow their members to execute particular commands without root
privilege.
Table 1.8. List of notable system provided groups for particular command executions.
group | accessible commands |
---|---|
sudo
|
execute sudo without their password.
|
lpadmin
|
execute commands to add, modify, and remove printers from printer databases. |
plugdev
|
execute pmount (1) for removable devices such as USB memories.
|
For the full listing of the system provided users and groups, see the recent version of the "Users and Groups" document in "/usr/share/doc/base-passwd/users-and-groups.html
" provided by the base-passwd
package.
See passwd
(5), group
(5), shadow
(5), newgrp
(1), vipw
(8), vigr
(8), and pam_group
(8) for management commands of the user and group system.
There are three types of timestamps for a GNU/Linux file.
Table 1.9. List of types of timestamps.
type | meaning |
---|---|
mtime |
the file modification time (ls -l )
|
ctime |
the file status change time (ls -lc )
|
atime |
the last file access time (ls -lu )
|
Note that ctime is not file creation time.
noatime
" or "relatime
" option will let the system skip this operation and will result in faster file access for the read. This is often recommended for laptops, because it reduces hard drive activity and saves power. See mount
(8).
Use touch
(1) command to change timestamps of existing files.
For timestamps, the ls
command outputs different strings under the modern English locale ("en_US.UTF-8
") from under the old one ("C
").
$ LANG=en_US.UTF-8 ls -l foo -rw-r--r-- 1 penguin penguin 3 2008-03-05 00:47 foo $ LANG=C ls -l foo -rw-r--r-- 1 penguin penguin 3 Mar 5 00:47 foo
![]() |
Tip |
---|---|
See Section 9.2.5, “Customized display of time and date” to cutomize " |
There are two methods of associating a file "foo
" with a different filename "bar
".
ln foo bar
),
ln -s foo bar
).
See the following example for changes in link counts and the subtle differences in the result of the rm
command.
$ echo "Original Content" > foo $ ls -li foo 2398521 -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:15 foo $ ln foo bar # hard link $ ln -s foo baz # symlink $ ls -li foo bar baz 2398521 -rw-r--r-- 2 penguin penguin 17 2007-04-29 08:15 bar 2398538 lrwxrwxrwx 1 penguin penguin 3 2007-04-29 08:16 baz -> foo 2398521 -rw-r--r-- 2 penguin penguin 17 2007-04-29 08:15 foo $ rm foo $ echo "New Content" > foo $ ls -li foo bar baz 2398521 -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:15 bar 2398538 lrwxrwxrwx 1 penguin penguin 3 2007-04-29 08:16 baz -> foo 2398540 -rw-r--r-- 1 penguin penguin 12 2007-04-29 08:17 foo $ cat bar Original Content $ cat baz New Content
The hardlink can be made within the same file system and shares the same inode number which the "-i
" option with ls
(1) reveals.
The symlink always has nominal file access permissions of "rwxrwxrwx
", as shown in the above example, with the effective access permissions dictated by permissions of the file that it points to.
![]() |
Caution |
---|---|
It is generally good idea not to create complicated symbolic links or hardlinks at all unless you have a very good reason. It may cause nightmares where the logical combination of the symbolic links results in loops in the filesystem. |
![]() |
Note |
---|---|
It is generally preferable to use symbolic links rather than hardlinks unless you have a good reason for using a hardlink. |
The ".
" directory links to the directory that it appears in, thus the link count of any new directory starts at 2. The "..
" directory links to the parent directory, thus the link count of the directory increases with the addition of new subdirectories.
If you are just moving to Linux from Windows, it will soon become clear how well-designed the filename linking of Unix is, compared with the nearest Windows equivalent of "shortcuts". Because it is implemented in the file system, applications can't see any difference between a linked file and the original. In the case of hardlinks, there really is no difference.
A named pipe is a file that acts like a pipe. You put something into the file, and it comes out the other end. Thus it's called a FIFO, or First-In-First-Out: the first thing you put in the pipe is the first thing to come out the other end.
If you write to a named pipe, the process which is writing to the pipe doesn't terminate until the information being written is read from the pipe. If you read from a named pipe, the reading process waits until there is nothing to read before terminating. The size of the pipe is always zero --- it does not store data, it just links two processes like the shell "|
". However, since this pipe has a name, the two processes don't have to be on the same command line or even be run by the same user. Pipes were a very influential innovation of Unix.
You can try it by doing the following:
$ cd; mkfifo mypipe $ echo "hello" >mypipe & # put into background [1] 8022 $ ls -l mypipe prw-r--r-- 1 penguin penguin 0 2007-04-29 08:25 mypipe $ cat mypipe hello [1]+ Done echo "hello" >mypipe $ ls mypipe mypipe $ rm mypipe
Sockets are used extensively by all the Internet communication, databases, and the operating system itself. It is similar to the named pipe (FIFO) and allows processes to exchange information even between different computers. For the socket, those processes do not need to be running at the same time nor to be running as the children of the same ancestor process. This is the endpoint for the inter process communication (IPC). The exchange of information may occur over the network between different hosts. The two most common ones are the Internet socket and the Unix domain socket.
![]() |
Tip |
---|---|
" |
Device files refer to physical or virtual devices on your system, such as your hard disk, video card, screen, or keyboard. An example of a virtual device is the console, represented by "/dev/console
".
Table 1.10. The device types.
device type | meaning |
---|---|
character device | This can be accessed one character at a time, that is, the smallest unit of data which can be written to or read from the device is a character (byte). |
block device | This must be accessed in larger units called blocks, which contain a number of characters. Your hard disk is a block device. |
You can read and write device files, though the file may well contain binary data which may be an incomprehensible-to-humans gibberish. Writing data directly to these files is sometimes useful for the troubleshooting of hardware connections. For example, you can dump a text file to the printer device "/dev/lp0
" or send modem commands to the appropriate serial port "/dev/ttyS0
". But, unless this is done carefully, it may cause a major disaster. So be cautious.
![]() |
Note |
---|---|
For the normal access to a printer, use |
The device node number are displayed by executing ls
(1) as:
$ ls -l /dev/hda /dev/ttyS0 /dev/zero brw-rw---- 1 root cdrom 3, 0 2007-04-29 07:00 /dev/hda crw-rw---- 1 root dialout 4, 64 2007-04-29 07:00 /dev/ttyS0 crw-rw-rw- 1 root root 1, 5 2007-04-29 07:00 /dev/zero
Here,
/dev/hda
" has the major device number 3 and the minor device number 0. This is read/write accessible by the user who belongs to disk
group,
/dev/ttyS0
" has the major device number 4 and the minor device number 64. This is read/write accessible by the user who belongs to dialout
group, and
/dev/zero
" has the major device number 1 and the minor device number 5. This is read/write accessible by anyone.
In the Linux 2.6 system, the filesystem under "/dev/
" is automatically populated by the udev
(7) mechanism.
There are some special device files.
Table 1.11. List of special device files.
device file | action | response |
---|---|---|
/dev/null
|
read | it returns "end-of-file (EOF) character". |
/dev/null
|
write | it is a bottomless data dump pit. |
/dev/zero
|
read |
it returns "the \0 (NUL) character" (not the same as the number zero ASCII).
|
/dev/random
|
read | it returns random characters from a true random number generator, delivering real entropy. (slow) |
/dev/urandom
|
read | it returns random characters from a cryptographically secure pseudorandom number generator. |
/dev/full
|
write | it returns the disk-full (ENOSPC) error. |
These are frequently used in conjunction with the shell redirection (see Section 1.5.5, “Typical command sequences and shell redirection”).
The procfs and sysfs mounted on "/proc
" and "/sys
" are the pseudo-filesystem and expose internal data structures of the kernel to the userspace. In other word, these entries are virtual, meaning that they act as a convenient window into the operation of the operating system.
The directory "/proc
" contains (among other things) one subdirectory for each process running on the system, which is named after the process ID (PID). System utilities that access process information, such as ps
(1), get their information from this directory structure.
The directories under "/proc/sys/
" contain interface to change certain kernel parameters at run time. (You may do the same through specialized sysctl
(8) command or its preload/configuration file "/etc/sysctrl.conf
".)
![]() |
Note |
---|---|
The Linux kernel may complain "Too many open files". You can fix this by increasing " |
People frequently panic when they notice one file in particular - "/proc/kcore
" - which is generally huge. This is (more or less) a copy of the content of your computer's memory. It's used to debug the kernel. It is a virtual file that points to computer memory, so don't worry about its size.
The directory under "/sys
" contains exported kernel data structures, their attributes, and their linkages between them. It also contains interface to change certain kernel parameters at run time.
See "proc.txt(.gz)
", "sysfs.txt(.gz)
" and other related documents in the Linux kernel documentation ("/usr/share/doc/linux-doc-2.6.*/Documentation/filesystems/*
") provided by the linux-doc-2.6.*
package.
Midnight Commander (MC) is a GNU "Swiss army knife" for the Linux console and other terminal environments. This gives newbie a menu driven console experience which is much easier to learn than standard Unix commands.
You may need to install the Midnight Commander package which is titled "mc
".
$ sudo aptitude install mc
Use the mc
(1) command to explore the Debian system. This is the best way to learn. Please explore few interesting locations just using the cursor keys and Enter key:
/etc
" and its subdirectories.
/var/log
" and its subdirectories.
/usr/share/doc
" and its subdirectories.
/sbin
" and "/bin
"
In order to make MC to change working directory upon exit and cd
to the directory, I suggest to modify "~/.bashrc
" to include:
. /usr/share/mc/bin/mc.sh
See mc
(1) (under the "-P
" option) for the reason. (If you do not understand what exactly I am talking here, you can do this later.)
MC can be started by:
$ mc
MC takes care of all file operations through its menu, requiring minimal user effort. Just press F1 to get the help screen. You can play with MC just by pressing cursor-keys and function-keys.
![]() |
Note |
---|---|
In some consoles such as |
If you encounter character encoding problem which displays garbage characters, adding "-a
" to MC's command line may help prevent problems.
If this doesn't clear up your display problems with MC, see Section 9.6.6, “The terminal configuration”.
The default is two directory panels containing file lists. Another useful mode is to set the right window to "information" to see file access privilege information, etc. Following are some essential keystrokes. With the gpm
(8) daemon running, one can use a mouse on Linux character consoles, too. (Make sure to press the shift-key to obtain the normal behavior of cut and paste in MC.)
Table 1.12. The key bindings of MC.
key | key binding |
---|---|
F1
|
Help menu |
F3
|
Internal file viewer |
F4
|
Internal editor |
F9
|
Activate pull down menu |
F10
|
Exit Midnight Commander |
Tab
|
Move between two windows |
Insert or Ctrl-T
|
Mark file for a multiple-file operation such as copy |
Del
|
Delete file (be careful---set MC to safe delete mode) |
Cursor keys | Self-explanatory |
cd
command will change the directory shown on the selected screen.
Ctrl-Enter
or Alt-Enter
will copy a filename to the command line. Use this with cp
(1) and mv
(1) commands together with command-line editing.
Alt-Tab
will show shell filename expansion choices.
mc /etc /root
".
Esc
+ n-key
→ Fn
(i.e., Esc
+ 1
→ F1
, etc.; Esc
+ 0
→ F10
)
Esc
before the key has the same effect as pressing the Alt
and the key together.; i.e., type Esc
+ c
for Alt-C
. Esc
is called meta-key and sometimes noted as "M-
"
The internal editor has an interesting cut-and-paste scheme. Pressing F3
marks the start of a selection, a second F3
marks the end of selection and highlights the selection. Then you can move your cursor. If you press F6, the selected area will be moved to the cursor location. If you press F5, the selected area will be copied and inserted at the cursor location. F2
will save the file. F10
will get you out. Most cursor keys work intuitively.
This editor can be directly started on a file:
$ mc -e filename_to_edit $ mcedit filename_to_edit
This is not a multi-window editor, but one can use multiple Linux consoles to achieve the same effect. To copy between windows, use Alt-F<n> keys to switch virtual consoles and use "File→Insert file" or "File→Copy to file" to move a portion of a file to another file.
This internal editor can be replaced with any external editor of choice.
Also, many programs use the environment variables "$EDITOR
" or "$VISUAL
" to decide which editor to use. If you are uncomfortable with vim
(1) or nano
(1) initially, you may set these to "mcedit
" by adding these lines to "~/.bashrc
":
... export EDITOR=mcedit export VISUAL=mcedit ...
I do recommend setting these to "vim
" if possible.
If you are uncomfortable with vim
(1), you can keep using mcedit
(1) for most system maintenance tasks.
Very smart viewer. This is a great tool for searching words in documents. I always use this for files in the "/usr/share/doc
" directory. This is the fastest way to browse through masses of Linux information. This viewer can be directly started like so:
$ mc -v path/to/filename_to_view $ mcview path/to/filename_to_view
Press Enter on a file, and the appropriate program will handle the content of the file (see Section 9.5.11, “Customizing program to be started”). This is a very convenient MC feature.
Table 1.13. The reaction to the enter key in MC.
file type | reaction to enter key |
---|---|
executable file | Execute command |
man file | Pipe content to viewer software |
html file | Pipe content to web browser |
"*.tar.gz " and "*.deb " file
|
Browse its contents as if subdirectory |
In order to allow these viewer and virtual file features to function, viewable files should not be set as executable. Change their status using chmod
(1) or via the MC file menu.
MC can be used to access files over the Internet using FTP. Go to the menu by pressing F9
, then type "p
" to activate the FTP virtual filesystem. Enter a URL in the form "username:passwd@hostname.domainname
", which will retrieve a remote directory that appears like a local one.
Try "[http.us.debian.org/debian]" as the URL and browse the Debian archive.
Although MC enables you to do almost everything, it is very important for you to learn how to use the command line tools invoked from the shell prompt and become familiar with the Unix-like work environment.
You can select your login shell with chsh
(1).
Table 1.14. List of shell programs.
package | popcon | size | POSIX shell | description |
---|---|---|---|---|
bash
|
V:91, I:99 | 1336 | Yes | Bash: the GNU Bourne Again SHell. (de facto standard) |
tcsh
|
V:8, I:55 | 736 | No | TENEX C Shell: an enhanced version of Berkeley csh. |
dash
|
V:3, I:12 | 236 | Yes | The Debian Almquist Shell. Good for shell script. |
zsh
|
V:2, I:5 | 12752 | Yes | Z shell: the standard shell with many enhancements. |
pdksh
|
V:0.3, I:1.2 | 464 | Yes | A public domain version of the Korn shell. |
csh
|
V:0.6, I:1.8 | 404 | No | OpenBSD C Shell, a version of Berkeley csh. |
sash
|
V:0.2, I:1.0 | 836 | Yes |
Stand-alone shell with builtin commands. (Not meant for standard "/bin/sh ".)
|
ksh
|
V:0.4, I:1.5 | 2860 | Yes | The real, AT&T version of the Korn shell. |
rc
|
V:0.09, I:0.7 | 204 | No | An implementation of the AT&T Plan 9 rc shell. |
posh
|
V:0.01, I:0.14 | 232 | Yes |
Policy-compliant Ordinary SHell. A pdksh derivative.
|
In this tutorial chapter, the interactive shell always means bash
.
You can customize bash
(1) behavior by "~/.bashrc
". For example, I added followings to "~/.bashrc
":
# CD upon exiting MC . /usr/share/mc/bin/mc.sh # set CDPATH to good one CDPATH=.:/usr/share/doc:~/Desktop/src:~/Desktop:~ export CDPATH PATH="${PATH}":/usr/sbin:/sbin # set PATH so it includes user's private bin if it exists if [ -d ~/bin ] ; then PATH=~/bin:"${PATH}" fi export PATH EDITOR=vim export EDITOR
![]() |
Tip |
---|---|
You can find more |
In the Unix-like environment, there are few key strokes which have special meanings. Please note that on a normal Linux character console, only the left-hand Ctrl
and Alt
keys work as expected. Here are few notable key strokes to remember.
Table 1.15. List of key bindings for bash.
key | key binding |
---|---|
Ctrl-U
|
Erase line before cursor. |
Ctrl-H
|
Erase a character before cursor. |
Ctrl-D
|
Terminate input. (exit shell if you are using shell) |
Ctrl-C
|
Terminate a running program. |
Ctrl-Z
|
Temporarily stop program by moving it to the background job |
Ctrl-S
|
Halt output to screen. |
Ctrl-Q
|
Reactivate output to screen. |
Ctrl-Alt-Del
|
Reboot/halt the system, see inittab (5).
|
Left-Alt-key (optionally, Windows-key )
|
Meta-key for Emacs and the similar UI. |
Up-arrow
|
Start command history search under bash .
|
Ctrl-R
|
Start incremental command history search under bash .
|
Tab
|
Complete input of the filename to the command line under bash .
|
Ctrl-V
Tab
|
Input Tab without expansion to the command line under bash .
|
![]() |
Tip |
---|---|
The terminal feature of |
Unix style mouse operations are based on the 3 button mouse system.
Table 1.16. List of Unix style mouse operations.
action | response |
---|---|
Left-click-and-drag mouse | Select and copy to the clipboard. |
Left-click | Select the start of selection. |
Right-click | Select the end of selection and copy to the clipboard. |
Middle-click | Paste clipboard at the cursor. |
The center wheel on the modern wheel mouse is considered middle mouse button and can be used for middle-click. Clicking left and right mouse buttons together serves as the middle-click under the 2 button mouse system situation. In order to use a mouse in Linux character consoles, you need to have gpm
(8) running as daemon.
less
(1) is the enhanced pager (file content browser). Hit "h
" for help. It can do much more than more
(1) and can be supercharged by executing "eval $(lesspipe)
" or "eval $(lessfile)
" in the shell startup script. See more in "/usr/share/doc/lessf/LESSOPEN
". The "-R
" option allows raw character output and enables ANSI color escape sequences. See less
(1).
You should become proficient in one of variants of Vim or Emacs programs which are popular in the Unix-like system.
I think getting used to Vim commands is the right thing to do, since Vi-editor is always there in the Linux/Unix world. (Actually, original vi
or new nvi
are programs you find everywhere. I chose Vim instead for newbie since it offers you help through F1
key while it is similar enough and more powerful.)
If you chose either Emacs or XEmacs instead as your choice of the editor, that is another good choice indeed, particularly for programming. Emacs has a plethora of other features as well, including functioning as a newsreader, directory editor, mail program, etc.. When used for programming or editing shell scripts, it intelligently recognizes the format of what you are working on, and tries to provide assistance. Some people maintain that the only program they need on Linux is Emacs. Ten minutes learning Emacs now can save hours later. Having the GNU Emacs manual for reference when learning Emacs is highly recommended.
All these programs usually come with tutoring program for you to learn them by practice. Start Vim by typing "vim
" and press F1-key. You should at least read the first 35 lines. Then do the online training course by moving cursor to "|tutor|
" and pressing Ctrl-]
.
![]() |
Note |
---|---|
Good editors, such as Vim and Emacs, can be used to handle UTF-8 and other exotic encoding texts correctly with proper option in the x-terminal-emulator on X under UTF-8 locale with proper font settings. Please refer to their documentation on multibyte text. |
Debian comes with a number of different editors. We recommend to install the vim
package, as mentioned above.
Debian provides unified access to the system default editor via command "/usr/bin/editor
" so other programs (e.g., reportbug
(1)) can invoke it. You can change it by:
$ sudo update-alternatives --config editor
The choice "/usr/bin/vim.basic
" over "/usr/bin/vim.tiny
" is my recommendation for newbies since it supports syntax highlighting.
![]() |
Tip |
---|---|
Many programs use the environment variables " |
You can customize vim
(1) behavior by "~/.vimrc
". For example, I use:
" ------------------------------- " Local configuration " set nocompatible set nopaste set pastetoggle=<f2> syn on if $USER == "root" set nomodeline set noswapfile else set modeline set swapfile endif " filler to avoid the line above being recognized as a modeline " filler " filler
The output of the shell command may roll off your screen and may be lost forever. It is good practice to log shell activities into the file for you to review them later. This kind of record is essential when you perform any system administration tasks.
The basic method of recording the shell activity is to run it under script
(1).
$ script Script started, file is typescript
Ctrl-D
to exit script
.
$ vim typescript
See Section 9.2.3, “Recording the shell activities cleanly” .
Let's learn basic Unix commands. Here I use "Unix" in its generic sense. Any Unix clone OSs usually offer equivalent commands. The Debian system is no exception. Do not worry if some commands do not work as you wish now. If alias
is used in the shell, its corresponding command outputs are different. These examples are not meant to be executed in this order.
Try all following commands from the non-privileged user account:
Table 1.17. List of basic Unix commands.
command | description |
---|---|
pwd
|
Display name of current/working directory. |
whoami
|
Display current user name. |
id
|
Display current user identity (name, uid, gid, and associated groups). |
file <foo>
|
Display a type of file for the file "<foo> ".
|
type -p <commandname>
|
Display a file location of command "<commandname> ".
|
which <commandname>
|
, , |
type <commandname>
|
Display information on command "<commandname> ".
|
apropos <key-word>
|
Find commands related to "<key-word> ".
|
man -k <key-word>
|
, , |
whatis <commandname>
|
Display one line explanation on command "<commandname> ".
|
man -a <commandname>
|
Display explanation on command "<commandname> ". (Unix style)
|
info <commandname>
|
Display rather long explanation on command "<commandname> ". (GNU style)
|
ls
|
List contents of directory. (non-dot files and directories) |
ls -a
|
List contents of directory. (all files and directories) |
ls -A
|
List contents of directory. (almost all files and directories, i.e., skip ".. " and ". ")
|
ls -la
|
List all contents of directory with detail information. |
ls -lai
|
List all contents of directory with inode number and detail information. |
ls -d
|
List all directories under the current directory. |
tree
|
Display file tree contents. |
lsof <foo>
|
List open status of file "<foo> ".
|
lsof -p <pid>
|
List files opened by the process ID: "<pid> ".
|
mkdir <foo>
|
Make a new directory "<foo> " in the current directory.
|
rmdir <foo>
|
Remove a directory "<foo> " in the current directory.
|
cd <foo>
|
Change directory to the directory "<foo> " in the current directory or in the directory listed in the variable "$CDPATH ".
|
cd /
|
Change directory to the root directory. |
cd
|
Change directory to the current user's home directory. |
cd /<foo>
|
Change directory to the absolute path directory "/<foo> ".
|
cd ..
|
Change directory to the parent directory. |
cd ~<foo>
|
Change directory to the home directory of the user "<foo> ".
|
cd -
|
Change directory to the previous directory. |
</etc/motd pager
|
Display contents of "/etc/motd " using the default pager.
|
touch <junkfile>
|
Create a empty file "<junkfile> ".
|
cp <foo> <bar>
|
Copy a existing file "<foo> " to a new file "<bar> ".
|
rm <junkfile>
|
Remove a file "<junkfile> ".
|
mv <foo> <bar>
|
Rename an existing file "<foo> " to a new name "<bar> ". The directory "<bar> " must not exist.
|
mv <foo> <bar>
|
Move an existing file "<foo> " to a new location "<bar>/<foo> ". The directory "<bar> " must exist.
|
mv <foo> <bar>/<baz>
|
Move an existing file "<foo> " to a new location with a new name "<bar>/<baz> ". The directory "<bar> " must exist but the directory "<bar>/<baz> " must not exist.
|
chmod 600 <foo>
|
Make an existing file "<foo> " to be non-readable and non-writable by the other people. (non-executable for all)
|
chmod 644 <foo>
|
Make an existing file "<foo> " to be readable but non-writable by the other people. (non-executable for all)
|
chmod 755 <foo>
|
Make an existing file "<foo> " to be readable but non-writable by the other people. (executable for all)
|
find . -name <pattern>
|
find matching filenames using shell "<pattern> ". (slower)
|
locate -d . <pattern>
|
find matching filenames using shell "<pattern> ". (quicker using regularly generated database)
|
grep -e "<pattern>" *.html
|
Find a "<pattern>" in all files ending with ".html " in current directory and display them all.
|
top
|
Display process information using full screen. Type "q " to quit.
|
ps aux | pager
|
Display information on all the running processes using BSD style output. |
ps -ef | pager
|
Display information on all the running processes using Unix system-V style output. |
ps aux | grep -e "[e]xim4*"
|
Display all processes running "exim " and "exim4 ".
|
ps axf | pager
|
Display information on all the running processes with ASCII art output. |
kill <1234>
|
Kill a process identified by the process ID: "<1234>". |
gzip <foo>
|
Compress "<foo> " to create "<foo>.gz " using the Lempel-Ziv coding (LZ77).
|
gunzip <foo>.gz
|
Decompress "<foo>.gz " to create "<foo> ".
|
bzip2 <foo>
|
Compress "<foo> " to create "<foo>.bz2 " using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. (Better compression than gzip )
|
bunzip2 <foo>.bz2
|
Decompress "<foo>.bz2 " to create "<foo> ".
|
tar -xvf <foo>.tar
|
Extract files from "<foo>.tar " archive.
|
tar -xvzf <foo>.tar.gz
|
Extract files from gzipped "<foo>.tar.gz " archive.
|
tar -xvf -j <foo>.tar.bz2
|
Extract files from "<foo>.tar.bz2 " archive.
|
tar -cvf <foo>.tar <bar>/
|
Archive contents of folder "<bar>/ " in "<foo>.tar " archive.
|
tar -cvzf <foo>.tar.gz <bar>/
|
Archive contents of folder "<bar>/ " in compressed "<foo>.tar.gz " archive.
|
tar -cvjf <foo>.tar.bz2 <bar>/
|
Archive contents of folder "<bar>/ " in "<foo>.tar.bz2 " archive.
|
zcat README.gz | pager
|
Display contents of compressed "README.gz " using the default pager.
|
zcat README.gz > foo
|
Create a file "foo " with the decompressed content of "README.gz ".
|
zcat README.gz >> foo
|
Append the decompressed content of "README.gz " to the end of the file "foo ". (If it does not exist, create it first.)
|
![]() |
Note |
---|---|
Unix has a tradition to hide filenames which start with " |
![]() |
Note |
---|---|
For |
![]() |
Note |
---|---|
The default pager of the bare bone Debian system is |
![]() |
Note |
---|---|
The " |
Please traverse directories and peek into the system using the above commands as training. If you have questions on any of console commands, please make sure to read the manual page. For example, these commands are the good start:
$ man man $ man bash $ man builtins $ man grep $ man ls
The style of man pages may be a little hard to get used to, because they are rather terse, particularly the older, very traditional ones. But once you get used to it, you come to appreciate their succinctness.
Please note that many Unix-like commands including ones from GNU and BSD will display brief help information if you invoke them in one of the following ways (or without any arguments in some cases):
$ <commandname> --help $ <commandname> -h
Now you have some feel on how to use the Debian system. Let's look deep into the mechanism of the command execution in the Debian system. Here, I have simplified reality for the newbie. See bash
(1) for the exact explanation.
A simple command is a sequence of:
>
, >>
, <
, <<
, etc.)
&&
, ||
, <newline> , ;
, &
, (
, )
)
Values of some environment variables change the behavior of some Unix commands.
Default values of environment variables are initially set by the PAM system and then some of them may be reset by some application programs:
gdm
, and
~/bash_profile
" and "~/.bashrc
".
The full locale value given to "$LANG
" variable consists of 3 parts: "xx_YY.ZZZZ
".
Table 1.18. 3 parts of locale value.
locale value | meaning |
---|---|
xx
|
ISO 639 language codes (lower case) such as "en" |
YY
|
ISO 3166 country codes (upper case) such as "US" |
ZZZZ
|
codeset, always set to "UTF-8" |
For language codes and country codes, see pertinent description in the "info gettext
".
For the codeset on the modern Debian system, you should always set it to UTF-8
unless you specifically want to use the historic one with good reason and background knowledge.
For fine details of the locale configuration, see Section 8.3, “The locale”.
![]() |
Note |
---|---|
The " |
Table 1.19. List of locale recommendations.
Language (area) | locale recommendation |
---|---|
English(USA) |
en_US.UTF-8
|
English(Great_Britain) |
en_GB.UTF-8
|
French(France) |
fr_FR.UTF-8
|
German(Germany) |
de_DE.UTF-8
|
Italian(Italy) |
it_IT.UTF-8
|
Spanish(Spain) |
es_ES.UTF-8
|
Catalan(Spain) |
ca_ES.UTF-8
|
Swedish(Sweden) |
sv_SE.UTF-8
|
Portuguese(Brasil) |
pt_BR.UTF-8
|
Russian(Russia) |
ru_RU.UTF-8
|
Chinese(P.R._of_China) |
zh_CN.UTF-8
|
Chinese(Taiwan_R.O.C.) |
zh_TW.UTF-8
|
Japanese(Japan) |
ja_JP.UTF-8
|
Korean(Republic_of_Korea) |
ko_KR.UTF-8
|
Vietnamese(Vietnam) |
vi_VN.UTF-8
|
Typical command execution uses a shell line sequence like the following:
$ date Sun Jun 3 10:27:39 JST 2007 $ LANG=fr_FR.UTF-8 date dimanche 3 juin 2007, 10:27:33 (UTC+0900)
Here, the program date
(1) is executed in the foreground job. The environment variable "$LANG
" is:
Most command executions usually do not have preceding environment variable definition. For the above example, you can alternatively execute:
$ LANG=fr_FR.UTF-8 $ date dimanche 3 juin 2007, 10:27:33 (UTC+0900)
As you can see here, the output of command is affected by the environment variable to produce French output. If you want the environment variable to be inherited to subprocesses (e.g., when calling shell script), you need to "export" it instead by using:
$ export LANG
![]() |
Tip |
---|---|
When filing a bug report, running and checking the command under " |
See locale
(5) and locale
(7) for "$LANG
" and related environment variables.
![]() |
Note |
---|---|
I recommend you to configure the system environment just by the " |
When you type a command into the shell, the shell searches the command in the list of directories contained in the "$PATH
" environment variable. The value of the "$PATH
" environment variable is also called the shell's search path.
In the default Debian installation, the "$PATH
" environment variable of user accounts may not include "/sbin
" and "/usr/sbin
". For example, the ifconfig
command needs to be issued with full path as "/sbin/ifconfig
". (Similar ip
command is located in "/bin
".)
You can change the "$PATH
" environment variable of Bash shell by "~/.bash_profile
" or "~/.bashrc
" files.
Many commands stores user specific configuration in the home directory and changes their behavior by their contents. The home directory is identified by the environment variable "$HOME
":
Table 1.20. List of "$HOME
" values.
program execution situation |
value of "$HOME "
|
---|---|
program run by the init process (daemon) |
/
|
program run from the normal root shell |
/root
|
program run from the normal user shell |
/home/<normal_user>
|
program run from the normal user GUI desktop menu |
/home/<normal_user>
|
program run as root with "sudo program "
|
/home/<normal_user>
|
program run as root with "sudo -H program "
|
/root
|
![]() |
Tip |
---|---|
Shell expands " |
Some commands take arguments. Arguments starting with "-
" or "--
" are called options and control the behavior of the command.
$ date Mon Oct 27 23:02:09 CET 2003 $ date -R Mon, 27 Oct 2003 23:02:40 +0100
Here the command-line argument "-R
" changes date
(1) behavior to output RFC2822 compliant date string.
Often you want a command to work with a group of files without typing all of them. The filename expansion pattern using the shell glob, (sometimes referred as wildcards), facilitate this need.
Table 1.21. Shell glob patterns.
shell glob pattern | match |
---|---|
*
|
This matches filename (segment) not started with ". ".
|
.*
|
This matches filename (segment) started with ". ".
|
?
|
This matches exactly one character. |
[…]
|
This matches exactly one character with any character enclosed in brackets. |
[a-z]
|
This matches exactly one character with any character between "a " and "z ".
|
[^…]
|
This matches exactly one character other than any character enclosed in brackets (excluding "^ ").
|
For example, try the following and think for yourself:
$ mkdir junk; cd junk; .[^.]*touch 1.txt 2.txt 3.c 4.h .5.txt ..6.txt $ echo *.txt 1.txt 2.txt $ echo * 1.txt 2.txt 3.c 4.h $ echo *.[hc] 3.c 4.h $ echo .* . .. .5.txt ..6.txt $ echo .*[^.]* .5.txt ..6.txt $ echo [^1-3]* 4.h $ cd ..; rm -rf junk
See glob
(7) for more.
![]() |
Note |
---|---|
Unlike normal filename expansion by the shell, the shell pattern " |
![]() |
Note |
---|---|
BASH can be tweaked to change its glob behavior with its shopt builtin options such as " |
Each command returns its exit status (variable: "$?
") as the return value.
Table 1.22. Command exit codes.
command exit status | numeric return value | logical return value |
---|---|---|
success | zero, 0 | TRUE |
error | non-zero, -1 | FALSE |
Thus:
$ [ 1 = 1 ] ; echo $? 0 $ [ 1 = 2 ] ; echo $? 1
![]() |
Note |
---|---|
Please note that, in the logical context for the shell, success is treated as the logical TRUE which has 0 (zero) as its value. This is somewhat non-intuitive and needs to be reminded here. |
Let's try to remember following shell command idioms.
Table 1.23. Shell command idioms.
command idiom (type in one line) | description |
---|---|
command &
|
The command is executed in the subshell in the background.
|
command1 | command2
|
The standard output of command1 is piped to the standard input of command2 . Both commands may be running concurrently.
|
command1 2>&1 | command2
|
Both standard output and standard error of command1 are piped to the standard input of command2 . Both commands may be running concurrently.
|
command1 ; command2
|
The command1 and command2 are executed sequentially.
|
command1 && command2
|
The command1 is executed. If successful, command2 is also executed sequentially. Return success if both command1 and command2 are successful.
|
command1 || command2
|
The command1 is executed. If not successful, command2 is also executed sequentially. Return success if command1 or command2 are successful.
|
command > foo
|
Redirect standard output of command to a file foo . (overwrite)
|
command 2> foo
|
Redirect standard error of command to a file foo . (overwrite)
|
command >> foo
|
Redirect standard output of command to a file foo . (append)
|
command 2>> foo
|
Redirect standard error of command to a file foo . (append)
|
command > foo 2>&1
|
Redirect both standard output and standard error of command to a file "foo ".
|
command < foo
|
Redirect standard input of command to a file foo .
|
command << delimiter
|
Redirect standard input of command to the following lines until "delimiter " is met. (Here documents)
|
command <<- delimiter
|
Redirect standard input of command to the following lines until "delimiter " is met. The leading tab characters are stripped from input lines. (Here documents)
|
The Debian system is a multi-tasking system. Background jobs allow users to run multiple programs in a single shell. The management of the background process involves the shell builtins: jobs
, fg
, bg
, and kill
. Please read sections of bash(1) under "SIGNALS", and "JOB CONTROL", and builtins
(1).
Let's try simple examples of redirection:
$ </etc/motd pager
$ pager </etc/motd
$ pager /etc/motd
$ cat /etc/motd | pager
Although all 4 examples display the same thing, the last example runs an extra cat
command and wastes resources with no reason.
The shell allows you to open files using the exec
builtin with an arbitrary file descriptor.
$ echo Hello >foo $ exec 3<foo 4>bar # open files $ cat <&3 >&4 # redirect stdin to 3, stdout to 4 $ exec 3<&- 4>&- # close files $ cat bar Hello
Here, "n<&-
" and "n>&-
" mean to close the file descriptor "n
".
The file descriptor 0-2 are predefined:
Table 1.24. Predefined file descriptors.
device | description | file descriptor |
---|---|---|
stdin
|
standard input | 0 |
stdout
|
standard output | 1 |
stderr
|
standard error | 2 |
You can set an alias for the frequently used command. For example:
$ alias la='ls -la'
Now, "la
" works as a short hand for "ls -la
" which lists all files in the long listing format.
You can list any existing aliases:
$ alias
You can identity exact path or identity of the command using type
builtins command. For example:
$ type ls ls is hashed (/bin/ls) $ type la la is aliased to ls -la $ type echo echo is a shell builtin $ type file file is /usr/bin/file
Here ls
was recently searched while "file
" was not, thus "ls
" is "hashed", i.e., the shell has an internal record for the quick access to the location of the "ls
" command.
![]() |
Tip |
---|---|
In Unix-like work environment, text processing is done by piping text through chains of standard text processing tools. This was another crucial Unix innovation.
There are few standard text processing tools which are used very often on the Unix-like system.
No regular expression is used:
cat
(1) concatenates files and outputs the whole content.
tac
(1) concatenates files and outputs in reverse.
cut
(1) selects parts of lines and outputs.
head
(1) outputs the first part of files.
tail
(1) outputs the last part of files.
sort
(1) sorts lines of text files.
uniq
(1) removes duplicate lines from a sorted file.
tr
(1) translates or deletes characters.
diff
(1) compares files line by line.
Basic regular expression (BRE) is used:
grep
(1) matches text with patterns.
ed
(1) is a primitive line editor.
sed
(1) is a stream editor.
vim
(1) is a screen editor.
emacs
(1) is a screen editor. (somewhat extended BRE)
Extended regular expression (ERE) is used:
egrep
(1) matches text with patterns.
awk
(1) does simple text processing.
tcl
(3tcl) can do every conceivable text processing: re_syntax
(3). Often used with tk
(3tk).
perl
(1) can do every conceivable text processing. perlre
(1).
pcregrep
(1) from the pcregrep
package matches text with Perl Compatible Regular Expressions (PCRE) pattern.
python
(1) with the re
module can do every conceivable text processing. See "/usr/share/doc/python/html/index.html
".
If you are not sure what exactly these commands do, please use "man command
" to figure it out by yourself.
![]() |
Note |
---|---|
Sort order and range expression are locale dependent. If you wish to obtain traditional behavior for a command, use C locale instead of UTF-8 ones by prepnding command with " |
![]() |
Note |
---|---|
Perl regular expressions ( |
Regular expressions are used in many text processing tools. They are analogous to the shell globs, but they are more complicated and powerful.
The regular expression describes the matching pattern and is made up of text characters and metacharacters.
The metacharacter is just a character with a special meaning. There are 2 major styles, BRE and ERE, depending on the text tools as described above.
Table 1.25. Metacharacters for BRE and ERE.
BRE | ERE | The meaning of the regular expression |
---|---|---|
\ . [ ] ^ $ *
|
\ . [ ] ^ $ *
|
Common metacharacters |
\+ \? \( \) \{ \} \|
|
BRE only "\ " escaped metacharacters
|
|
+ ? ( ) { } |
|
ERE only non-"\ " escaped metacharacters
|
|
c
|
c
|
This matches the non-metacharacter "c ".
|
\c
|
\c
|
This sequence matches the literal character "c " even if "c " is metacharacter by itself.
|
.
|
.
|
This matches any character including newline. |
^
|
^
|
This matches the beginning of a string. |
$
|
$
|
This matches the end of a string. |
\<
|
\<
|
This matches the beginning of a word. |
\>
|
\>
|
This matches the end of a word. |
\[abc…\]
|
[abc…]
|
This character list matches any characters "abc… ".
|
\[^abc…\]
|
[^abc…]
|
This negated character list matches any characters except "abc… ".
|
r*
|
r*
|
This matches zero or more regular expressions identified by "r ".
|
r\+
|
r+
|
This matches one or more regular expressions identified by "r ".
|
r\?
|
r?
|
This matches zero or one regular expressions identified by "r ".
|
r1\|r2
|
r1|r2
|
This matches one of the regular expressions identified by "r1 " or "r2 ".
|
\(r1\|r2\)
|
(r1|r2)
|
This matches one of the regular expressions identified by "r1 " or "r2 " and treats it as a bracketed regular expression.
|
The regular expression of emacs
is basically BRE but has been extended to treat "+
"and "?
" as the metacharacters as in ERE. Thus, there are no needs to escape them with "\
" in the regular expression of emacs
.
For example, grep
(1) can be used to perform the text search using the regular expression:
$ egrep 'GNU.*LICENSE|Yoyodyne' /usr/share/common-licenses/GPL GNU GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE Yoyodyne, Inc., hereby disclaims all copyright interest in the program
![]() |
Tip |
---|---|
For the replacement expression, following characters have special meanings:
Table 1.26. The replacement expression.
character | meaning |
---|---|
&
|
This represents what the regular expression matched. (use \& in emacs )
|
\n
|
This represents what the n-th bracketed regular expression matched. ("n" being number) |
For Perl replacement string, "$n
" is used instead of "\n
" and "&
" has no special meaning.
For example:
$ echo zzz1abc2efg3hij4 | \ sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/=&=/' zzz=1abc2efg3hij4= $ echo zzz1abc2efg3hij4 | \ sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/\2===\1/' zzzefg3hij4===1abc $ echo zzz1abc2efg3hij4 | \ perl -pe 's/(1[a-z]*)[0-9]*(.*)$/$2===$1/' zzzefg3hij4===1abc $ echo zzz1abc2efg3hij4 | \ perl -pe 's/(1[a-z]*)[0-9]*(.*)$/=&=/' zzz=&=
Here please pay extra attention to the style of the bracketed regular expression and how the matched strings are used in the text replacement process on different tools.
These regular expressions can be used for cursor movements and text replacement actions in some editors too.
The back slash "\
" at the end of line in the shell commandline escapes newline as a white space character and continues shell command line input to the next line.
Please read all the related manual pages to learn these commands.
The ed
(1) command can replace all instances of "FROM_REGEX
" with "TO_TEXT
" in "file
" by:
$ ed file <<EOF ,s/FROM_REGEX/TO_TEXT/g w q EOF
The vim
(1) command can replace all instances of "FROM_REGEX
" with "TO_TEXT
" in "file
" by using ex
(1) commands:
$ vim '+%s/FROM_REGEX/TO_TEXT/gc' '+w' '+q' file
![]() |
Tip |
---|---|
The " |
Multiple files ("file1
", "file2
", and "file3
") can be processed with regular expressions similarly with vim
(1) or perl
(1):
$ vim '+argdo %s/FROM_REGEX/TO_TEXT/ge|update' '+q' file1 file2 file3
![]() |
Tip |
---|---|
The " |
$ perl -i -p -e 's/FROM_REGEX/TO_TEXT/g;' file1 file2 file3
In the perl(1) example, "-i
" is for in-place editing, "-p
" is for implicit loop over files.
![]() |
Tip |
---|---|
Use of argument " |
![]() |
Note |
---|---|
While |
Let's consider a text file called "DPL
" in which some pre-2004 Debian project leader's names and their initiation days are listed in a
space-separated format.
Ian Murdock August 1993 Bruce Perens April 1996 Ian Jackson January 1998 Wichert Akkerman January 1999 Ben Collins April 2001 Bdale Garbee April 2002 Martin Michlmayr March 2003
![]() |
Tip |
---|---|
See "A Brief History of Debian" for the latest Debian leadership history. |
Awk is frequently used to extract data from these types of files:
$ awk '{ print $3 }' <DPL # month started August April January January April April March $ awk '($1=="Ian") { print }' <DPL # DPL called Ian Ian Murdock August 1993 Ian Jackson January 1998 $ awk '($2=="Perens") { print $3,$4 }' <DPL # When Perens started April 1996
Shells such as Bash can be also used to parse this kind of file:
$ while read first last month year; do echo $month done <DPL
Here, the read
builtin command uses characters in "$IFS
" (internal field
separators) to split lines into words.
If you change "$IFS
" to ":
", you can parse "/etc/passwd
" with shell nicely:
$ oldIFS="$IFS" # save old value $ IFS=':' $ while read user password uid gid rest_of_line; do if [ "$user" = "bozo" ]; then echo "$user's ID is $uid" fi done < /etc/passwd bozo's ID is 1000 $ IFS="$oldIFS" # restore old value
(If Awk is used to do the equivalent, use "FS=':'
" to set the field separator.)
IFS is also used by the shell to split results of parameter expansion, command substitution, and arithmetic expansion. These do not occur within double or single quoted words. The default value of IFS is <space>, <tab>, and <newline> combined.
Be careful about using this shell IFS tricks. Strange things may happen, when shell interprets some parts of the script as its input.
$ IFS=":," # use ":" and "," as IFS $ echo IFS=$IFS, IFS="$IFS" # echo is a Bash builtin IFS= , IFS=:, $ date -R # just a command output Sat, 23 Aug 2003 08:30:15 +0200 $ echo $(date -R) # sub shell --> input to main shell Sat 23 Aug 2003 08 30 36 +0200 $ unset IFS # reset IFS to the default $ echo $(date -R) Sat, 23 Aug 2003 08:30:50 +0200
The following scripts will do nice things as a part of a pipe.
Table 1.27. List of script snippets for piping commands.
script snippet (type in one line) | effect of command |
---|---|
find /usr -print
|
find all files under "/usr ".
|
seq 1 100
|
print 1 to 100. |
| xargs -n 1 <command>
|
run command repeatedly with each item from pipe as its argument. |
| xargs -n 1 echo
|
split white-space-separated items from pipe into lines. |
| xargs echo
|
merge all lines from pipe into a line. |
| grep -e <regex_pattern>
|
extract lines from pipe containing <regex_pattern>. |
| grep -v -e <regex_pattern>
|
extract lines from pipe not containing <regex_pattern>. |
| cut -d: -f3 -
|
extract third field from pipe separated by ": " (passwd file etc.).
|
| awk '{ print $3 }'
|
extract third field from pipe separated by whitespaces. |
| awk -F'\t' '{ print $3 }'
|
extract third field from pipe separated by tab. |
| col -bx
|
remove backspace and expand tabs to spaces. |
| expand -
|
expand tabs. |
| sort| uniq
|
sort and remove duplicates. |
| tr 'A-Z' 'a-z'
|
convert uppercase to lowercase. |
| tr -d '\n'
|
concatenate lines into one line. |
| tr -d '\r'
|
remove CR. |
| sed 's/^/# /'
|
add "# " to the start of each line.
|
| sed 's/\.ext//g'
|
remove ".ext ".
|
| sed -n -e 2p
|
print the second line. |
| head -n 2 -
|
print the first 2 lines. |
| tail -n 2 -
|
print the last 2 lines. |
One-line shell script can loop over many files using find
(1) and xargs
(1) to perform quite complicated tasks. See Section 10.1.5, “Idioms for the selection of files” and Section 9.5.9, “Repeating a command looping over files”.
When using the shell interactive mode becomes too complicated, please consider to write a shell script (see Section 12.1, “The shell script”).
![]() |
Note |
---|---|
This chapter is written assuming the latest stable release is codename: |
Debian is a volunteer organization which builds consistent distributions of pre-compiled binary packages of free software and distributes them from its archive.
The Debian archive is offered by many remote mirror sites for access through HTTP and FTP methods. It is also available as CD-ROM/DVD.
The Debian package management system, when used properly, offers the user to install consistent sets of binary packages to the system from the archive. Currently, there are 25441 packages available for the amd64 architecture.
The Debian package management system has a rich history and many choices for the front end user program and back end archive access method to be used. Currently, we recommend aptitude
(8) as the main front end program for the Debian package management activity.
Table 2.1. List of Debian package management tools.
package | popcon | size | description |
---|---|---|---|
aptitude
|
V:26, I:97 | 9832 |
terminal-based package manager (current standard, front-end for apt )
|
apt
|
V:90, I:99 | 5128 |
Advanced Packaging Tool (APT), front-end for dpkg providing "http ", "ftp ", and "file " archive access methods (apt-get /apt-cache commands included)
|
tasksel
|
V:7, I:93 | 900 | tool for selecting tasks for installation on Debian system (front-end for APT) |
unattended-upgrades
|
V:4, I:26 | 212 | enhancement package for APT to enable automatic installation of security upgrades |
dselect
|
V:4, I:57 | 2060 | terminal-based package manager (previous standard, front-end for APT and other old access methods) |
dpkg
|
V:91, I:99 | 6864 | package management system for Debian |
dpkg-ftp
|
V:0.08, I:0.5 | 136 |
older ftp method for dselect
|
synaptic
|
V:20, I:48 | 6104 | graphical package manager (GNOME front-end for APT) |
gnome-apt
|
V:0.3, I:1.9 | NOT_FOUND | graphical package manager (GNOME front-end for APT) |
kpackage
|
V:5, I:14 | 1064 | graphical package manager (KDE front-end for APT) |
apt-utils
|
V:52, I:99 | 460 |
APT utility programs: apt-extracttemplates (1), apt-ftparchive (1), and apt-sortpkgs (1)
|
apt-listchanges
|
V:3, I:6 | 264 | package change history notification tool |
apt-listbugs
|
V:1.4, I:2 | 436 | lists critical bugs before each APT installation |
apt-file
|
V:2, I:9 | 172 | APT package searching utility — command-line interface |
apt-rdepends
|
V:0.16, I:0.9 | 92 | recursively lists package dependencies |
![]() |
Note |
---|---|
The annoying bug #411123 for the mixed use of |
Here are some key points for package configuration on the Debian system:
debconf
(7) to help initial installation process of the package.
![]() |
Warning |
---|---|
Do not install packages from random mixture of suites. It probably will break the package consistency which requires deep system management knowledge, such as compiler ABI, library version, interpreter features, etc. |
The newbie Debian system administrator should stay with the stable
release of Debian while applying only security updates. I mean that some of the following valid actions are better avoided, as a precaution, until you understand the Debian system very well:
testing
or unstable
in "/etc/apt/sources.list
",
/etc/apt/sources.list
",
/etc/apt/preferences
",
dpkg -i <random_package>
",
dpkg --force-all -i <random_package>
",
/var/lib/dpkg/
", or
/usr/local
" or "/opt
".)
The non-compatible effects caused by above actions to the Debian package management system may leave your system unusable.
The serious Debian system administrator who runs mission critical servers, should use extra precautions:
Despite my warnings above, I know many readers of this document wish to run the testing
or unstable
suites of Debian as their main system for self-administered Desktop environments. This is because they work very well, are updated frequently, and offer the latest features.
![]() |
Caution |
---|---|
For your production server, the |
It takes no more than simply setting the distribution string in the "/etc/apt/sources.list
" to the suite name: "testing
" or "unstable
"; or the codename: "squeeze
" or "sid
". This will let you live the life of eternal upgrades.
The use of testing
or unstable
is a lot of fun but comes with some risks. Even though the unstable
suite of Debian system looks very stable for most of the times, there have been some package problems on the testing
and unstable
suite of Debian system and a few of them were not so trivial to resolve. It may be quite painful for you. Sometimes, you may have a broken package or missing functionality for a few weeks.
Here are some ideas to ensure quick and easy recovery from bugs in Debian packages:
stable
suite of Debian system to another partition.
apt-listbugs
to check the Debian Bug Tracking System (BTS) information before the upgrade.
(If you can not do any one of these precautionary actions, you are probably not ready for the testing
and unstable
suites.)
Enlightenment with the following will save a person from the eternal karmic struggle of upgrade hell and let him reach Debian nirvana.
Let's look into the Debian archive from a system user's perspective.
![]() |
Tip |
---|---|
Official policy of the Debian archive is defined at Debian Policy Manual, Chapter 2 - The Debian Archive. |
For the typical HTTP access, the archive is specified in the "/etc/apt/sources.list
" file as, e.g. for the current stable
= lenny
system:
deb http://ftp.XX.debian.org/debian/ lenny main contrib non-free deb-src http://ftp.XX.debian.org/debian/ lenny main contrib non-free deb http://security.debian.org/ lenny/updates main contrib deb-src http://security.debian.org/ lenny/updates main contrib
Please note "ftp.XX.debian.org
" must be replaced with appropriate mirror site URL for your location, for USA "ftp.us.debian.org
", which can be found in the list of Debian worldwide mirror sites. The status of these servers can be checked at Debian Mirror Checker site.
Here, I tend to use codename "lenny
" instead of suite name "stable
" to avoid surprises when the next stable
is released.
The meaning of "/etc/apt/sources.list
" is described in sources.list
(5) and key points are:
deb
" line defines for the binary packages.
deb-src
" line defines for the source packages.
The "deb-src
" lines can safely be omitted (or commented out by placing "#" at the start of the line) if it is just for aptitude
which does not access source related meta data. It will speed up the updates of the archive meta data. The URL can be "http://
", "ftp://
", "file://
", ….
![]() |
Tip |
---|---|
If " |
Here is the list of URL of the Debian archive sites and suite name or codename used in the configuration file:
Table 2.2. List of Debian archive sites.
archive URL | suite name (codename) | purpose |
---|---|---|
http://ftp.XX.debian.org/debian/ |
stable (lenny )
|
stable (lenny) release |
http://ftp.XX.debian.org/debian/ |
testing (squeeze )
|
testing (squeeze) release |
http://ftp.XX.debian.org/debian/ |
unstable (sid )
|
unstable (sid) release |
http://ftp.XX.debian.org/debian/ |
experimental
|
experimental pre-release (optional, only for developer) |
http://ftp.XX.debian.org/debian/ |
stable-proposed-updates
|
Updates for the next stable point release (optional) |
http://security.debian.org/ |
stable/updates
|
Security updates for stable release (important) |
http://security.debian.org/ |
testing/updates
|
Security updates for testing release (important) |
http://volatile.debian.org/debian-volatile/ |
volatile
|
Compatible updates for spam filter, IM clients, etc. |
http://volatile.debian.org/debian-volatile/ |
volatile-sloppy
|
Non-compatible updates for spam filter, IM clients, etc. |
http://backports.org/debian/ |
lenny-backports
|
Newer backported packages for lenny. (non-official, optional) |
![]() |
Caution |
---|---|
Only pure |
![]() |
Caution |
---|---|
You should basically list only one of |
![]() |
Note |
---|---|
For the Debian system with the |
Each Debian archive consists of 3 components. Components are alternatively called categories in "Debian Policy" or areas in "Debian Social Contract". The component is grouped by the compliance to "The Debian Free Software Guidelines" (DFSG):
Table 2.3. List of Debian archive components.
component | number of packages | criteria |
---|---|---|
main
|
24828 |
The package is fully compliant to DSFG and does not depend the non-free package.
|
contrib
|
207 |
The package is compliant to the DSFG but depends on the non-free package.
|
non-free
|
406 | The package is not compliant to the DSFG but distributable and useful. |
Here the number of packages in the above is for the amd64 architecture. Strictly speaking, only the main
component archive shall be considered as the Debian system.
The Debian archive organization can be studied best by pointing your browser to the each archive URL appended with dists
or pool
.
The distribution is referred by two ways, the suite or codename. The word distribution is alternatively used as the synonym to the suite in many documentations. The relationship between the suite and the codename can be summarized as:
Table 2.4. The relationship between suite and codename.
Timing |
suite = stable
|
suite = testing
|
suite = unstable
|
---|---|---|---|
after the lenny release
|
codename = lenny
|
codename = squeeze
|
codename = sid
|
after the squeeze release
|
codename = squeeze
|
codename = squeeze+1
|
codename = sid
|
The history of codenames are described in Debian FAQ: 6.3.1 Which other codenames have been used in the past?
In the stricter Debian archive terminology, the word "section" is specifically used for the categorization of packages by the application area. (Although, the word "main section" may sometimes be used to describe the Debian archive section which provides the main component.)
Every time a new upload is done by the Debian developer (DD) to the unstable
archive (via incoming processing), DD is required to ensure uploaded packages to be compatible with the latest set of packages in the latest unstable
archive.
If DD breaks this compatibility intentionally for important library upgrade etc, there is usually announcement to the debian-devel mailing list etc.
Before a set of packages are moved by the Debian archive maintenance script from the unstable
archive to the testing
archive, the archive maintenance script not only checks the maturity (about 10 days old) and the status of the RC bug reports for the packages but also tries to ensure them to be compatible with the latest set of packages in the testing
archive. This process makes the testing
archive very current and usable.
Through the gradual archive freeze process led by the release team, the testing
archive will be matured to make it completely consistent and bug free with some manual interventions. Then the new stable
release is created by assigning the codename for the old testing
archive to the new stable
archive and creating the new codename for the new testing
archive. The initial contents of the new testing
archive is exactly the same as that of the newly released stable
archive.
Both the unstable
and the testing
archives may suffer temporary glitches due to:
unstable
),
unstable
),
testing
and unstable
),
testing
), etc.
So if you ever decide to use these archives, you should be able to fix or work around these kinds of glitches.
![]() |
Caution |
---|---|
For about few months after a new |
![]() |
Tip |
---|---|
When tracking the |
See Debian Policy Manual for definition of:
The Debian system offers a consistent set of binary packages through its versioned binary dependency declaration mechanism in the control file fields. Here is a bit over simplified definition for them.
Table 2.5. List of package dependencies.
dependency | meaning |
---|---|
Depends | This declares an absolute dependency and all of the packages listed in this field must be installed at the same time or in advance. |
Pre-Depends | This is like Depends, except that it requires completed installation of the listed packages in advance. |
Recommends | This declares a strong, but not absolute, dependency. Most users would not want the package unless all of the packages listed in this field are installed. |
Suggests | This declares a weak dependency. Many users of this package may benefit from installing packages listed in this field but can have reasonable functions without them. |
Enhances | This declares a week dependency like Suggests but works in the opposite direction. |
Conflicts | This declares an absolute incompatibility. All of the packages listed in this field must be removed to install this package. |
Replaces | This is declared when files installed by this package replace files in the listed packages. |
Provides | This is declared when this package provide all of the files and functionality in the listed packages. |
![]() |
Note |
---|---|
Please note that defining, Provides, Conflicts and Replaces simultaneously to an virtual package is the sane configuration. This ensures that only one real package providing this virtual package can be installed at any one time. |
The official definition including source dependency can be found in the Policy Manual: Chapter 7 - Declaring relationships between packages.
Here is a summary of the simplified event flow of the package management by APT.
update ("aptitude update
" or "apt-get update
"):
upgrade ("aptitude safe-upgrade
" and "aptitude full-upgrade
", or "apt-get upgrade
" and "apt-get dist-upgrade
"):
install ("aptitude install …
" or "apt-get install …
"):
remove ("aptitude remove …
" or "apt-get remove …
"):
purge ("aptitude purge …
" or "apt-get purge …
"):
Here, I intentionally skipped technical details for the sake of big picture.
You should read the fine official documentation. The first document to read is the Debian specific "/usr/share/doc/<package_name>/README.Debian
". Other documentation in "/usr/share/doc/<package_name>/
" should be consulted too. If you set shell as Section 1.4.2, “Customizing bash”, type:
$ cd <package_name> $ pager README.Debian $ mc
You may need to install the corresponding documentation package named with "-doc
" suffix for detailed information.
If you are experiencing problems with a specific package, make sure to check out these sites first:
Table 2.6. List of key web site to resolving problems with a specific package.
site | command |
---|---|
Home page of the Debian bug tracking system (BTS). |
sensible-browser "http://bugs.debian.org/"
|
The bug report of a known package name. |
sensible-browser "http://bugs.debian.org/<package_name>"
|
The bug report of known bug number. |
sensible-browser "http://bugs.debian.org/<bug_number>"
|
Search Google with search words including "site:debian.org
", "site:wiki.debian.org
", "site:lists.debian.org
", etc..
When you file a bug report, please use reportbug
(1) command.
Aptitude is the current preferred package management tool for the Debian system. It can be used as the commandline alternative to apt-get
/ apt-cache
and also as the full screen interactive package management tool.
For the package management operation which involves package installation or updates package metadata, you need to have root privilege.
Here are package management operations with commandline using aptitude
(8) and apt-get
(8) /apt-cache
(8).
Table 2.7. Package management operations with commandline using aptitude and apt-get / apt-cache.
aptitude syntax
|
apt-get /apt-cache syntax
|
description |
---|---|---|
aptitude update
|
apt-get update
|
Update package archive metadata. |
aptitude install foo
|
apt-get install foo
|
Install candidate version of "foo " package with its dependencies.
|
aptitude safe-upgrade
|
apt-get upgrade
|
Install candidate version of installed packages without removing any other packages. |
aptitude full-upgrade
|
apt-get dist-upgrade <package>
|
Install candidate version of installed packages while removing other packages if needed. |
aptitude remove foo
|
apt-get remove foo
|
Remove "foo " package while leaving its configuration files.
|
N/A |
apt-get autoremove
|
Remove auto-installed packages which is no longer required. |
aptitude purge foo
|
apt-get purge foo
|
Purge "foo " package with its configuration files.
|
aptitude clean
|
apt-get clean
|
Clear out the local repository of retrieved package files completely. |
aptitude autoclean
|
apt-get autoclean
|
Clear out the local repository of retrieved package files for outdated packages. |
aptitude show foo
|
apt-cache show <package>
|
Display detailed information about "foo " package.
|
aptitude search <regex>
|
apt-cache search <regex>
|
Search packages which match <regex>. |
aptitude why <regex>
|
N/A | Explain the reason why <regex> matching packages should be installed. |
aptitude why-not <regex>
|
N/A | Explain the reason why <regex> matching packages can not be installed. |
Although it is now safe to mix different package tools on the Debian system, it is best to continue using aptitude
as much as possible.
The difference between "safe-upgrade
"/"upgrade
" and "full-upgrade
"/"dist-upgrade
" only appears when new versions of packages stand in different dependency relationships from old versions of those packages. The "aptitude safe-upgrade
" command will never install new packages nor remove installed packages.
The "aptitude why <regex>
" can list more information by "aptitude -v why <regex>
". Similar information can be obtained by "apt-cache rdepends <package>
".
When aptitude
command is started in the commandline mode and faces some issues such as package conflicts, you can switch to the full screen interactive mode by pressing "e
"-key later at the prompt.
You may provide command options right after "aptitude
".
Table 2.8. Notable command options for aptitude
(8).
command option | description |
---|---|
-s
|
simulate the result of the command. |
-d
|
download only but no install/upgrade. |
-D
|
show brief explanations before the automatic installations and removals. |
See aptitude
(8) and "aptitude user's manual" at "/usr/share/doc/aptitude/README
" for more.
![]() |
Tip |
---|---|
The |
For the interactive package management, you start aptitude
in interactive mode from the console shell prompt as:
$ sudo aptitude -u Password:
This will update the local copy of the archive information and display the package list in the full screen with menu. Aptitude places its configuration at "~/.aptitude/config
".
![]() |
Tip |
---|---|
If you want to use root's configuration instead of user's one, use " |
![]() |
Tip |
---|---|
|
Notable key strokes to browse status of packages and to set "planned action" on them in this full screen mode are:
Table 2.9. List of key bindings for aptitude.
key | key binding |
---|---|
F10 or Ctrl-t
|
Menu |
?
|
Display help for keystroke (more complete listing) |
F10 → Help → User's Manual
|
Display User's Manual |
u
|
Update package archive information |
+
|
Mark the package for the upgrade or the install |
-
|
Mark the package for the remove (keep configuration files) |
_
|
Mark the package for the purge (remove configuration files) |
=
|
Place the package on hold |
U
|
Mark all upgradable packages (function as full-upgrade) |
g
|
Start downloading and installing selected packages |
q
|
Quit current screen and save changes |
x
|
Quit current screen and discard changes |
Enter
|
View information about a package |
C
|
View a package's changelog |
l
|
Change the limit for the displayed packages |
/
|
Search for the first match |
\
|
Repeat the last search |
The file name specification of the command line and the menu prompt after pressing "l
" and "//
" take the aptitude regex as described below. Aptitude regex can explicitly match a package name using a string started by "~n
and followed by the package name.
![]() |
Tip |
---|---|
You need to press " |
In the interactive full screen mode of aptitude
(8), packages in the package list are displayed like this by default:
idA libsmbclient -2220kB 3.0.25a-1 3.0.25a-2
Here, this line means from the left as:
![]() |
Tip |
---|---|
The full list of flags are given at the bottom of Help screen shown by pressing " |
The candidate version is chosen according to the current local preferences (see apt_preferences
(5) and Section 2.7.3, “Tweaking candidate version”).
Several types of package views are available under the menu "Views
":
Table 2.10. List of views for aptitude.
view | categorization | status |
---|---|---|
Package View
|
See Table 2.11, “The categorization of standard package views.”. (default) | Good |
Audit Recommendations
|
Packages which are recommended by some installed packages but not yet installed are listed. | Good |
Flat Package List
|
Packages are listed without categorization (for use with regex). | Good |
Debtags Browser
|
Packages are categorized according to their debtags entries. | Very usable |
Categorical Browser
|
Packages are categorized according to their category. | Deprecated (Use debtags!) |
![]() |
Note |
---|---|
Please help us improving tagging packages with debtags! |
The standard "Package View
" categorizes packages somewhat like dselect
with few extra features.
Table 2.11. The categorization of standard package views.
category | organization |
---|---|
Upgradable Packages
|
Organized as section → component → package |
New Packages
|
, , |
Installed Packages
|
, , |
Not Installed Packages
|
, , |
Obsolete and Locally Created Packages
|
, , |
Virtual Packages
|
You can pick a particular package from a set of packages with the same function. |
Tasks
|
You can cherry pick particular packages from a set of packages of a task. |
Aptitude offers several options for you to search packages using its regex formula:
aptitude search '<aptitude_regex>'
" to list installation status, package name and short description of matching packages.
aptitude show '<package_name>'
" to list detailed description of the package.
l
" in the full screen mode.
/
" in the full screen mode. "n
" for find-next, "\
" for backward search.
Here, the string for <package_name> is treated as the exact string match to the package name unless it is started explicitly with "~
" to be the regex formula.
The aptitude regex formula is mutt-like extended ERE (see Section 1.6.2, “Regular expressions”) and the meanings of the aptitude
specific special match rule extensions are as below:
Table 2.12. List of the aptitude regex formula.
meaning of the extended match rule | regex formula |
---|---|
match on package name |
~n<regex_name>
|
match on description |
~d<regex_description>
|
match on task name |
~t<regex_task>
|
match on debtag |
~G<regex_debtag>
|
match on maintainer |
~m<regex_maintainer>
|
match on package section |
~s<regex_section>
|
match on package version |
~V<regex_version>
|
match archive |
~A{sarge,etch,sid }
|
match origin |
~O{debian,… }
|
match priority |
~p{extra,important,optional,required,standard }
|
match essential packages |
~E
|
match virtual packages |
~v
|
match new packages |
~N
|
match with pending action |
~a{install,upgrade,downgrade,remove,purge,hold,keep }
|
match installed packages |
~i
|
match installed packages with A-mark (auto installed package) |
~M
|
match installed packages without A-mark (administrator selected package) |
~i!~M
|
match installed and upgradable packages |
~U
|
match removed but not purged packages |
~c
|
match removed, purged or can-be-removed packages |
~g
|
match packages with broken relation |
~b
|
match packages with broken depends/predepends/conflict |
~B<type>
|
match packages whose control files define relation <type> to the <term> package |
~D[<type>:]<term>
|
match packages whose control files define broken relation <type> to the <term> package |
~DB[<type>:]<term>
|
match packages to which the <term> package defines relation <type> |
~R[<type>:]<term>
|
match packages to which the <term> package defines broken relation <type> |
~RB[<type>:]<term>
|
match packages to which some other installed packages depend on |
~R~i
|
match packages to which no other installed packages depend on |
!~R~i
|
match packages to which some other installed packages depend or recommend on |
~R~i|~Rrecommends:~i
|
match <term> package with filtered version |
~S filter <term>
|
match all packages (true) |
~T
|
match no packages (false) |
~F
|
Here,
^
", ".*
", "$
" etc. as in egrep
(1), awk
(1) and perl
(1).
![]() |
Tip |
---|---|
When <regex_pattern> is a null string, place " |
Short cuts:
~P<term>
" == "~Dprovides:<term>
"
~C<term>
" == "~Dconflicts:<term>
"
…~W term
" == "(…|term)
"
Users familiar with mutt
will pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in the "User's Manual" "/usr/share/doc/aptitude/README
".
![]() |
Note |
---|---|
With the |
The selection of a package in aptitude
not only pulls in packages which are defined in its "Depends:
" list but also defined in the "Recommends:
" list if the menu "F10
→ Options → Dependency handling" is set accordingly. These auto installed packages are removed automatically if they are no longer needed under aptitude
.
![]() |
Note |
---|---|
Before the |
You can check package activity history in the log files.
Table 2.13. The log files for package activities.
file | content |
---|---|
/var/log/dpkg.log
|
Log of dpkg level activity for all package activities.
|
/var/log/apt/term.log
|
Log of generic APT activity. |
/var/log/aptitude
|
Log of aptitude command activity.
|
In reality, it is not so easy to get meaningful understanding quickly out from these logs. See Section 9.2.9, “Recording changes in configuration files” for easier way.
Aptitude has advantages over other APT based packaging systems (apt-get, apt-cache, synaptic, …):
aptitude
removes unused auto installed packages automatically using its own extra layer of package state file (/var/lib/aptitude/pkgstates
). (For new "lenny
", other APT does the same.)
aptitude
makes it easy to resolve package conflicts and to add recommended packages.
aptitude
makes it easy to keep track of obsolete software by listing under "Obsolete and Locally Created Packages".
aptitude
gives a log of its history in "/var/log/aptitude
".
aptitude
offers access to all versions of the package if available.
aptitude
includes a fairly powerful regex based system for searching particular packages and limiting the package display.
aptitude
in the full screen mode has su
functionality embedded and can be run from normal user until you really need administrative privileges.
For the old etch
release version, synaptic
also gives you the history log; apt-get
did not but you can rely on the log of dpkg
.
Anyway, aptitude
is nice for interactive console use.
Here are few examples of aptitude
(8) operations.
The following command lists packages with regex matching names.
$ aptitude search '~n(pam|nss).*ldap' p libnss-ldap - NSS module for using LDAP as a naming service p libpam-ldap - Pluggable Authentication Module allowing LDAP interfaces
This is quite handy for you to find the exact name of a package.
The regex "~dipv6
" in the "New Flat Package List" view with "l
" prompt, limits view to packages with the matching description and let you browse their information interactively.
You can purge all remaining configuration files of removed packages:
# aptitude search '~c'
# aptitude purge '~c'
You may want to do the similar in the interactive mode for fine grained control.
You provide the regex "~c
" in the "New Flat Package List" view with "l
" prompt. This limits the package view only to regex matched packages, i.e., "removed but not purged". All these regex matched packages can be shown by pressing "[
" at top level headings.
Then you press "_
" at top level headings such as "Installed Packages". Only regex matched packages under the heading are marked to be purged by this. You can exclude some packages to be purged by pressing "=
" interactively for each of them.
This technique is quite handy and works for many other command keys.
Here is how I tidy auto/manual install status for packages (after using non-aptitude package installer etc.):
aptitude
in interactive mode as root.
u
", "U
", "f
" and "g
" to update and upgrade package list and packages.
l
" to enter the package display limit as "~i(~R~i|~Rrecommends:~i)
" and type "M
" over "Installed Packages
" as auto installed.
l
" to enter the package display limit as "~prequired|~pimportant|~pstandard|~E
" and type "m
" over "Installed Packages
" as manual installed.
l
" to enter the package display limit as "~i!~M
" and remove unused package by typing "-
" over each of them after exposing them by typing "[
" over "Installed Packages
".
l
" to enter the package display limit as "~i
" and type "m
" over "Tasks
" as manual installed.
aptitude
.
apt-get -s autoremove|less
" as root to check what are not used.
aptitude
in interactive mode and mark needed packages as "m
".
apt-get -s autoremove|less
" as root to recheck REMOVED contain only expected packages.
apt-get autoremove|less
" as root to autoremove unused packages.
The "m
" action over "Tasks
" is an optional one to prevent mass package removal situation in future.
![]() |
Note |
---|---|
When moving to a new release etc, you should consider to perform a clean installation of new system even though Debian is upgradable as described below. This provides you a chance to remove garbages collected and exposes you to the best combination of latest packages. Of course, you should make a full backup of system to a safe place (see Section 10.1.6, “Backup and recovery”) before doing this. I recommend to make a dual boot configuration using different partition to have the smoothest transition. |
You can perform system wide upgrade to a newer release by changing contents of the "/etc/apt/sources.list
" file pointing to a new release and running the "aptitude update; aptitude full-upgrade
" command.
To upgrade from stable
to testing
or unstable
, you replace "lenny
" in the "/etc/apt/sources.list
" example of Section 2.1.4, “Debian archive basics” with "squeeze
" or "sid
".
In reality, you may face some complications due to some package transition issues, mostly due to package dependencies. The larger the difference of the upgrade, the more likely you face larger troubles. For the transition from the old stable
to the new stable
after its release, you can read its new Release Notes and follow the exact procedure described in it to minimize troubles.
When you decide to move from stable
to testing
before its formal release, there are no Release Notes to help you. The difference between stable
and testing
could have grown quite large after the previous stable
release and makes upgrade situation complicated.
You should make some precautionary moves while gathering latest information from mailing list and using common sense:
script
(1).
aptitude unmarkauto vim
", to prevent removal.
/etc/apt/preferences
" file. (disable apt-pinning)
oldstable
→ stable
→ testing
→ unstable
.
/etc/apt/sources.list
" file to point to new archive only and run "aptitude update
".
aptitude install perl
".
aptitude full-upgrade -s
" command to assess impact.
aptitude full-upgrade
" command.
![]() |
Caution |
---|---|
It is not wise to skip major Debian release when upgrading between |
![]() |
Caution |
---|---|
In previous "Release Notes", GCC, Linux Kernel, initrd-tools, Glibc, Perl, APT tool chain, etc. have required some special attention for system wide upgrade. |
For daily upgrade in unstable
, see Section 2.4.3, “Safeguard for package problems”.
Here are list of other package management operations for which aptitude
is too high-level or lacks required functionalities.
Table 2.14. List of advanced package management operations.
action | command |
---|---|
list status of an installed package for the bug report. |
COLUMNS=120 dpkg -l <package_name_pattern>
|
list contents of an installed package. |
dpkg -L <package_name>
|
list manpages for an installed package. |
dpkg -L <package_name> | egrep '/usr/share/man/man.*/.+'
|
list installed packages which have matching file name. |
dpkg -S <file_name_pattern>
|
list packages in archive which have matching file name. |
apt-file search <file_name_pattern>
|
list contents of matching packages in archive. |
apt-file list <package_name_pattern>
|
reconfigure the exact package . |
dpkg-reconfigure <package_name>
|
reconfigure the exact package with the most detailed question. |
dpkg-reconfigure -p=low <package_name>
|
reconfigure packages from the full screen menu. |
configure-debian
|
audit system for partially installed packages. |
dpkg --audit
|
configures all partially installed packages. |
dpkg --configure -a
|
show available version, priority, and archive information of a binary package. |
apt-cache policy <binary_package_name>
|
show available version, archive information of a package. |
apt-cache madison <package_name>
|
show source package information of a binary package. |
apt-cache showsrc <binary_package_name>
|
install required packages to build package. |
apt-get build-dep <package_name>
|
download a source. (from standard archive) |
apt-get source <package_name>
|
download a source packages. (from other archive) |
dget <URL for dsc file>
|
build a source tree from a set of source packages ("*.tar.gz " and "*.diff.gz ").
|
dpkg-source -x <package_name>_<version>-<debian_version>.dsc
|
build package(s) from a local source tree. |
debuild binary
|
build a kernel package from a kernel source tree. |
make-kpkg kernel_image
|
build a kernel package from a kernel source tree with initramfs enabled. |
make-kpkg --initrd kernel_image
|
install a local package to the system. |
dpkg -i <package_name><version>-<debian_version><arch>.deb
|
install local package(s) to the system. |
debi <package_name><version>-<debian_version><arch>.dsc
|
save dpkg level package selection state information.
|
dpkg --get-selection '*' >selection.txt
|
set dpkg level package selection state information.
|
dpkg --set-selection <selection.txt
|
![]() |
Caution |
---|---|
Lower level package tools such as " |
Please note:
aptitude
which uses regex (see Section 1.6.2, “Regular expressions”), other package management commands use pattern like shell glob (see Section 1.5.3, “Shell glob”).
apt-file
(1) provided by the apt-file
package must run "apt-file update
" in advance.
configure-debian
(8) provided by the configure-debian
package runs dpkg-reconfigure
(8) as its backend.
dpkg-reconfigure
(8) runs package scripts using debconf
(1) as its backend.
apt-get build-dep
", "apt-get source
" and "apt-cache showsrc
" commands require "deb-src
" entry in "/etc/apt/sources.list
".
dget
(1), debuild
(1), and debi
(1) require devscripts
package.
apt-get source
" in Section 2.7.10, “Port a package to the stable system”.
make-kpkg
command requires the kernel-package
package (see Section 9.7, “The kernel”).
![]() |
Tip |
---|---|
The source package format described here as a set of source packages (" |
The installation of debsums
enables verification of installed package files against MD5sum values in the "/var/lib/dpkg/info/*.md5sums
" file with debsums
(1). See Section 10.3.5, “The MD5 sum” for how MD5sum works.
![]() |
Note |
---|---|
Because MD5sum database may be tampered by the intruder, |
Many users prefer to follow the unstable release of the Debian system for its new features and packages. This makes the system more prone to be hit by the critical package bugs.
The installation of the apt-listbugs
package will provide safeguard to the critical bugs by checking Debian BTS automatically for critical bugs when upgrading with APT system.
The installation of the apt-listchanges
package will provide important news in "NEWS.Debian
" when upgrading with APT system.
Although visiting Debian site http://packages.debian.org/ facilitates easy ways to search on the package meta data these days, let's look into more traditional ways.
The grep-dctrl
(1), grep-status
(1), and grep-available
(1) commands can be used to search any file which has the general format of a Debian package control file.
The "dpkg -S <file_name_pattern>
" can be used search package names which contain files with the matching name installed by dpkg
. But this overlooks files created by the maintainer scripts.
If you need to make more elaborate search on the dpkg meta data, you need to run "grep -e regex_pattern *
" command in the "/var/lib/dpkg/info/
" directory. This will let you identify:
If you wish to look up package dependency recursively, you should use apt-rdepends
(8).
Let's learn how the Debian package management system works internally. This should help you to create your own solution to some package problems.
Meta data files for each distribution are stored under "dist/<codename>
" on each Debian mirror sites, e.g., "http://ftp.us.debian.org/debian/
". Its archive structure can be browsed by the web browser. There are 6 types of key meta data:
Table 2.15. The content of the Debian archive meta data.
file | location | content |
---|---|---|
Release
|
top of distribution | archive description and integrity information |
Release.gpg
|
top of distribution |
signature file for the "Release " file signed with the archive key
|
Contents-<architecture>
|
top of distribution | list of all files for all the packages in the pertinent archive |
Release
|
top of each distribution/component/architecture combination |
archive description used for the rule of apt_preferences (5)
|
Packages
|
top of each distribution/component/binary-architecture combination |
concatenated debian/control for binary packages
|
Sources
|
top of each distribution/component/source combination |
concatenated debian/control for source packages
|
In the recent archive, these meta data are stored as the compressed and differential files to reduce network traffic.
![]() |
Tip |
---|---|
The top level " |
Each suite of the Debian archive has a top level "Release
" file, e.g., "http://ftp.us.debian.org/debian/dists/unstable/Release
":
Origin: Debian Label: Debian Suite: unstable Codename: sid Date: Sat, 26 Jan 2008 20:13:58 UTC Architectures: alpha amd64 arm hppa hurd-i386 i386 ia64 m68k mips mipsel powerpc s390 sparc Components: main contrib non-free Description: Debian x.y Unstable - Not Released MD5Sum: e9f11bc50b12af7927d6583de0a3bd06 22788722 main/binary-alpha/Packages 43524d07f7fa21b10f472c426db66168 6561398 main/binary-alpha/Packages.gz ...
![]() |
Note |
---|---|
Here, you can find my rationale to use the "suite", "codeneme", and "components" in Section 2.1.4, “Debian archive basics”. The "distribution" is used when referring to both "suite" and "codeneme". |
The integrity of the top level "Release
" file is verified by cryptographic infrastructure called the secure apt.
Release.gpg
" is created from the authentic top level "Release
" file and the secret Debian archive key.
The public Debian archive key can be seeded into "/etc/apt/trusted.gpg
":
base-files
package, or
gpg
or apt-key
tool with the latest public archive key posted on the ftp-master.debian.org .
Release
" file cryptographically by this "Release.gpg
" file and the public Debian archive key in "/etc/apt/trusted.gpg
".
The integrity of all the "Packages
" and "Sources
" files are verified by using MD5sum values in its top level "Release
" file. The integrity of all package files are verified by using MD5sum values in the "Packages
" and "Sources
" files. See debsums
(1) and Section 2.4.2, “Verify installed package files”.
Since the cryptographic signature verification is very CPU intensive process than the MD5sum value calculation, use of MD5sum value for each package while using cryptographic signature for the top level "Release
" file provides the good security with the performance (see Section 10.3, “Data security infrastructure”).
![]() |
Tip |
---|---|
The archive level " |
There are archive level "Release
" files for all archive locations specified by "deb
" line in "/etc/apt/sources.list
", such as "http://ftp.us.debian.org/debian/dists/unstable/main/binary-amd64/Release
" or "http://ftp.us.debian.org/debian/dists/sid/main/binary-amd64/Release
":
Archive: unstable Component: main Origin: Debian Label: Debian Architecture: amd64
![]() |
Caution |
---|---|
For " |
For some archives, such as experimental
, volatile-sloppy
, and lenny-backports
, which contain packages which should not be installed automatically, there is an extra line, e.g., "http://ftp.us.debian.org/debian/dists/experimental/main/binary-amd64/Release
":
Archive: experimental Component: main Origin: Debian Label: Debian NotAutomatic: yes Architecture: amd64
Please note that for normal archives without "NotAutomatic: yes
", the default Pin-Priority value is 500, while for special archives with "NotAutomatic: yes
", the default Pin-Priority value is 1 (see apt_preferences
(5) and Section 2.7.3, “Tweaking candidate version”).
When APT tools, such as aptitude
, apt-get
, synaptic
, apt-file
, auto-apt
…, are used, we need to update the local copies of the meta data containing the Debian archive information. These local copies have file names corresponding to the specified distribution
, component
, and architecture
names in the "/etc/apt/sources.list
" (see Section 2.1.4, “Debian archive basics”) as:
/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_Release
",
/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_Release.gpg
",
/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_<component>_binary-<architecture>_Packages
",
/var/lib/apt/lists/ftp.us.debian.org_debian_dists_<distribution>_<component>_source_Sources
", and
/var/cache/apt/apt-file/ftp.us.debian.org_debian_dists_<distribution>_Contents-<architecture>.gz
" (for apt-file
).
First 4 types of files are shared by all the pertinent APT commands and updated from command line by "apt-get update
" and "aptitude update
". The "Packages
" meta data are updated if there is the "deb
" line in "/etc/apt/sources.list
". The "Sources
" meta data are updated if there is the "deb-src
" line in "/etc/apt/sources.list
".
The "Packages
" and "Sources
" meta data contain "Filename:
" stanza pointing to the file location of the binary and source packages. Currently, these packages are located under the "pool/
" directory tree for the improved transition over the releases.
Local copies of "Packages
" meta data can be interactively searched with the help of aptitude
. The specialized search command grep-dctrl
(1) can search local copies of "Packages
" and "Sources
" meta data.
Local copy of "Contents-<architecture>
" meta data can be updated by "apt-file update
" and its location is different from other 4 ones. See apt-file
(1). (The auto-apt
uses different location for local copy of "Contents-<architecture>.gz
" as default.)
In addition to the remotely fetched meta data, the APT tool after lenny
stores its locally generated installation state information in the "/var/lib/apt/extended_states
" which is used by all APT tools to track all auto installed packages.
In addition to the remotely fetched meta data, the aptitude
command stores its locally generated installation state information in the "/var/lib/aptitude/pkgstates
" which is used only by it.
All the remotely fetched packages via APT mechanism are stored in the "/var/cache/apt/packages
" until they are cleaned.
The Debian package files has particular name structures:
Table 2.16. The name structure of Debian packages.
package type | name structure |
---|---|
The binary package (a.k.a deb) |
<package-name>_<epoch>:<upstream-version>-<debian.version>-<architecture>.deb
|
The binary package for the debian-installer (a.k.a udeb) |
<package-name>_<epoch>:<upstream-version>-<debian.version>-<architecture>.udeb
|
The source package (upstream source) |
<package-name>_<epoch>:<upstream-version>-<debian.version>.tar.gz
|
The source package (Debian changes) |
<package-name>_<epoch>:<upstream-version>-<debian.version>.diff.gz
|
The source package (description) |
<package-name>_<epoch>:<upstream-version>-<debian.version>.dsc
|
where,
Table 2.17. The usable characters for each component in the Debian package names.
name component | usable characters (regex) | existance |
---|---|---|
<package-name>
|
[a-z,A-Z,0-9,.,
|
required |
<epoch>:
|
[0-9]+:
|
optional |
<upstream-version>
|
[a-z,A-Z,0-9,.,
|
required |
<debian.version>
|
[a-z,A-Z,0-9,.,
|
optional |
![]() |
Note |
---|---|
You can check package version order by |
![]() |
Note |
---|---|
The debian-installer (d-i) uses |
dpkg
(1) is the lowest level tool for the Debian package management. This is very powerful and needs to be used with care.
The fetched package is processed by dpkg
in the following order:
ar -x
" equivalent)
debconf
(1)
tar -x
" equivalent)
debconf
(1)
The debconf
system provides standardized user interaction with I18N and L10N (Chapter 8, I18N and L10N) supports.
While installing package called "<package_name>
", dpkg
creates several files and executes several scripts.
Table 2.18. The notable files for dpkg.
file | contents |
---|---|
/var/lib/dpkg/info/<package_name>.conffiles
|
list of configuration files. (user modifiable) |
/var/lib/dpkg/info/<package_name>.list
|
list of files and directories installed by the package. |
/var/lib/dpkg/info/<package_name>.md5sums
|
list of MD5 hash values for files installed by the package. |
/var/lib/dpkg/info/<package_name>.preinst
|
package script run before the package installation. |
/var/lib/dpkg/info/<package_name>.postinst
|
package script run after the package installation. |
/var/lib/dpkg/info/<package_name>.prerm
|
package script run before the package removal. |
/var/lib/dpkg/info/<package_name>.postrm
|
package script run after the package removal. |
/var/lib/dpkg/info/<package_name>.config
|
package script for debconf system.
|
/var/lib/dpkg/alternatives/<package_name>
|
the alternative information used by the update-alternatives command.
|
/var/lib/dpkg/available
|
the availability information for all the package. |
/var/lib/dpkg/diversions
|
the diversions information used by dpkg (1) and set by`dpkg-divert`(8)
|
/var/lib/dpkg/statoverride
|
the stat override information used by dpkg (1) and set by`dpkg-statoverride`(8)
|
/var/lib/dpkg/status
|
the status information for all the packages. |
/var/lib/dpkg/status-old
|
the first-generation backup of the "var/lib/dpkg/status " file.
|
/var/backups/dpkg.status*
|
the second-generation backup and older ones of the "var/lib/dpkg/status " file.
|
The "status
" file is also used by the tools such as dpkg
(1), "dselect update
" and "apt-get -u dselect-upgrade
".
The specialized search command grep-dctrl
(1) can search the local copies of "status
" and "available
" meta data.
![]() |
Tip |
---|---|
In the debian-installer environment, the |
The Debian system has mechanism to install somewhat overlapping programs peacefully using update-alternatives
(8). For example, you can make the vi
command select to run vim
while installing both vim
and nvi
packages:
$ ls -l $(type -p vi) lrwxrwxrwx 1 root root 20 2007-03-24 19:05 /usr/bin/vi -> /etc/alternatives/vi $ sudo update-alternatives --display vi ... $ sudo update-alternatives --config vi Selection Command ---------------------------------------------- 1 /usr/bin/vim *+ 2 /usr/bin/nvi Enter to keep the default[*], or type selection number: 1
The Debian alternatives system keeps its selection as symlinks in "/etc/alternatives/
". The selection process uses corresponding file in "/var/lib/dpkg/alternatives/
".
Stat overrides provided by the dpkg-statoverride
(8) command are a way to tell dpkg
(1) to use a different owner or mode for a file when a package is installed. If "--update
" is specified and file exists, it is immediately set to the new owner and mode.
![]() |
Caution |
---|---|
The direct alteration of owner or mode for a file owned by the package using |
![]() |
Note |
---|---|
I use the word file here, but in reality this can be any filesystem object that |
File diversions provided by the dpkg-divert
(8) command are a way of forcing dpkg
(1) not to install a file into its default location, but to a diverted location. The use of dpkg-divert
is meant for the package maintenance scripts. Its use by the system administrator is deprecated.
When running unstable
system, the administrator is expected to recover from broken package management situation.
![]() |
Caution |
---|---|
Some methods described here are high risk actions. You have been warned! |
If a desktop GUI program experienced instability after significant upstream version upgrade, you should suspect interferences with old local configuration files created by it. If it is stable under newly created user account, this hypothesis is confirmed. (This is a bug of packaging and usually avoided by the packager.)
To recover stability, you should move corresponding local configuration files and restart the GUI program. You may need to read old configuration file contents to recover configuration information later. (Do not erase them too quickly.)
Archive level package management systems, such as aptitude
(8) or apt-get
(1), will not even try to install packages with overlapped files using package dependencies (see Section 2.1.5, “Package dependencies”).
Errors by the package maintainer or deployment of inconsistently mixed source of archives (see Section 2.7.2, “Packages from mixed source of archives”) by the system administrator may create situation with incorrectly defined package dependencies. When you install a package with overlapped files using aptitude
(8) or apt-get
(1) under such situation, dpkg
(1) which unpacks package ensures to return error to the calling program without overwriting existing files.
![]() |
Caution |
---|---|
The use of third party packages introduces significant system risks via maintainer scripts which are run with root privilege and can do anything to your system. The |
You can work around such broken installation by removing the old offending package, <old-package>
, first:
$ sudo dpkg -P <old-package>
When a command in the package script returns error for some reason and the script exits with error, the package management system aborts their action and ends up with partially installed packages. When a package contains bugs in its removal scripts, the package may become impossible to remove and quite nasty.
For the package script problem of "<package_name>
", you should look for:
/var/lib/dpkg/info/<package_name>.preinst
",
/var/lib/dpkg/info/<package_name>.postinst
",
/var/lib/dpkg/info/<package_name>.prerm
", and
/var/lib/dpkg/info/<package_name>.postrm
".
You edit the offending part of the script from the root:
#
" or,
|| true
".
Then configures all partially installed packages by:
# dpkg --configure -a
Since dpkg
is very low level package tool, it can function under the very bad situation such as unbootable system without network connection. Let's assume foo
package was broken and needs to be replaced.
You may still find cached copies of older bug free version of foo
package in the package cache directory: "/var/cache/apt/archives/
". (If not, you can download it from archve of http://snapshot.debian.net/ or copy it from package cache of a functioning machine.)
If you can boot the system, you may install it by:
# dpkg -i /path/to/foo_<old_version>_<arch>.deb
![]() |
Tip |
---|---|
If system breakage is minor, you may alternatively downgrade the whole system as Section 2.7.7, “Emergency downgrading” using the higher level APT system. |
If your system is unbootable from harddisk, you should seek other ways to boot it. For example, you can:
/target
",
foo
package by:
# dpkg --root /target -i /path/to/foo_<old_version>_<arch>.deb
This second example works even if the dpkg
command on the harddisk is broken.
![]() |
Tip |
---|---|
Any GNU/Linux system started by another system on harddisk, live GNU/Linux CD, bootable USB-key drive, or netboot can be used similarly to rescue broken system. |
If attempting to install a package this way fails due to some dependency violations and you really need to do this as the last resort, you can override dependency using dpkg
's "--ignore-depends
", "--force-depends
" and other options. If you do this, you need to make serious effort to restore proper dependency later. See dpkg
(8) for details.
![]() |
Note |
---|---|
When your system is seriously broken, you should make a full backup of system to a safe place (see Section 10.1.6, “Backup and recovery”) and should perform a clean installation. This is less time consuming and produces better results in the end. |
If "/var/lib/dpkg/status
" becomes corrupt for any reason, the Debian system loses package selection data and suffers severely. Look for the old "/var/lib/dpkg/status
" file at "/var/lib/dpkg/status-old
" or "/var/backups/dpkg.status.*
".
Keeping "/var/backups/
" in a separate partition may be a good idea since this directory contains lots of important system data.
For serious breakage, I recommend to make fresh re-install after making backup of the system. Even if everything in "/var/
" is gone, you can still recover some information from directories in "/usr/share/doc/
" to guide your new installation.
/path/to/old/system/
"
# cd /path/to/old/system/usr/share/doc # ls -1 >~/ls1.txt # cd /usr/share/doc # ls -1 >>~/ls1.txt # cd # sort ls1.txt | uniq | less
Then you will be presented with package names to install. (There may be some non-package names such as "texmf
".)
You can seek packages which satisfy your needs with aptitude
from the package description or from the list under "Tasks".
When you encounter more than 2 similar packages and wonder which one to install without "trial and error" efforts, you should use some common sense. I consider following points are good indications of preferred packages.
python2.4
by python
)
Debian being a volunteer project with distributed development model, its archive contains many packages with different focus and quality. You must make your own decision what to do with them.
![]() |
Caution |
---|---|
Installing packages from mixed source of archives is not supported by the official Debian distribution except for officially supported particular combinations of archives such as |
Here is an example of operations to include specific newer upstream version packages found in unstable
while tracking testing
for single occasion:
/etc/apt/sources.list
" file temporarily to single "unstable
" entry
aptitude update
"
aptitude install <package-name>
"
/etc/apt/sources.list
" file for testing
aptitude update
"
You do not create the "/etc/apt/preferences
" file nor need to worry about apt-pinning with this manual approach. But this is very cumbersome.
![]() |
Caution |
---|---|
When using mixed source of archives, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. If package incompatibility exists, you may break system. You must be able to judge these technical requirements. The use of mixed source of random archives is completely optional operation and its use is not something I encourage you to use. |
General rules for installing packages from different archives are:
Non-binary packages ("Architecture: all
") are safer to install.
Completely statically linked binary packages are safe to install.
Binary packages (non "Architecture: all
") usually face many road blocks and unsafe to install.
![]() |
Note |
---|---|
Except to avoid broken package for a short term, installing binary packages from officially unsupported archives is generally bad idea. This is true even if you use apt-pinning (see Section 2.7.3, “Tweaking candidate version”). You should consider chroot or similar techniques (see Section 9.8.2, “Chroot system”) to run programs from different archives. |
![]() |
Warning |
---|---|
In |
Without the "/etc/apt/preferences
" file, APT system choses the latest available version as the candidate version using the version string. This is the normal state and most recommended usage of APT system. All officially supported combinations of archives do not require the "/etc/apt/preferences
" file since some archives which should not be used as the automatic source of upgrades are marked as NotAutomatic and dealt properly.
![]() |
Tip |
---|---|
The version string comparison rule can be verified with, e.g., " |
When you install packages from mixed source of archives (see Section 2.7.2, “Packages from mixed source of archives”) regularly, you can automate these complicated operations by creating the "/etc/apt/preferences
" file with proper entries and tweaking the package selection rule for candidate version as described in apt_preferences
(5). This is called apt-pinning.
![]() |
Warning |
---|---|
Use of apt-pinning by a novice user is sure call for major troubles. You must avoid using apt-pinning except when you absolutely need it. |
![]() |
Caution |
---|---|
When using apt-pinning, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. The apt-pinning is completely optional operation and its use is not something I encourage you to use. |
![]() |
Caution |
---|---|
Archive level Release files (see Section 2.5.3, “Archive level "Release" files”) are used for the rule of |
![]() |
Caution |
---|---|
When you use non-Debian archive as a part of apt-pinning, you should check what they are intended for and also check their credibility. For example, Ubuntu and Debian are not meant to be mixed. |
![]() |
Note |
---|---|
Even if you do not create the " |
Here is a simplified explanation of apt-pinning technique.
APT system choses highest Pin-Priority upgrading package from available package sources defined in the "/etc/apt/sources.list
" file as the candidate version package. If the Pin-Priority of the package is larger than 1000, this version restriction for upgrading is dropped to enable downgrading (see Section 2.7.7, “Emergency downgrading”).
Pin-Priority value of each package is defined by "Pin-Priority" entries in the "/etc/apt/preferences
" file or uses its default value.
Table 2.19. List of the default Pin-Priority value for each package source type.
default Pin-Priority | package source type |
---|---|
990 | the target release archive |
500 | the normal archive |
100 | the installed package |
1 | the NotAutomatic archive |
The target release archive can be set:
/etc/apt/apt.conf
", e.g., "APT::Default-Release "stable";
" line in it, or
-t
" option argument, e.g., "apt-get install -t testing some-package
".
The NotAutomatic archive can be set:
NotAutomatic: yes
" in archive.
The apt-pinning situation of <package> from multiple archive sources is displayed by "apt-cache policy <package>
":
Package pin:
" lists the package version of pin if association just with <package> is defined, e.g., "Package pin: 0.190
".
Package pin:
" exists if no association just with <package> is defined.
0.181 700
".
0
" is listed right side of all version strings if no association just with <package> is defined, e.g., "0.181 0
".
Package: *
" in the "/etc/apt/preferences
" file) are listed left side of all archive paths, e.g., "200 http://backports.org etch-backports/main Packages
".
Here is an example of apt-pinning technique to include specific newer upstream version packages found in unstable
regularly upgraded while tracking testing
. You list all required archives in the "/etc/apt/sources.list
" file as:
deb http://ftp.us.debian.org/debian/ testing main contrib non-free deb http://ftp.us.debian.org/debian/ unstable main contrib non-free deb http://security.debian.org/ testing/updates main contrib
and set the "/etc/apt/preferences
" file as:
Package: * Pin: release a=testing Pin-Priority: 500 Package: * Pin: release a=unstable Pin-Priority: 200
When you wish to install a package named "<package-name>
" with its dependencies from unstable
archive under this configuration, you issue the following command which switches target release with "-t
" option (Pin-Priority of unstable
becomes 990.):
$ sudo apt-get install -t unstable <package-name>
With this configuration, usual execution of "apt-get upgrade
" and "apt-get dist-upgrade
" (or "aptitude safe-upgrade
" and "aptitude full-upgrade
" for squeeze
) will upgrade packages which were installed from testing
archive using current testing
archive and packages which were installed from unstable
archive using current unstable
archive.
![]() |
Caution |
---|---|
Be careful not to remove " |
![]() |
Tip |
---|---|
I usually edit the " |
![]() |
Tip |
---|---|
If " |
If you wish to track particular packages in unstable
automatically without initial "-t unstable
" installation, you must create the "/etc/apt/preferences
" file and explicitly lists all those packages at the top of it as:
Package: <package-1> Pin: release a=unstable Pin-Priority: 700 Package: <package-2> Pin: release a=unstable Pin-Priority: 700 ...
These will set Pin-Priority value for each specific package. For example, in order to track the latest unstable
version of this "Debian Reference" in English, you should have following entries in the "/etc/apt/preferences
" file:
Package: debian-reference-en Pin: release a=unstable Pin-Priority: 700 Package: debian-reference-common Pin: release a=unstable Pin-Priority: 700
![]() |
Tip |
---|---|
This apt-pinning technique is valid even when you are tracking |
Here is another example of apt-pinning technique to include specific newer upstream version packages found in experimental
while tracking unstable
. You list all required archives in the "/etc/apt/sources.list
" file as:
deb http://ftp.us.debian.org/debian/ unstable main contrib non-free deb http://ftp.us.debian.org/debian/ experimental main contrib non-free deb http://security.debian.org/ testing/updates main contrib
The default Pin-Priority value for experimental
archive is always 1 (<<100) since it is NotAutomatic archive (see Section 2.5.3, “Archive level "Release" files”). There is no need to set Pin-Priority value explicitly in the "/etc/apt/preferences
" file just to use experimental
archive unless you wish to track particular packages in it automatically for next upgrading.
There are debian-volatile project and backports.org archives which provide updgrade packages for stable
.
![]() |
Warning |
---|---|
Do not use all packages available in the NotAutomatic archives such as |
![]() |
Caution |
---|---|
backports.org is a non-Debian archive, although its packages are signed by Debian developers. |
![]() |
Caution |
---|---|
Archive level Release files (see Section 2.5.3, “Archive level "Release" files”) are used for the rule of |
Here is an example of apt-pinning technique to include specific newer upstream version packages found in lenny-backports
while tracking lenny
and volatile
. You list all required archives in the "/etc/apt/sources.list
" file as:
deb http://ftp.us.debian.org/debian/ lenny main contrib non-free deb http://security.debian.org/ lenny/updates main contrib deb http://volatile.debian.org/debian-volatile/ lenny/volatile main contrib non-free deb http://volatile.debian.org/debian-volatile/ lenny/volatile-sloppy main contrib non-free deb http://backports.org/debian/ lenny-backports main contrib non-free
The default Pin-Priority value for backports.org and volatile-sloppy
archives are always 1 (<<100) since they are NotAutomatic archive (see Section 2.5.3, “Archive level "Release" files”). There is no need to set Pin-Priority value explicitly in the "/etc/apt/preferences
" file just to use for backports.org and volatile-sloppy
archive unless you wish to track packages automatically for next upgrading.
So whenever you wish to install a package named "<package-name>
" with its dependency from lenny-backports
archive, you use following command while switching target release with "-t
" option:
$ sudo apt-get install -t lenny-backports <package-name>
If you wish to upgrade particular packages, you must create the "/etc/apt/preferences
" file and explicitly lists all packages in it as:
Package: <package-1> Pin: release o=Backports.org archive Pin-Priority: 700 Package: <package-2> Pin: release o=volatile.debian.org Pin-Priority: 700 ...
Alternatively, with the "/etc/apt/preferences
" file as:
Package: * Pin: release a=stable , o=Debian Pin-Priority: 500 Package: * Pin: release a=lenny, o=volatile.debian.org Pin-Priority: 500 Package: * Pin: release a=lenny-backports, o=Backports.org archive Pin-Priority: 200 Package: * Pin: release a=lenny-sloppy, o=volatile.debian.org Pin-Priority: 200
execution of "apt-get upgrade
" and "apt-get dist-upgrade
" (or "aptitude safe-upgrade
" and "aptitude full-upgrade
" for squeeze
) will upgrade packages which were installed from stable
archive using current stable
archive and packages which were installed from other archives using current corresponding archive for all archives in the "/etc/apt/sources.list
" file.
The apt
package comes with its own cron script "/etc/cron.daily/apt
" to support the automatic download of packages. This script can be enhanced to perform the automatic upgrade of packages by installing the unattended-upgrades
package. These can be customized by parameters in "/etc/apt/apt.conf.d/02backup
" and "/etc/apt/apt.conf.d/50unattended-upgrades
" as described in "/usr/share/doc/unattended-upgrades/README
".
The unattended-upgrades
package is mainly intended for the security upgrade for the stable
system. If the risk of breaking an existing stable
system by the automatic upgrade is smaller than that of the system broken by the intruder using its security hole which has been closed by the security update, you should consider using this automatic upgrade with configuration parameters:
APT::Periodic::Update-Package-Lists "1"; APT::Periodic::Download-Upgradeable-Packages "1"; APT::Periodic::Unattended-Upgrade "1";
If you are running an unstable
system, you do not want to use the automatic upgrade since it will certainly break system some day. Even for such unstable
case, you may still want to download packages in advance to save time for the interactive upgrade with configuration parameters:
APT::Periodic::Update-Package-Lists "1"; APT::Periodic::Download-Upgradeable-Packages "1"; APT::Periodic::Unattended-Upgrade "0";
If you want to limit the download bandwidth for APT to e.g. 800Kib/sec (=100kiB/sec), you should configure APT with its configuration parameter with:
APT::Acquire::http::Dl-Limit "800";
![]() |
Caution |
---|---|
Downgrading is not officially supported by the Debian by design. It should be done only as a part of emergency recovery process. Despite of this situation, it is known to work well in many incidents. For critical systems, You should backup all important data on the system after the recovery operation and re-install the new system from the scratch. |
You may be lucky to downgrade from newer archive to older archive to recover from broken system upgrade by manipulating candidate version (see Section 2.7.3, “Tweaking candidate version”). This is lazy alternative to tedious actions of many "dpkg -i <broken-package>_<old-version>.deb
" commands (see Section 2.6.4, “Rescue using the dpkg command”).
For downgrading system tracking unstable
to testing
, change the "/etc/apt/sources.list
" file from:
deb http://ftp.us.debian.org/debian/ sid main contrib non-free
to:
deb http://ftp.us.debian.org/debian/ squeeze main contrib
and set the "/etc/apt/preferences
" file as:
Package: * Pin: release a=testing Pin-Priority: 1010
Then run "apt-get dist-upgrade
" to force downgrading of packages across the system. You should remove this special "/etc/apt/preferences
" file after the downgrading.
![]() |
Tip |
---|---|
It is good idea to remove (not purge!) as much packages to minimize dependency problems. You may need to manually remove and install some packages to get system downgraded. Linux kernel, bootloader, udev, PAM, APT, and networking related packages and their configuration files require special attention. |
Although the maintainer name listed in "/var/lib/dpkg/available
" and "/usr/share/doc/package_name/changelog
" provide some information on "who is behind the packaging activity", the actual uploader of the package is somewhat obscure. who-uploads
(1) in the devscripts
package identifies the actual uploader of Debian source packages.
If you are to compile a program from source to replace the Debian package, it is best to make it into a real local debianized package (*.deb
) and use private archive.
If you chose to compile a program from source and to install them under "/usr/local
" instead, you may need to use equivs
as a last resort to satisfy the missing package dependency.
Package: equivs Priority: extra Section: admin Description: Circumventing Debian package dependencies This is a dummy package which can be used to create Debian packages, which only contain dependency information.
For partial upgrades of the stable
system, rebuilding a package within its environment using the source package is desirable. This avoids massive package upgrades due to their dependencies. First, add the following entries to the "/etc/apt/sources.list
" of a stable
system:
deb-src http://http.us.debian.org/debian unstable main contrib non-free
Then install required packages for the compilation and download the source package by:
# apt-get update # apt-get dist-upgrade # apt-get install fakeroot devscripts build-essential $ apt-get build-dep foo $ apt-get source foo $ cd foo*
$ dch -i
+bp1
".
$ debuild $ cd .. # debi foo*.changes
Since mirroring whole subsection of Debian archive wastes disk space and network bandwidth, deployment of a local proxy server for APT is desirable consideration when you administer many systems on LAN. APT can be configure to use generic web (http) proxy servers such as squid
(see Section 6.5, “Other network application servers”) as described in apt.conf
(5) and in "/usr/share/doc/apt/examples/configure-index.gz
". The "$http_proxy
" environment variable can be used to override proxy server setting in the "/etc/apt/apt.conf
" file.
There are proxy tools specially for Debian archive. You should check BTS before using them.
Table 2.20. List of the proxy tools specially for Debian archive
package | popcon | size | description |
---|---|---|---|
approx
|
V:0.2, I:0.3 | 3860 | caching proxy server for Debian archive files (compiled OCaml program) |
apt-proxy
|
V:0.4, I:0.5 | 428 | Debian archive proxy and partial mirror builder (Python program) |
apt-cacher
|
V:0.3, I:0.5 | 244 | Caching proxy for Debian package and source files (Perl program) |
apt-cacher-ng
|
V:0.17, I:0.2 | 708 | Caching proxy for distribution of software packages (compiled C++ program) |
debtorrent
|
V:0.14, I:0.2 | 1173 | Bittorrent proxy for downloading Debian packages (Python program) |
![]() |
Caution |
---|---|
When Debian reorganizes its archive structure, these specialized proxy tools tend to require code rewrites by the package maintainer and may not be functional for a while. On the other hand, generic web (http) proxy servers are more robust and easier to cope with such changes. |
Here is an example for creating a small public package archive compatible with the modern secure APT system (see Section 2.5.2, “Top level "Release" file and authenticity”). Let's assume few things:
foo
"
www.example.com
"
apt-utils
, gnupg
, and other packages.
http://www.example.com/~foo/
" displays "/home/foo/public_html/index.html
"
amd64
"
One time setup of APT archive on your server system:
$ ssh foo@www.example.com $ gpg --gen-key ... $ gpg -K ... sec 1024D/3A3CB5A6 2008-08-14 uid Foo (ARCHIVE KEY) <foo@www.example.com> ssb 2048g/6856F4A7 2008-08-14 $ gpg --export -a 3A3CB5A6 >foo.public.key
3A3CB5A6
"
foo.public.key
" file.
$ umask 022 $ mkdir -p ~/public_html/debian/pool/main $ mkdir -p ~/public_html/debian/dists/unstable/main/binary-amd64 $ mkdir -p ~/public_html/debian/dists/unstable/main/source $ cd ~/public_html/debian $ cat > dists/unstable/main/binary-amd64/Release << EOF Archive: unstable Version: 4.0 Component: main Origin: Foo Label: Foo Architecture: amd64 EOF $ cat > dists/unstable/main/source/Release << EOF Archive: unstable Version: 4.0 Component: main Origin: Foo Label: Foo Architecture: source EOF $ cat >aptftp.conf <<EOF APT::FTPArchive::Release { Origin "Foo"; Label "Foo"; Suite "unstable"; Codename "sid"; Architectures "amd64"; Components "main"; Description "Public archive for Foo"; }; EOF $ cat >aptgenerate.conf <<EOF Dir::ArchiveDir "."; Dir::CacheDir "."; TreeDefault::Directory "pool/"; TreeDefault::SrcDirectory "pool/"; Default::Packages::Extensions ".deb"; Default::Packages::Compress ". gzip bzip2"; Default::Sources::Compress "gzip bzip2"; Default::Contents::Compress "gzip bzip2"; BinDirectory "dists/unstable/main/binary-amd64" { Packages "dists/unstable/main/binary-amd64/Packages"; Contents "dists/unstable/Contents-amd64"; SrcPackages "dists/unstable/main/source/Sources"; }; Tree "dists/unstable" { Sections "main"; Architectures "amd64 source"; }; EOF
Repetitive update of APT archive contents on your server system:
~foo/public_html/debian/pool/main/
" by executing "dupload -t foo changes_file
" in client while having "~/.dupload.conf
" containing:
$cfg{'foo'} = { fqdn => "www.example.com", method => "scpb", incoming => "/home/foo/public_html/debian/pool/main", # The dinstall on ftp-master sends emails itself dinstall_runs => 1, }; $cfg{'foo'}{postupload}{'changes'} = " echo 'cd public_html/debian ; apt-ftparchive generate -c=aptftp.conf aptgenerate.conf; apt-ftparchive release -c=aptftp.conf dists/unstable >dists/unstable/Release ; rm -f dists/unstable/Release.gpg ; gpg -u 3A3CB5A6 -bao dists/unstable/Release.gpg dists/unstable/Release'| ssh foo@www.example.com 2>/dev/null ; echo 'Package archive created!'";
The postupload hook script initiated by dupload
(1) creates updated archive files for each upload.
You can add this small public archive to the apt-line of your client system:
$ sudo bash # echo "deb http://www.example.com/~foo/debian/ unstable main" \ >> /etc/apt/sources.list # apt-key add foo.public.key
![]() |
Tip |
---|---|
If the archive is located on the local file system, you can use " |
To make a local copy of the package and debconf selection states:
# dpkg --get-selections '*' > selection.dpkg # debconf-get-selections > selection.debconf
Here, "*
" makes "selection.dpkg
" to include package entries for "purge" too.
You can transfer these 2 files to another computer, and install there with:
# dselect update # debconf-set-selections < myselection.debconf # dpkg --set-selections < myselection.dpkg # apt-get -u dselect-upgrade # or dselect install
If you are thinking about managing many servers in a cluster with practically the same configuration, you should consider to use specialized package such as fai
to manage the whole system.
alien
(1) enables the conversion of binary packages provided in Red Hat rpm
, Stampede slp
, Slackware tgz
, and Solaris pkg
file formats into a Debian deb
package. If you want to use a package from another Linux distribution than the one you have installed on your system, you can use alien
to convert it from your preferred package format and install it. alien
also supports LSB packages.
![]() |
Warning |
---|---|
|
The current "*.deb
" package contents can be extracted without using dpkg
(1) on any Unix-like environment using standard ar
(1) and tar
(1).
# ar x /path/to/dpkg_<version>_<arch>.deb # ls total 24 -rw-r--r-- 1 bozo bozo 1320 2007-05-07 00:11 control.tar.gz -rw-r--r-- 1 bozo bozo 12837 2007-05-07 00:11 data.tar.gz -rw-r--r-- 1 bozo bozo 4 2007-05-07 00:11 debian-binary # mkdir control # mkdir data # tar xvzf control.tar.gz -C control # tar xvzf data.tar.gz -C data
You can also browse package content using the mc
command.
You should read:
aptitude
(8), dpkg
(1), tasksel
(8), apt-get
(8), apt-config
(8), apt-key
(8), sources.list
(5), apt.conf
(5), and apt_preferences
(5);
/usr/share/doc/apt-doc/guide.html/index.html
" and "/usr/share/doc/apt-doc/offline.html/index.html
" from the apt-doc
package; and
/usr/share/doc/aptitude/html/en/index.html
" from the aptitude-doc-en
package.
The official and detailed secondary information on the Debian archive are given by:
The tutorial for building of a Debian package for Debian users is given by:
It is wise for you as the system administrator to know roughly how the Debian system is started and configured. Although the exact details are in the source files of the packages installed and their documentations, it is a bit overwhelming for most of us.
I did my best to provide a quick overview of the key points of the Debian system and their configuration for your reference, based on the current and previous knowledge of mine and others. Since the Debian system is a moving target, the situation over the system may have been changed. Before making any changes to the system, you should refer to the latest documentation for each package.
The computer system undergoes several phases of boot strap processes from the power-on event until it offers the fully functional operating system (OS) to the user.
For simplicity, I will limit discussion to the typical PC platform with the default installation.
The typical boot strap process is like a four-stage rocket. Each stage rocket hands over the system control to the next stage one. Here each stage corresponds to:
Of course, these can be configured differently. For example, if you compiled your own kernel, you may be skipping the step with the mini-Debian system. So please do not assume this is the case for your system until you check it yourself.
![]() |
Note |
---|---|
For non-legacy PC platform such as the SUN or the Macintosh system, the BIOS on ROM and the partition on the disk may be quite different (Section 9.3.1, “Partition configuration”). Please seek the platform specific documentations elsewhere for such a case. |
The BIOS is the 1st stage of the boot process which is started by the power-on event. The BIOS residing on the read only memory (ROM) is excuted from the particular memory address to which the program counter of CPU is initialized by the power-on event.
This BIOS performs the basic initialization of the hardware (POST: power on self test) and hands the system control to the next step which you provide. The BIOS is usually provided with the hardware.
The BIOS startup screen usually indicates what key(s) to press to enter the BIOS setup screen to configure the BIOS behavior. Popular keys used are F1, F2, F10, Esc, Ins, and Del. If your BIOS startup screen is hidden by a nice graphics screen, you may press some keys such as Esc to disable this. These keys are highly dependent on the hardware.
The hardware location and the priority of the code started by the BIOS can be selected from the BIOS setup screen. Typically, the first few sectors of the first found selected device (hard disk, floppy disk, CD-ROM, …) are loaded to the memory and this initial code is executed. This initial code can be:
Typically, the system is booted from the specified partition of the primary hard disk partition. First 2 sectors of the hard disk on legacy PC contain the master boot record (MBR). The disk partition information including the boot selection is recorded at the end of this MBR. The first boot loader code executed from the BIOS occupies the rest of this MBR.
The boot loader is the 2nd stage of the boot process which is started by the BIOS. It loads the system kernel image and the initrd image to the memory and hands control over to them. This initrd image is the root filesystem image and its support depends on the bootloader used.
The Debian system normally uses the Linux kernel as the default system kernel. The initrd image for the current 2.6 Linux kernel is technically the initramfs (initial RAM filesystem) image. The initramfs image is a gzipped cpio archive of files in the root filesystem.
The default install of the Debian system places first-stage GRUB boot loader code into the MBR for the PC platform. There are many boot loaders and configuration options available.
Table 3.1. List of boot loaders.
bootloader | package | popcon | size | initrd | description |
---|---|---|---|---|---|
GRUB Legacy | grub | V:24, I:91 | 1908 | Supported |
This is smart enough to understand disk partitions and file systems such as vfat, ext3, …. (lenny default)
|
GRUB 2 | grub-pc | V:0.9, I:3 | 1600 | Supported | This is smart enough to understand disk partitions and file systems such as vfat, ext3, …. |
GRUB 2 | grub-rescue-pc | V:0.04, I:0.5 | 2852 | Supported | This is GRUB 2 bootable rescue images (CD and floppy) (PC/BIOS version) |
Lilo | lilo | V:0.7, I:3 | 1192 | Supported | This relies on the sector locations of data on the hard disk. (Old) |
Isolinux | syslinux | V:1.1, I:7 | 160 | Supported | This understands the ISO9660 file system. This is used by the boot CD. |
Syslinux | syslinux | V:1.1, I:7 | 160 | Supported | This understands the MSDOS file system (FAT). This is used by the boot floppy. |
Loadlin | loadlin | V:0.02, I:0.2 | 140 | Supported | New system is started from the FreeDOS/MSDOS system. |
MBR by Neil Turton | mbr | V:1.0, I:6 | 96 | Not supported | This is free software which substitutes MSDOS MBR. This only understands disk partitions. |
![]() |
Warning |
---|---|
Do not play with boot loaders without having bootable rescue media (CD or floppy) created from images in the |
For GRUB Legacy, the menu configuration file is located at "/boot/grub/menu.lst
". For example, it has entries like:
title Debian GNU/Linux root (hd0,2) kernel /vmlinuz root=/dev/hda3 ro initrd /initrd.img
For GRUB 2, the menu configuration file is located at "/boot/grub/grub.cfg
". It is automatically generated by "/usr/sbin/update-grub
" using templates from "/etc/grub.d/*
" and settings from "/etc/default/grub
". For example, it has entries like:
menuentry "Debian GNU/Linux" { set root=(hd0,3) linux /vmlinuz root=/dev/hda3 initrd /initrd.img }
For these examples, these GRUB parameters mean:
Table 3.2. The meaning of GRUB parameters.
GRUB parameter | meaning |
---|---|
root
|
Use 3rd partition on the primary disk by setting it as "(hd0,2) " in GRUB legacy or as "(hd0,3) " in GRUB 2.
|
kernel
|
Use kernel located at "/vmlinuz " with kernel parameter: "root=/dev/hda3 ro ".
|
initrd
|
Use initrd/initramfs image located at "/initrd.img ".
|
![]() |
Note |
---|---|
The value of the partition number used by GRUB legacy program is one less than normal one used by Linux kernel and utility tools. GRUB 2 program fixes this problem. |
![]() |
Tip |
---|---|
UUID (see Section 9.3.2, “Accessing partition using UUID”) may be used to identify a block special device instead of its file name such as " |
![]() |
Tip |
---|---|
You can start a boot loader from another boot loader using techniques called chain loading. |
See "info grub
" and grub-install
(8).
The mini-Debian system is the 3rd stage of the boot process which is started by the boot loader. It runs the system kernel with its root filesystem on the memory. This is an optional preparatory stage of the boot process.
![]() |
Note |
---|---|
The term "the mini-Debian system" is coined by the author to describe this 3rd stage boot process for this document. This system is commonly referred as the initrd or initramfs system. Similar system on the memory is used by the Debian Installer. |
The "/init
" script is executed as the first program in this root filesystem on the memory. It is a shell script program which initializes the kernel in user space and hands control over to the next stage. This mini-Debian system offers flexibility to the boot process such as adding kernel modules before the main boot process or mounting the root file system as an encrypted one.
You can interrupt this part of the boot process to gain root shell by providing "break=init
" etc. to the kernel boot parameter. See the "/init
" script for more break conditions. This shell environment is sophisticated enough to make a good inspection of your machine's hardware.
Commands available in this mini-Debian system are stripped down ones and mainly provided by a GNU tool called busybox
(1).
![]() |
Caution |
---|---|
You need to use " |
The normal Debian system is the 4th stage of the boot process which is started by the mini-Debian system. The system kernel for the mini-Debian system continues to run in this environment. The root filesystem is switched from the one on the memory to the one on the real harddisk filesystem.
The "/sbin/init
" program is executed as the first program and performs the main boot process. The Debian normally uses the traditional sysvinit scheme with the sysv-rc
package. See init
(8), inittab
(5), and "/usr/share/doc/sysv-rc/README.runlevels.gz
" for the exact explanation. Following is a simplified overview of this main boot process:
/etc/inittab
" description.
The initial runlevel used for multi-user mode is specified with the "init=
" kernel boot parameter or in the "initdefault" line of the "/etc/inittab
". The Debian system as installed starts at the runlevel 2.
All scripts executed by the init system are located in the directory "/etc/init.d/
".
![]() |
Tip |
---|---|
For alternative boot mechanism to the |
Each runlevel uses a directory for its configuration and has specific meaning:
Table 3.3. List of runlevels and meanings.
runlevel | directory | meaning |
---|---|---|
N
|
none |
System bootup (NONE). There is no "/etc/rcN.d/ " directory.
|
0
|
/etc/rc0.d/
|
Halt the system. |
S
|
/etc/rcS.d/
|
Single-user mode on boot. The lower case "s " can be used as alias.
|
1
|
/etc/rc1.d/
|
Single-user mode switched from multi-user mode. |
2
|
/etc/rc2.d/
|
Multi-user mode. |
3
|
/etc/rc3.d/
|
,, |
4
|
/etc/rc4.d/
|
,, |
5
|
/etc/rc5.d/
|
,, |
6
|
/etc/rc6.d/
|
Reboot the system. |
7
|
/etc/rc7.d/
|
Valid multi-user mode but not normally used. |
8
|
/etc/rc8.d/
|
,, |
9
|
/etc/rc9.d/
|
,, |
You can change the runlevel from the console to, e.g., 4 by:
$ sudo telinit 4
![]() |
Caution |
---|---|
The Debian system does not pre-assign any special meaning differences among the runlevels between 2 and 5. The system administrator on the Debian system may change this. (I.e., Debian is not Red Hat Linux nor Solaris by Sun Microsystems nor HP-UX by Hewlett Packard nor AIX by IBM nor …) |
![]() |
Caution |
---|---|
The Debian system does not populate directories for the runlevels between 7 and 9 when the package is installed. Traditional Unix variants don’t use these runlevels. |
The names of the symlinks in the runlevel directories have the form "S<2-digit-number><original-name>
" or "K<2-digit-number><original-name>
". The 2-digit-number is used to determine the order in which to run the scripts. "S
" is for "Start" and "K
" is for "Kill".
When init
(8) or telinit
(8) commands goes into the runlevel to "<n>":
K
" in "/etc/rc<n>.d/
" are executed in alphabetical order with the single argument "stop
". (killing services)
S
" in "/etc/rc<n>.d/
" are executed in alphabetical order with the single argument "start
". (starting services)
For example, if you had the links "S10sysklogd
" and "S20exim4
" in a runlevel directory, "S10sysklogd
" would run before "S20exim4
".
![]() |
Warning |
---|---|
It is not advisable to make any changes to symlinks in " |
For example, let's set up runlevel system somewhat like Red Hat Linux, i.e.:
gdm
(1) in runlevel=(0,1,2,6), and
gdm
(1) in runlevel=(3,4,5).
This can be done by using editor on the "/etc/inittab
" file to change starting runlevel and using user friendly runlevel management tools such as sysv-rc-conf
or bum
to edit the runlevel. If you are to use command line only instead, here is how you do it (after the default installation of the gdm
package and selecting it to be the choice of display manager):
# cd /etc/rc2.d ; mv S21gdm K21gdm # cd /etc ; perl -i -p -e 's/^id:.:/id:3:/' inittab
Please note the "/etc/X11/default-display-manager
" file is checked when starting the display manager daemons: xdm
, gdm
, kdm
, and wdm
.
![]() |
Note |
---|---|
You can still start X from any console shell with the |
The default parameter for each init script in "/etc/init.d/
" is given by the corresponding file in "/etc/default/
" which contains environment variable assignments only. This choice of directory name is specific to the Debian system. It is roughly the equivalent of the "/etc/sysconfig
" directory found in Red Hat Linux and other distributions. For example, "/etc/default/cron
" can be used to control how "/etc/init.d/cron
" works.
The "/etc/default/rcS
" file can be used to customize boot-time defaults for motd
(5), sulogin
(8), etc.
If you cannot get the behavior you want by changing such variables then you may modify the init scripts themselves. These are configuration files editable by system administrators.
The kernel maintains the system hostname. The initscript "/etc/init.d/hostname.sh
" sets the system hostname at boot time (using the hostname
command) to the name stored in "/etc/hostname
". This file should contain only the system hostname, not a fully qualified domain name.
To print out the current hostname run hostname
(1) without an argument.
Network interfaces are initialized under single-user mode on boot by the initscript "/etc/init.d/ifupdown-clean
" and "/etc/init.d/ifupdown
". See Chapter 5, Network setup for how to configure them.
Many network services (see Chapter 6, Network applications) are started directly as daemon processes at boot time by the initscript, e.g., "/etc/rc2.d/S20exim4
" (for RUNLEVEL=2) which is a symlink to "/etc/init.d/exim4
".
Some network services can be started on demand using the super-server, inetd
(or its equivalents). The inetd
is started at boot time by "/etc/rc2.d/S20inetd
" (for RUNLEVEL=2) which is a symlink to "/etc/init.d/inetd
". Essentially, inetd
allows one running daemon to invoke several others, reducing load on the system.
Whenever a request for service arrives, its protocol and service are identified by looking them up in the databases in "/etc/protocols
" and "/etc/services
". inetd
then looks up a normal Internet service in the "/etc/inetd.conf
" database, or a Open Network Computing Remote Procedure Call (ONC RPC)/Sun RPC based service in "/etc/rpc.conf
".
For system security, make sure to disable unused services in "/etc/inetd.conf
". Sun-RPC services need to be active for NFS and other RPC-based programs.
Sometimes, inetd
does not start the intended server directly but starts the TCP wrapper, tcpd
, with the intended server name as its argument in "/etc/inetd.conf
". In this case, tcpd
runs the appropriate server program after logging the request and doing some additional checks using "/etc/hosts.deny
" and "/etc/hosts.allow
".
If you have problems with remote access in a recent Debian system, comment out "ALL: PARANOID" in "/etc/hosts.deny
" if it exists. (But you must be careful on security risks involved with this kind of action.)
For details, see inetd
(8), inetd.conf
(5), protocols
(5), services
(5), tcpd
(8), hosts_access
(5), and hosts_options
(5).
For more information on Sun-RPC, see rpcinfo
(8), portmap
(8), and "/usr/share/doc/portmap/portmapper.txt.gz
".
The system message can be customized by "/etc/syslog.conf
" for both the log file and on-screen display. See syslogd
(8) and syslog.conf
(5). See also Section 9.2.2, “Log analyzer”.
The kernel message can be customized by "/etc/init.d/klogd
" for both the log file and on-screen display. Set "KLOGD='-c 3'
" in this script and run "/etc/init.d/klogd restart
". See klogd
(8).
You may directly change the error message level by:
# dmesg -n3
Here:
Table 3.4. List of kernel error levels.
error level value | error level name | meaning |
---|---|---|
0 | KERN_EMERG | system is unusable |
1 | KERN_ALERT | action must be taken immediately |
2 | KERN_CRIT | critical conditions |
3 | KERN_ERR | error conditions |
4 | KERN_WARNING | warning conditions |
5 | KERN_NOTICE | normal but significant condition |
6 | KERN_INFO | informational |
7 | KERN_DEBUG | debug-level messages |
For Linux kernel 2.6, the udev system provides mechanism for the automatic hardware discovery and initialization (see udev
(7)). Upon discovery of each device by the kernel, the udev system starts a user process which uses information from the sysfs filesystem (see Section 1.2.12, “procfs and sysfs”), loads required kernel modules supporting it using the modprobe
(8) program (see Section 3.5.11, “The kernel module initialization”), and creates corresponding device nodes.
![]() |
Tip |
---|---|
If " |
The name of device nodes can be configured by files in "/etc/udev/rules.d/
" (see "/usr/share/doc/udev/writing_udev_rules/index.html
").
Since the udev system is somewhat a moving target, I leave details to other documentations and describe the minimum information here.
The modprobe
(8) program enables us to configure running Linux kernel from user process by adding and removing kernel modules. The udev system (see Section 3.5.10, “The udev system”) automates its invocation to help the kernel module initialization.
There are non-hardware modules and special hardware driver modules, such as:
iptables
(8), Section 5.8, “Netfilter”),
/etc/modules
" file (see modules
(5)).
The configuration files for the modprobe
(8) program are located under the "/etc/modprobes.d/
" directory as explained in modprobe.conf
(5). (If you want to avoid some kernel modules to be auto-loaded, consider to blacklist them in the "/etc/modprobes.d/blacklist
" file.)
The "/lib/modules/<version>/modules.dep
" file generated by the depmod
(8) program describes module dependencies used by the modprobe
(8) program.
![]() |
Note |
---|---|
If you experience module loading issues with boot time module loading or with |
The modinfo
(8) program shows information about a Linux kernel module.
The lsmod
(8) program nicely formats the contents of the "/proc/modules
", showing what kernel modules are currently loaded.
![]() |
Tip |
---|---|
You can identify exact hardware on your system. See Section 9.6.3, “Hardware identification”. |
![]() |
Tip |
---|---|
You may configure hardware at boot time to activate expected hardware features. See Section 9.6.4, “Hardware configuration”. |
![]() |
Tip |
---|---|
You can add support for your device by recompiling kernel. See Section 9.7, “The kernel”. |
When a person (or a program) requests access to the system, authentication confirms the identity to be a trusted one.
![]() |
Warning |
---|---|
Configuration errors of PAM may lock you out of your own system. You must have a rescue CD handy or setup an alternative boot partition. To recover, boot the system with them and correct things from there. |
Normal Unix authentication is provided by the pam_unix
(8) module under the PAM (Pluggable Authentication Modules). Its 3 important configuration files, with ":
" separated entries, are:
Table 4.1. 3 important configuration files for pam_unix
(8).
file | permission | user | group | description |
---|---|---|---|---|
/etc/passwd
|
-rw-r--r--
|
root
|
root
|
The (sanitized) user account information. |
/etc/shadow
|
-rw-r-----
|
root
|
shadow
|
The secure user account information. |
/etc/group
|
-rw-r--r--
|
root
|
root
|
The group information. |
"/etc/passwd
" contains:
... user1:x:1000:1000:User1 Name,,,:/home/user1:/bin/bash user2:x:1001:1001:User2 Name,,,:/home/user2:/bin/bash ...
As explained in passwd
(5), each ":
" separated entry of this file means:
The second entry of "/etc/passwd
" was used for the encrypted password entry. After the introduction of "/etc/shadow
", this entry is used for the password specification entry.
Table 4.2. The second entry content of "/etc/passwd
".
content | meaning |
---|---|
(empty) | passwordless account |
x |
the encrypted password is in "/etc/shadow "
|
* | no login for this account |
! | no login for this account |
"/etc/shadow
" contains:
... user1:$1$Xop0FYH9$IfxyQwBe9b8tiyIkt2P4F/:13262:0:99999:7::: user2:$1$vXGZLVbS$ElyErNf/agUDsm1DehJMS/:13261:0:99999:7::: ...
As explained in shadow
(5), each ":
" separated entry of this file means:
$1$
" indicates use of the MD5 encryption. The "*" indicates no login.
"/etc/group
" contains:
... group1:x:20:user1,user2 ...
As explained in shadow
(5), each ":
" separated entry of this file means:
"/etc/gshadow
" provides the similar function as "/etc/shadow
" for "/etc/group
" but is not really used.
![]() |
Note |
---|---|
The actual group membership of a user may be dynamically added if " |
![]() |
Note |
---|---|
The |
Here are few notable commands to manage account information:
Table 4.3. List of commands to manage account information.
command | function |
---|---|
getent passwd <user_name>
|
browse account information of "<user_name> "
|
getent shadow <user_name>
|
browse shadowed account information of "<user_name> "
|
getent group <group_name>
|
browse group information of "<group_name> "
|
passwd
|
manage password for the account |
passwd -e
|
set one-time password for the account activation |
chage
|
manage password aging information |
You may need to have the root privilege for some functions to work. See crypt
(3) for the password and data encryption.
![]() |
Note |
---|---|
On the system set up with PAM and NSS as the Debian alioth machine, the content of local " |
When creating an account during your system installation or with the passwd
(1) command, you should choose a good password which consists of 6 to 8 characters including one or more characters from each of the following sets according to passwd
(1):
![]() |
Warning |
---|---|
Do not chose guessable words for the password. |
There are independent tools to generate encrypted password with salt:
Table 4.4. List of tools to generate password.
package | popcon | size | command | function |
---|---|---|---|---|
whois
|
V:11, I:89 | 344 |
mkpasswd
|
over-featured front end to the crypt (3) library
|
openssl
|
V:29, I:90 | 2360 |
openssl passwd
|
compute password hashes (OpenSSL). passwd (1ssl)
|
Modern Unix-like systems such as the Debian system provide PAM (Pluggable Authentication Modules) and NSS (Name Service Switch) mechanism to the local system administrator to configure his system. The role of these can be summarizes as:
ls
(1) and id
(1).
These PAM and NSS systems need to be configured consistently.
The notable packages of PAM and NSS systems are:
Table 4.5. List of notable PAM and NSS systems.
package | popcon | size | description |
---|---|---|---|
libpam-modules
|
V:82, I:99 | 972 | Pluggable Authentication Modules (basic service) |
libpam-ldap
|
V:1.8, I:4 | 404 | Pluggable Authentication Module allowing LDAP interfaces |
libpam-cracklib
|
V:0.8, I:1.8 | 132 | Pluggable Authentication Module to enable cracklib support |
libpam-doc
|
I:0.8 | 1156 | Pluggable Authentication Modules (documentation in html and text) |
libc6
|
V:95, I:99 | 11496 | GNU C Library: Shared libraries which also provides "Name Service Switch" service |
glibc-doc
|
I:4 | 1800 | GNU C Library: Manpages |
glibc-doc-reference
|
I:1.5 | 12160 | GNU C Library: Reference manual in info, pdf and html format (non-free) |
libnss-mdns
|
I:54 | 144 | NSS module for Multicast DNS name resolution |
libnss-ldap
|
I:4 | 312 | NSS module for using LDAP as a naming service |
libnss-ldapd
|
V:0.12, I:0.2 | 356 |
NSS module for using LDAP as a naming service (new folk of libnss-ldap )
|
libpam-doc
is essential for learning PAM configuration.
glibc-doc-reference
is essential for learning NSS configuration.
![]() |
Note |
---|---|
You can see more extensive and current list by " |
![]() |
Note |
---|---|
PAM is the most basic way to initialize environment variables for each program with the system wide default value. |
Here are few notable configuration files accessed by the PAM:
Table 4.6. List of configuration files accessed by the PAM.
configuration file | function |
---|---|
/etc/pam.d/<program_name>
|
set up PAM configuration for the "<program_name> " program. See pam (7) and pam.d (5).
|
/etc/nsswitch.conf
|
set up NSS configuration with the entry for each service. See nsswitch.conf (5).
|
/etc/nologin
|
limit the user login by the pam_nologin (8) module.
|
/etc/securetty
|
limit the tty for the root access by the pam_securetty (8) module.
|
/etc/security/access.conf
|
set access limit by the pam_access (8) module.
|
/etc/security/group.conf
|
set group based restraint by the pam_group (8) module.
|
/etc/security/pam_env.conf
|
set environment variables by the pam_env (8) module.
|
/etc/environment
|
set additional environment variables by the pam_env (8) module with the "readenv=1 " argument.
|
/etc/default/locale
|
set locale by pam_env (8) module with the "readenv=1 envfile=/etc/default/locale " argument. (Debian)
|
/etc/security/limits.conf
|
set resource restraint (ulimit, core, …) by the pam_linits (8) module.
|
/etc/security/time.conf
|
set time restraint by the pam_time (8) module.
|
The limitation of the password selection is implemented by the PAM modules, pam_unix
(8) and pam_cracklib
(8). They can be configured by their arguments.
![]() |
Tip |
---|---|
PAM modules use suffix " |
The modern centralized system management can be deployed using the centralized Lightweight Directory Access Protocol (LDAP) server to administer many Unix-like and non-Unix-like systems on the network. The open source implementation of the Lightweight Directory Access Protocol is OpenLDAP Software.
The LDAP server provides the account information through the use of PAM and NSS with libpam-ldap
and libnss-ldap
packages for the Debian system. Several actions are required to enable this (I have not used this setup and the following is purely secondary information. Please read this in this context.):
slapd
(8).
/etc/pam.d/
" directory to use "pam_ldap.so
" instead of the default "pam_unix.so
".
/etc/nsswitch.conf
" file to use "ldap
" instead of the default ("compat
" or "file
").
/etc/pam_ldap.conf
" as the configuration file for libpam-ldap
and "/etc/pam_ldap.secret
" as the file to store the password of the root.
/etc/libnss-ldap.conf
" as the configuration file for libnss-ldap
.
libpam-ldap
to use SSL (or TLS) connection for the security of password.
libnss-ldap
to use SSL (or TLS) connection to ensure integrity of data at the cost of the LDAP network overhead.
nscd
(8) locally to cache any LDAP search results in order to reduce the LDAP network traffic.
See documentations in pam_ldap.conf
(5) and "/usr/share/doc/libpam-doc/html/
" offered by the libpam-doc
package and "info libc 'Name Service Switch'
" offered by the glibc-doc
package.
Similarly, you can set up alternative centralized systems with:
This is the famous phrase at the bottom of the old "info su
" page by Richard M. Stallman. Not to worry: the current su
command in Debian uses PAM, so that one can restrict the ability to use su
to the root
group by enabling the line with "pam_wheel.so
" in "/etc/pam.d/su
".
Installing the libpam-cracklib
package will enable you to force stricter password rule, for example, by having active lines in "/etc/pam.d/common-password
" as:
For lenny
:
password required pam_cracklib.so retry=3 minlen=9 difok=3 password required pam_unix.so use_authtok nullok md5
For squeeze
:
password required pam_cracklib.so retry=3 minlen=9 difok=3 password [success=1 default=ignore] pam_unix.so use_authtok nullok md5 password requisite pam_deny.so password required pam_permit.so
![]() |
Note |
---|---|
See Section 9.5.15, “Alt-SysRq” for restricting the kernel secure attention key (SAK) feature. |
sudo
(8) is a program designed to allow a sysadmin to give limited root privileges to users and log root activity. sudo
requires only an ordinary user's password. Install sudo
package and activate it by setting options in "/etc/sudoers
". See configuration example at "/usr/share/doc/sudo/examples/sudoers
".
My usage of sudo
for the single user system (see Section 1.1.12, “sudo configuration”) is aimed to protect myself from my own stupidity. Personally, I consider using sudo
a better alternative to using the system from the root account all the time. For example, following will change the owner of "<some_file>
" to "<my_name>
":
$ sudo chown <my_name> <some_file>
Of course if you know the root password (as self-installed Debian users do), any command can be run under root from any user's account using "su -c
".
Security-Enhanced Linux (SELinux) is a framework to tighten privilege model tighter than the ordinary Unix-like security model with the mandatory access control (MAC) policies. The root power may be restricted under some conditions.
The Internet super-server, inetd
(8), is started at boot time by "/etc/rc2.d/S20inetd
" (for RUNLEVEL=2), which is a symlink to "/etc/init.d/inetd
". Essentially, inetd
allows one running daemon to invoke several others, reducing load on the system.
Whenever a request for service arrives, its protocol and service are identified by looking them up in the databases in "/etc/protocols
" and "/etc/services
". inetd
then looks up a normal Internet service in "/etc/inetd.conf
", or a Sun RPC based service in "/etc/rpc.conf
".
For system security, make sure to disable unused services in "/etc/inetd.conf
". Sun RPC services need to be active for NFS and other RPC based programs.
Sometimes, inetd
does not start the intended server directly but starts the TCP wrapper program, tcpd
(8), with the intended server name as its argument in "/etc/inetd.conf
". In this case, tcpd
runs the appropriate server program after logging the request and doing some additional checks using "/etc/hosts.deny
" and "/etc/hosts.allow
".
If you have problems with remote access in a recent Debian system, comment out "ALL: PARANOID
" in "/etc/hosts.deny
" if it exists.
For details, see inetd
(8), inetd.conf
(5), protocols
(5), services
(5), tcpd
(8), hosts_access
(5), and hosts_options
(5).
For more information on Sun RPC, see rpcinfo
(8), portmap
(8), and "/usr/share/doc/portmap/portmapper.txt.gz
".
There are also non-PAM based access control available for atd
(8) and cron
(8).
The information here may not be sufficient for your security needs but it should be a good start.
Many popular transportation layer services communicate messages including password authentication in the plain text. It is very bad idea to transmit password in the plain text over the wild Internet where it can be intercepted. You can run these services over "Transport Layer Security" (TLS) or its predecessor, "Secure Sockets Layer" (SSL) to secure entire communication including password by the encryption.
Table 4.7. List of insecure and secure services and ports.
insecure service name | port | secure service name | port |
---|---|---|---|
www (http) | 80 | https | 443 |
smtp (mail) | 25 | ssmtp (smtps) | 465 |
ftp-data | 20 | ftps-data | 989 |
ftp | 21 | ftps | 990 |
telnet | 23 | telnets | 992 |
imap2 | 143 | imaps | 993 |
pop3 | 110 | pop3s | 995 |
ldap | 389 | ldaps | 636 |
The encryption costs CPU time. As a CPU friendly alternative, you can keep communication in plain text while securing just password with the secure authentication protocol such as "Authenticated Post Office Protocol" (APOP) for POP and "Challenge-Response Authentication Mechanism MD5" (CRAM-MD5) for SMTP and IMAP. (For sending mail messages over the Internet to your mail server from your mail client, it is recently popular to use new message submission port 587 instead of traditional SMTP port 25 to avoid port 25 blocking by the network provider while authenticating yourself with CRAM-MD5.)
The Secure Shell (SSH) program provides secure encrypted communications between two untrusted hosts over an insecure network with the secure authentication. It consists of the OpenSSH client, ssh
(1), and the OpenSSH daemon, sshd
(8). This SSH can be used to tunnel the insecure protocol communication such as POP and X securely over the Internet with the port forwarding feature.
The client tries to authenticate itself using host-based authentication, public key authentication, challenge-response authentication, or password authentication. The use of public key authentication enables the remote password-less login. See Section 6.4, “The remote access server and utility (SSH)”.
Even when you run secure services such as Secure Shell (SSH) and Point-to-point tunneling protocol (PPTP) servers, there are still chances for the break-ins using brute force password guessing attack etc. from the Internet. Use of the firewall policy (see Section 5.8, “Netfilter”) together with the following secure tools may improve the security situation.
Table 4.8. List of tools to provide extra security measures.
package | popcon | size | description |
---|---|---|---|
knockd
|
V:0.14, I:0.3 | 164 |
small port-knock daemon knocked (1) and client konck (1)
|
denyhosts
|
V:1.5, I:1.9 | 432 | an utility to help sysadmins thwart ssh hackers |
fail2ban
|
V:3, I:3 | 616 | bans IPs that cause multiple authentication errors |
libpam-shield
|
V:0.01, I:0.05 | 172 | locks out remote attackers trying password guessing |
To prevent people to access your machine with root privilege, you need to:
With physical access to hard disk, resetting the password is relatively easy;
/etc/passwd
" in the root partition and make the second entry for the root
account empty.
If you have the edit access to the GRUB menu entry (see Section 3.3, “Stage 2: the boot loader”) for grub-rescue-pc
at the boot time, it is even easier:
root=/dev/hda6 rw init=/bin/sh
".
/etc/passwd
" and make the second entry for the root
account empty.
The root shell of the system is now accessible without password.
![]() |
Note |
---|---|
Once you have root shell access, you can compromise password for all user accounts using brute force password cracking tools such as |
The only reasonable software solution to avoid all these concerns is to use software encrypted root partition (or "/etc
" partition) using dm-crypt and initramfs (see Section 9.4, “Data encryption tips”). You always need password to boot the system, though.
![]() |
Tip |
---|---|
For general guide to the GNU/Linux networking, read the Linux Network Administrators Guide. |
The traditional TCP/IP network setup on Debian system uses ifupdown
package as a high level tool. There are 2 typical cases:
resolvconf
package and enable you to switch your network configuration easily (see Section 5.3.4, “The network interface served by the DHCP”).
resolvconf
package and keep your system simple (see Section 5.3.5, “The network interface with the static IP”).
We will describe these traditional cases in detail here.
We will also touch on some alternative high level tools such as network-manager
and wicd
which ease configuration of wireless networks (see Section 5.5.2, “Automatic network configuration”).
Let's review the basic network infrastructure on the modern Debian system.
The naming for the domain name is a tricky one for the normal PC workstation users. The PC workstation may be mobile one hopping around the network or located behind the NAT firewall inaccessible from the Internet. For such case, you may not want the domain name to be a valid domain name to avoid name collision.
According to rfc2606, "invalid
" seems to be a choice for the top level domain (TLD) to construct domain names that are sure to be invalid from the Internet.
The mDNS network discovery protocol (Apple Bonjour / Apple Rendezvous, Avahi on Debian) uses "local" as the pseudo-top-level domain. Microsoft also seems to promote "local" for the TLD of local area network.
Other popular choices for the invalid TLD seem to be "localdomain
", "lan
", "localnet
", or "home
" according to my incoming mail analysis.
The hostname resolution is currently supported by the NSS (Name Service Switch) mechanism too. The flow of this resolution is:
/etc/nsswitch.conf
" file with stanza like "hosts: files dns
" dictates the hostname resolution order. (This replaces the old functionality of the "order
" stanza in "/etc/host.conf
".)
files
method is invoked first. If the hostname is found in the "/etc/hosts
" file, it returns all valid addresses for it and exits. (The "/etc/host.conf
" file contains "multi on
".)
dns
method is invoked. If the hostname is found by the query to the Internet Domain Name System (DNS) identified by the "/etc/resolv.conf
" file, it returns all valid addresses for it and exits.
The "/etc/hosts
" file associates IP addresses with hostnames:
127.0.0.1 localhost 127.0.1.1 <host_name>.<domain_name> <host_name> # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts
Here the <host_name> in this matches the own hostname defined in the "/etc/hostname
". The <domain_name> in this is the fully qualified domain name (FQDN) of this host.
![]() |
Tip |
---|---|
For the mobile PC without real FQDN, you may pick a TLD such as bogus " |
The "/etc/resolv.conf
" is a static file if the resolvconf
package is not installed. If installed, it is a symbolic link. Either way, it contains information that initialize the resolver routines. If the DNS is found at IP="192.168.11.1
", it contains:
nameserver 192.168.11.1
The resolvconf
package makes this "/etc/resolv.conf
" into a symbolic link and manages its contents by the hook scripts automatically.
The hostname resolution via Multicast DNS (using Zeroconf, aka Apple Bonjour / Apple Rendezvous) which effectively allows name resolution by common Unix/Linux programs in the ad-hoc mDNS domain "local
", can be provided by installing the libnss-mdns
package. The "/etc/nsswitch.conf
" file should have stanza like "hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4
" to enable this functionality.
The network interface name, e.g. eth0
, is assigned to each hardware in the Linux kernel through the user space configuration mechanism, udev
(see Section 3.5.10, “The udev system”), as it is found. The network interface name is referred as physical interface in ifup
(8) and interfaces
(5).
In order to ensure each network interface to be named persistently for each reboot using MAC address etc., there is a record file "/etc/udev/rules.d/70-persistent-net.rules
". This file is automatically generated by the "/lib/udev/write_net_rules
" program, probably run by the "persistent-net-generator.rules
" rules file. You can modify it to change naming rule.
![]() |
Caution |
---|---|
When editing the " |
Let us be reminded of the IPv4 32 bit address ranges in each class reserved for use on the local area networks (LANs) by rfc1918. These addresses are guaranteed not to conflict with any addresses on the Internet proper.
Table 5.1. List of network address ranges.
Class | network addresses | net mask | net mask /bits | # of subnets |
---|---|---|---|---|
A | 10.x.x.x | 255.0.0.0 | /8 | 1 |
B | 172.16.x.x — 172.31.x.x | 255.255.0.0 | /16 | 16 |
C | 192.168.0.x — 192.168.255.x | 255.255.255.0 | /24 | 256 |
![]() |
Note |
---|---|
If one of these addresses is assigned to a host, then that host must not access the Internet directly but must access it through a gateway that acts as a proxy for individual services or else does Network Address Translation(NAT). The broadband router usually performs NAT for the consumer LAN environment. |
There are 2 types of low level networking programs for Linux networking system (see Section 5.6.1, “Iproute2 commands”).
ifconfig
(8), …) are from the Linux NET-3 networking system. Most of these are obsolete now.
ip
(8), …) are the current Linux networking system.
Although these low level networking programs are powerful, they are cumbersome to use. So high level network configuration systems have been created.
The ifupdown
package is the de facto standard for such high level network configuration system on Debian. It enables you to bring up network simply by doing , e.g., "ifup eth0
". Its configuration file is the "/etc/network/interfaces
" file and its typical contents are:
auto lo iface lo inet loopback auto eth0 iface eth0 inet dhcp
The resolvconf
package was created to supplement ifupdown
system to support smooth reconfiguration of network address resolution by automating rewrite of resolver configuration file "/etc/resolv.conf
". Now, most Debian network configuration packages are modified to use resolvconf
package (see "/usr/share/doc/resolvconf/README.Debian
").
Helper scripts to the ifupdown
package such as ifplugd
, guessnet
, ifscheme
, etc. are created to automate dynamic configuration of network environment such as one for mobile PC on wired LAN. These are relatively difficult to use but play well with existing ifupdown
system.
Alternative high level network configuration systems, independent of ifupdown
system, such as network-manager
, wicd
, etc. are created to ease configuration of network environment even for mobile PC on wireless network. Since these are relatively new system and their integration to Debian system is in progress, you may still need to disable the corresponding network interface configuration manually in "/etc/network/interfaces
" to avoid conflicts between these and ifupdown
(see Section 5.5.2, “Automatic network configuration”).
Table 5.2. List of network configuration tools.
packages | popcon | size | type | function |
---|---|---|---|---|
ifupdown
|
V:63, I:99 | 228 | config::ifupdown | Standardized tool to bring up and down the network (Debian specific) |
ifplugd
|
V:0.4, I:0.9 | 332 | , , | Manage the wired network automatically |
ifupdown-extra
|
V:0.04, I:0.2 | 124 | , , |
Network testing script to enhance "ifupdown " package
|
ifmetric
|
V:0.02, I:0.09 | 100 | , , | Set routing metrics for a network interface. |
guessnet
|
V:0.09, I:0.4 | 492 | , , |
Mapping script to enhance "ifupdown " package via "/etc/network/interfaces " file
|
ifscheme
|
V:0.03, I:0.10 | 80 | , , |
Mapping scripts to enhance "ifupdown " package
|
ifupdown-scripts-zg2
|
V:0.00, I:0.05 | 220 | , , | Zugschlus' interface scripts for ifupdown's manual method |
network-manager
|
V:30, I:38 | 2180 | config::NetworkManager | NetworkManager (daemon): Manage the network automatically |
network-manager-gnome
|
V:22, I:35 | 3372 | , , | NetworkManager (GNOME frontend) |
network-manager-kde
|
V:2, I:4 | 3088 | , , | NetworkManager (KDE frontend) |
wicd
|
V:1.2, I:1.5 | 1852 | config::wicd | Wired and wireless network manager |
iptables
|
V:25, I:99 | 1368 | config::Netfilter | Administration tools for packet filtering and NAT |
iproute
|
V:38, I:75 | 1000 | config::iproute2 |
IPv6 and other advanced network configuration: ip (8), tc (8), etc.
|
ifrename
|
V:0.2, I:0.8 | 108 | , , |
Rename network interfaces based on various static criteria: ifrename (8)
|
ethtool
|
V:3, I:11 | 256 | , , | Display or change Ethernet device settings |
iputils-ping
|
V:36, I:99 | 124 | test::iproute2 | Tools to test network reachability of a remote host by hostname or IP address |
iputils-arping
|
V:1.1, I:17 | 72 | , , | Tools to test network reachability of a remote host specified by the ARP address |
iputils-tracepath
|
V:0.4, I:2 | 108 | , , | Tools to trace the network path to a remote host |
net-tools
|
V:80, I:99 | 1016 | config::net-tools |
The NET-3 networking toolkit (IPv4 network configuration): ifconfig (8) etc.
|
inetutils-ping
|
V:0.05, I:0.14 | 268 | test::net-tools | Tools to test network reachability of a remote host by hostname or IP address (legacy, GNU) |
arping
|
V:0.5, I:2 | 64 | , , | Tools to test network reachability of a remote host specified by the ARP address (legacy) |
traceroute
|
V:14, I:98 | 184 | , , | Tools to trace the network path to a remote host (legacy, console) |
dhcp3-client
|
V:50, I:93 | 608 | config::low-level | DHCP client |
wpasupplicant
|
V:12, I:45 | 960 | , , | Client support for WPA and WPA2 (IEEE 802.11i) |
wireless-tools
|
V:9, I:27 | 276 | , , | Tools for manipulating Linux Wireless Extensions |
ppp
|
V:7, I:26 | 1100 | , , |
PPP/PPPoE connection with chat
|
pppoeconf
|
V:0.5, I:4 | 200 | config::helper | Configuration helper for PPPoE connection |
pppconfig
|
V:0.4, I:3 | 900 | , , |
Configuration helper for PPP connection with chat
|
wvdial
|
V:0.6, I:2 | 352 | , , |
Configuration helper for PPP connection with wvdial and ppp
|
mtr-tiny
|
V:4, I:54 | 120 | test::low-level | Tools to trace the network path to a remote host (curses) |
mtr
|
V:0.5, I:2 | 176 | , , | Tools to trace the network path to a remote host (curses and GTK+) |
gnome-nettool
|
V:4, I:40 | 1766 | , , | Tools for common network information operations (GNOME) |
nmap
|
V:6, I:30 | 3768 | , , | Network mapper / port scanner (Nmap, console) |
zenmap
|
V:0.2, I:1.0 | 1232 | , , | Network mapper / port scanner (GTK+) |
knmap
|
V:0.12, I:0.7 | 1992 | , , | Network mapper / port scanner (KDE) |
tcpdump
|
V:3, I:22 | 796 | , , | Network traffic analyzer (Tcpdump, console) |
wireshark
|
V:2, I:10 | 1604 | , , | Network traffic analyzer (Wireshark, GTK+) |
tshark
|
V:0.5, I:3 | 284 | , , | Network traffic analyzer (console) |
nagios3
|
V:0.8, I:1.1 | 4152 | , , | Monitoring and management system for hosts, services and networks (Nagios) |
tcptrace
|
V:0.07, I:0.4 | 432 | , , |
Tool to produce a summarization of the connections from tcpdump output
|
snort
|
V:0.7, I:0.9 | 1100 | , , | Flexible network intrusion detection system (Snort) |
ntop
|
V:1.2, I:2 | 15600 | , , | Display network usage in web browser |
dnsutils
|
V:14, I:91 | 388 | , , |
Network clients provided with BIND: nslookup (8), nsupdate (8), dig (8)
|
dlint
|
V:0.5, I:7 | 96 | , , | Checks DNS zone information using nameserver lookups |
dnstracer
|
V:0.09, I:0.6 | 88 | , , | Tool to trace a chain of DNS servers to the source |
Although most hardware devices are supported by the Debian system, there are some network devices which require DSFG non-free external hardware drivers to support them. Please see Section 9.7.7, “Non-free hardware drivers”.
![]() |
Caution |
---|---|
The connection test method described in this section are meant for testing purposes. It is not meant to be used directly for the daily network connection. You are advised to use them via the |
The typical network connection method and connection path for a PC can be summarized as:
Table 5.3. List of network connection methods and connection paths.
PC | connection method | connection path |
---|---|---|
Serial port (ppp0 )
|
PPP | ⇔ modem ⇔ POTS ⇔ dial-up access point ⇔ ISP |
Ethernet port (eth0 )
|
PPPoE/DHCP/Static | ⇔ BB-modem ⇔ BB service ⇔ BB access point ⇔ ISP |
Ethernet port (eth0 )
|
DHCP/Static | ⇔ LAN ⇔ BB-router with network address translation (NAT) (⇔ BB-modem …) |
Here is the summary of configuration script for each connection method:
Table 5.4. List of network connection configurations.
connection method | configuration | backend package(s) |
---|---|---|
PPP |
pppconfig to create deterministic chat
|
pppconfig , ppp
|
PPP (alternative) |
wvdialconf to create heuristic chat
|
ppp , wvdial
|
PPPoE |
pppoeconf to create deterministic chat
|
pppoeconf , ppp
|
DHCP |
described in "/etc/dhcp3/dhclient.conf "
|
dhcp3-client
|
static IP (IPv4) |
described in "/etc/network/interfaces "
|
net-tools
|
static IP (IPv6) |
described in "/etc/network/interfaces "
|
iproute
|
The network connection acronyms mean:
Table 5.5. List of network connection acronyms.
acronym | meaning |
---|---|
POTS | The plain old telephone service |
BB | The broadband |
BB-service | E.g., the digital subscriber line (DSL), the cable TV, or the fiber to the premises (FTTP). |
BB-modem | E.g., the DSL modem, the cable modem, or the optical network terminal (ONT). |
LAN | The local area network |
WAN | The wide area network |
DHCP | The dynamic host configuration protocol |
PPP | The point-to-point protocol |
PPPoE | The point-to-point protocol over Ethernet |
ISP | The Internet service provider |
![]() |
Note |
---|---|
The WAN connection services via cable TV are generally served by DHCP or PPPoE. The ones by ADSL and FTTP are generally served by PPPoE. You have to consult your ISP for exact configuration requirements of the WAN connection. |
![]() |
Note |
---|---|
When BB-router is used to create home LAN environment, PCs on LAN are connected to the WAN via BB-router with network address translation (NAT). For such case, PC's network interfaces on the LAN are served by static IP or DHCP from the BB-router. BB-router must be configured to connect the WAN following the instruction by your ISP. |
The typical modern home and small business network, i.e. LAN, are connected to the WAN(Internet) using some consumer grade broadband router. The LAN behind this router is usually served by the dynamic host configuration protocol (DHCP) server running on the router.
Just install the dhcp3-client
package for the Ethernet served by the dynamic host configuration protocol (DHCP).
No special action is needed for the Ethernet served by the static IP.
The configuration script pppconfig
will configure the PPP connection interactively just by selecting:
The configuration files are:
Table 5.6. List of configuration files for the PPP connection with pppconfig.
file | function |
---|---|
/etc/ppp/peers/<isp_name>
|
The pppconfig generated configuration file for pppd specific to <isp_name>
|
/etc/chatscripts/<isp_name>
|
The pppconfig generated configuration file for chat specific to <isp_name>
|
/etc/ppp/options
|
The general execution parameter for pppd
|
/etc/ppp/pap-secret
|
Authentication data for the PAP (security risk) |
/etc/ppp/chap-secret
|
Authentication data for the CHAP (more secure) |
![]() |
Caution |
---|---|
The "<isp_name>" value of " |
You can test configuration using low level network configuration tools:
$ sudo pon <isp_name> ... $ sudo poff <isp_name>
See "/usr/share/doc/ppp/README.Debian.gz
" for more information.
A different approach to using pppd
(8) is to run it from wvdial
(1) which comes in the wvdial
package. Instead of pppd
running chat
(8) to dial in and negotiate the connection, wvdial
does the dialing and initial negotiating and then starts pppd
to do the rest.
The configuration script wvdialconf
will configure the PPP connection interactively just by selecting:
wvdial
succeeds in making the connection in most cases and maintains authentication data list automatically.
The configuration files are:
Table 5.7. List of configuration files for the PPP connection with wvdialconf.
file | function |
---|---|
/etc/ppp/peers/wvdial
|
The wvdialconf generated configuration file for pppd specific to wvdial
|
/etc/wvdial.conf
|
The wvdialconf generated configuration file
|
/etc/ppp/options
|
The general execution parameter for pppd
|
/etc/ppp/pap-secret
|
Authentication data for the PAP (security risk) |
/etc/ppp/chap-secret
|
Authentication data for the CHAP (more secure) |
You can test configuration using low level network configuration tools:
$ sudo wvdial ... $ sudo killall wvdial
See wvdial
(1) and wvdial.conf
(5) for more information.
When your ISP serves you with PPPoE connection and you decide to connect your PC directly to the WAN, the network of your PC must be configured with the PPPoE. The PPPoE stand for PPP over Ethernet. The configuration script pppoeconf
will configure the PPPoE connection interactively.
The configuration files are:
Table 5.8. List of configuration files for the PPPoE connection with pppoeconf.
file | function |
---|---|
/etc/ppp/peers/dsl-provider
|
The pppoeconf generated configuration file for pppd specific to pppoe
|
/etc/ppp/options
|
The general execution parameter for pppd
|
/etc/ppp/pap-secret
|
Authentication data for the PAP (security risk) |
/etc/ppp/chap-secret
|
Authentication data for the CHAP (more secure) |
You can test configuration using low level network configuration tools:
$ sudo /sbin/ifconfig eth0 up $ sudo pon dsl-provider ... $ sudo poff dsl-provider $ sudo /sbin/ifconfig eth0 down
See "/usr/share/doc/pppoeconf/README.Debian
" for more information.
The ifupdown
package provides the standardized framework for the high level network configuration in the Debian system. In this section, we learn the basic network configuration with ifupdown
with simplified introduction and many typical examples.
The ifupdown
package contains 2 commands: ifup
(8) and ifdown
(8). They offer high level network configuration dictated by the configuration file "/etc/network/interfaces".
Table 5.9. List of basic network configuration commands with ifupdown.
command | action |
---|---|
ifup eth0
|
To bring up a network interface eth0 with the configuration eth0 if "iface eth0 " stanza exists.
|
ifdown eth0
|
To bring down a network interface eth0 with the configuration eth0 if "iface eth0 " stanza exists.
|
![]() |
Warning |
---|---|
Do not use low level configuration tools such as |
![]() |
Note |
---|---|
There is no command |
The key syntax of "/etc/network/interfaces
" as explained in interfaces
(5) can be summarized as:
Table 5.10. List of stanzas in "/etc/network/interfaces
"
stanza | meaning |
---|---|
"auto <interface_name> "
|
To start interface <interface_name> upon start of the system. |
"allow-auto <interface_name> "
|
, , |
"allow-hotplug <interface_name> "
|
To start interface <interface_name> when the kernel detects a hotplug event from the interface. |
Lines started with "iface <config_name> … "
|
To define the network configuration <config_name>. |
Lines started with "mapping <interface_name_glob> "
|
To define mapping value of <config_name> for the matching <interface_name>. |
A line starting with a hash "# "
|
To be ignored as comments. (end-of-line comments are not supported) |
A line ending with a backslash "\ "
|
To extend the configuration to the next line. |
Lines started with iface
stanza has the following syntax:
iface <config_name> <address_family> <method_name> <option1> <value1> <option2> <value2> ...
For the basic configuration, the mapping
stanza is not used and you use the network interface name as the network configuration name (See Section 5.4.5, “The mapping stanza”).
![]() |
Warning |
---|---|
Do not define duplicates of the " |
The following configuration entry in the "/etc/network/interfaces
" file brings up the loopback network interface lo
upon booting the system (via auto
stanza).
auto lo iface lo inet loopback
This one always exists in the "/etc/network/interfaces
" file.
After prepairing the system by Section 5.2.1, “The DHCP connection with the Ethernet”, the network interface served by the DHCP is configured by creating the configuration entry in the "/etc/network/interfaces
" file as:
allow-hotplug eth0 iface eth0 inet dhcp hostname "mymachine"
When the Linux kernel detects the physical interface eth0
, the allow-hotplug
stanza will cause ifup
to bring up the interface and the iface
stanza will cause ifup
to use DHCP to configure the interface.
The network interface served by the static IP is configured by creating the configuration entry in the "/etc/network/interfaces
" file as, e.g.,:
allow-hotplug eth0 iface eth0 inet static address 192.168.11.100 netmask 255.255.255.0 broadcast 192.168.11.255 gateway 192.168.11.1 dns-domain lan dns-nameservers 192.168.11.1
When the Linux kernel detects the physical interface eth0
, the allow-hotplug
stanza will cause ifup
to bring up the interface and the iface
stanza will cause ifup
to use the static IP to configure the interface.
Here, I assumed:
192.168.11.0
- 192.168.11.255
192.168.11.1
192.168.11.100
resolvconf
package is installed.
lan
".
192.168.11.1
When the resolvconf
package is not installed, DNS related configuration needs to be done manually by editing the "/etc/resolv.conf
" as:
nameserver 192.168.11.1 domain lan
![]() |
Caution |
---|---|
The IP addresses used in the above example are not meant to be copied literally. You have to adjust IP numbers to your actual network configuration. |
The wireless LAN (WLAN for short) provides the fast wireless connectivity through the spread-spectrum communication of unlicensed radio bands based on the set of standards called IEEE 802.11.
The WLAN interfaces are almost like normal Ethernet interfaces but require some network ID and encryption key data to be provided when they are initialized. Their high level network tools are exactly the same as that of Ethernet interfaces except interface names are a bit different like eth1
, wlan0
, ath0
, wifi0
, … depending on the kernel drivers used.
![]() |
Tip |
---|---|
The |
Here are some keywords to remember for the WLAN:
Table 5.11. List of acronyms for WLAN.
acronym | full word | meaning |
---|---|---|
NWID | Network ID | The 16 bit network ID used by pre-802.11 WaveLAN network. Very much deprecated. |
(E)SSID | (Extended) Service Set Identifier | The network name of the Wireless Access Points (APs) interconnected to form an integrated 802.11 wireless LAN. Domain ID. |
WEP, (WEP2) | Wired Equivalent Privacy | The 1st generation 64-bit (128-bit) wireless encryption standard with 40-bit key. Deprecated. |
WPA | Wi-Fi Protected Access | The 2nd generation wireless encryption standard (most of 802.11i), compatible with WEP. |
WPA2 | Wi-Fi Protected Access 2 | The 3rd generation wireless encryption standard (full 802.11i), non-compatible with WEP. |
The actual choice of protocol is usually limited by the wireless router you deploy.
You need to install the wpasupplicant
package to support the WLAN with the new WPA/WPA2.
In case of the DHCP served IP on WLAN connection, the "/etc/network/interfaces
" file entry should be:
allow-hotplug ath0 iface ath0 inet dhcp wpa-ssid homezone # hexadecimal psk is encoded from a plaintext passphrase wpa-psk 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f
See more on "/usr/share/doc/wpasupplicant/README.modes.gz
".
You need to install the wireless-tools
package to support the WLAN with the old WEP. (Your consumer grade router may still be using this insecure infrastructure but this is better than nothing.)
![]() |
Caution |
---|---|
Please note that your network traffic on WLAN with WEP may be sniffed by others. |
In case of the DHCP served IP on WLAN connection, the "/etc/network/interfaces
" file entry should be:
allow-hotplug eth0 iface eth0 inet dhcp wireless-essid Home wireless-key1 0123-4567-89ab-cdef wireless-key2 12345678 wireless-key3 s:password wireless-defaultkey 2 wireless-keymode open
See more on "/usr/share/doc/wireless-tools/README.Debian
".
You need to configure the PPP connection first as described before (see Section 5.2.3, “The PPP connection with pppconfig”). Then, add the "/etc/network/interfaces
" file entry for the primary PPP device ppp0
as:
iface ppp0 inet ppp provider <isp_name>
You need to configure the alternative PPP connection with wvdial
first as described before (see Section 5.2.4, “The alternative PPP connection with wvdialconf”). Then, add the "/etc/network/interfaces
" file entry for the primary PPP device ppp0
as:
iface ppp0 inet wvdial
For PC connected directly to the WAN served by the PPPoE, you need to configure system with the PPPoE connection as described before (see Section 5.2.5, “The PPPoE connection with pppoeconf”). Then, add the "/etc/network/interfaces
" file entry for the primary PPPoE device eth0
as:
allow-hotplug eth0 iface eth0 inet manual pre-up /sbin/ifconfig eth0 up up ifup ppp0=dsl down ifdown ppp0=dsl post-down /sbin/ifconfig eth0 down # The following is used internally only iface dsl inet ppp provider dsl-provider
The "/etc/network/run/ifstate
" file stores the intended network configuration states for all the currently active network interfaces managed by the ifupdown
package are listed. Unfortunately, even if the ifupdown
system fails to bring up the interface as intended, the "/etc/network/run/ifstate
" file lists it active.
Unless the output of the ifconfig
(8) command for an interface does not have a line like following example, it can not be used as a part of IPV4 network:
inet addr:192.168.11.2 Bcast:192.168.11.255 Mask:255.255.255.0
![]() |
Note |
---|---|
For the Ethernet device connected to the PPPoE, the output of the |
When you try to reconfigure the interface, e.g. eth0
, you must disable it first with the "sudo ifdown eth0
" command. This will remove the entry of eth0
from the "/etc/network/run/ifstate
" file. (This may result in some error message if eth0
is not active or it is configured improperly previously. So far, it seems to be safe to do this for the simple single user work station at any time.)
You are now free to rewrite the "/etc/network/interfaces
" contents as needed to reconfigure the network interface, eth0
.
Then, you can reactivate eth0
with the "sudo ifup eth0
" command.
![]() |
Tip |
---|---|
You can (re)initialize the network interface simply by " |
The ifupdown-extra
package provides the easy network connection test for use with the ifupdown
package by:
network-test
(1) command from the shell, and
ifup
command execution.
The network-test
command frees you from the execution of cumbersome low level commands to analyze the network problem.
The automatic scripts are installed in "/etc/network/*/
" and:
/etc/network/routes
" definition,
/var/log/syslog
" file.
This syslog record is quite useful for administration of the network problem on the remote system.
![]() |
Tip |
---|---|
The automatic behavior of the |
The functionality of the ifupdown
package can be improved beyond what was described in Section 5.3, “The basic network configuration with ifupdown” with the advanced knowledge.
The functionalities described here are completely optional. I, being lazy and minimalist, rarely bother to use these.
![]() |
Caution |
---|---|
If you could not set up network connection by information in Section 5.3, “The basic network configuration with ifupdown”, you will make situation worse by using information below. |
The ifplugd
package is older automatic network configuration tool which can manage only Ethernet connections. This solves unplugged/replugged Ethernet cable issues for mobile PC etc.. If you have NetworkManager or Wicd (see Section 5.5.2, “Automatic network configuration”) installled, you do not need this package.
This package runs daemon and replaces auto or allow-hotplug functionalities (see Table 5.10, “List of stanzas in "/etc/network/interfaces
"”) and starts interfaces upon their connection to the network.
Here is how to use the ifplugd
package for the internal Ethernet port, e.g. eth0
:
/etc/network/interfaces
": "auto eth0
" or "allow-hotplug eth0
",
/etc/network/interfaces
": "iface eth0 inet …
" and "mapping …
",
ifplugd
package,
sudo dpkg-reconfigure ifplugd
", and
eth0
as the "static interfaces to be watched by ifplugd".
Now, the network reconfiguration works as you desire:
Upon power-on or upon hardware discovery, the interface is not brought up by itself.
![]() |
Tip |
---|---|
The arguments for the |
The ifmeric
package enables us to manipulate metrics of routes a posteriori even for DHCP.
The following will set the eth0
interface to be preferred over the wlan0
interface:
ifmetric
package, and
metric 0
" just below the "iface eth0 inet dhcp
" line.
metric 1
" just below the "iface wlan0 inet dhcp
" line.
The metric 0 means the highest priority route and is the default one. The larger metric value means lower priority routes. The IP address of the active interface with the lowest metric value becomes the originating one. See ifmetric
(8).
A single physical Ethernet interface can be configured as multiple virtual interfaces with different IP addresses. Usually the purpose is to connect an interface to several IP subnetworks. For example, IP address based virtual web hosting by a single network interface is one such application.
For example, let's suppose that
192.168.0.x/24
,
eth0
for the Internet, and
192.168.0.1
with the virtual interface eth0:0
for the LAN,
then following stanzas in "/etc/network/interfaces
" will configure your network:
iface eth0 inet dhcp metric 0 iface eth0:0 inet static address 192.168.0.1 netmask 255.255.255.0 network 192.168.0.0 broadcast 192.168.0.255 metric 1
![]() |
Caution |
---|---|
Although this configuration example with network address translation (NAT) using netfilter/iptables (see Section 5.8, “Netfilter”) can provide cheap router for the LAN with only single interface, there is no real firewall capability with such set up. You should use 2 physical interfaces with NAT to secure the local network from Internet. |
The ifupdown
package offers advanced network configuration using the network configuration name and the network interface name. I use slightly different terminology from one used in ifup
(8) and interfaces
(5).
Table 5.12. List of terminology for network devices.
manpage terminology | my terminology | explanation | examples in the following text |
---|---|---|---|
physical interface name | network interface name |
A name given by the Linux kernel (using udev mechanism).
|
lo , eth0 , <interface_name>
|
logical interface name | network configuration name |
A name token following iface in the "/etc/network/interfaces ".
|
config1 , config2 , <config_name>
|
Basic network configuration commands in Section 5.3.1, “The command syntax simplified” require the network configuration name token of the iface
stanza to match the network interface name in the "/etc/network/interfaces
".
Advanced network configuration commands enables separation of the network configuration name and the network interface name in the "/etc/network/interfaces
":
Table 5.13. List of advanced network configuration commands with ifupdown.
command | action |
---|---|
ifup eth0=config1
|
To bring up a network interface eth0 with the configuration config1 .
|
ifdown eth0=config1
|
To bring down a network interface eth0 with the configuration config1 .
|
ifup eth0
|
To bring up a network interface eth0 with the configuration selected by mapping stanza.
|
ifdown eth0
|
To bring down a network interface eth0 with the configuration selected by mapping stanza.
|
We skipped explaining the mapping
stanza in the "/etc/network/interfaces
" in Section 5.3.2, “The basic syntax of "/etc/network/interfaces"” to avoid complication. This stanza has the following syntax:
mapping <interface_name_glob> script <script_name> map <script_input1> map <script_input2> map ...
This provides advanced feature to the "/etc/network/interfaces
" file by automating the choice of the configuration with the mapping script specified by <script_name>
.
When the "<interface_name_glob>
" matches "eth0
", the execution of
$ sudo ifup eth0
will produce the execution of:
$ sudo ifup eth0=$(echo -e '<script_input1> \n <script_input2> \n ...' | <script_name> eth0)
to configure eth0
automatically. Here, lines with "map
" are optional and can be repeated.
![]() |
Note |
---|---|
The glob for |
Here is how to switch manually among several network configurations without rewriting the "/etc/network/interfaces
" file as in Section 5.3.13, “The basic network reconfiguration” .
For all the network configuration you need to access, you create a single "/etc/network/interfaces
" file, e.g,:
auto lo iface lo inet loopback iface config1 inet dhcp hostname "mymachine" iface config2 inet static address 192.168.11.100 netmask 255.255.255.0 broadcast 192.168.11.255 gateway 192.168.11.1 dns-domain lan dns-nameservers 192.168.11.1 iface pppoe inet manual pre-up /sbin/ifconfig eth0 up up ifup ppp0=dsl down ifdown ppp0=dsl post-down /sbin/ifconfig eth0 down # The following is used internally only iface dsl inet ppp provider dsl-provider iface pots inet ppp provider provider
Please note the network configuration name which is the token after iface
does not use the token for the network interface name. Also, there are no auto
stanza nor allow-hotplug
stanza to start the network interface eth0
automatically upon events.
Now you are ready to switch the network configuration.
Let's move your PC to a LAN served by the DHCP. You bring up the network interface (the physical interface) eth0
by assigning the network configuration name (the logical interface name) config1
to it:
$ sudo ifup eth0=config1 Password: ...
The interface eth0
is up, configured by DHCP and connected to LAN.
$ sudo ifdown eth0=config1 ...
The interface eth0
is down and disconnected from LAN.
Let's move your PC to a LAN served by the static IP. You bring up the network interface eth0
by assigning the network configuration name config2
to it:
$ sudo ifup eth0=config2 ...
The interface eth0
is up, configured with static IP and connected to LAN. The additional parameters given as dns-*
configures "/etc/resolv.conf
" contents. This "/etc/resolv.conf
" is better manged if the resolvconf
package is installed.
$ sudo ifdown eth0=config2 ...
The interface eth0
is down and disconnected from LAN, again.
Let's move your PC to a port on BB-modem connected to the PPPoE served service. You bring up the network interface eth0
by assigning the network configuration name pppoe
to it:
$ sudo ifup eth0=pppoe ...
The interface eth0
is up, configured with PPPoE connection directly to the ISP.
$ sudo ifdown eth0=pppoe ...
The interface eth0
is down and disconnected, again.
Let's move your PC to a location without LAN or BB-modem but with POTS and modem. You bring up the network interface ppp0
by assigning the network configuration name pots
to it:
$ sudo ifup ppp0=pots ...
The interface ppp0
is up and connected to the Internet with PPP.
$ sudo ifdown ppp0=pots ...
The interface ppp0
is down and disconnected from the Internet.
You should check the "/etc/network/run/ifstate
" file for the current network configuration state of the ifupdown
system.
![]() |
Warning |
---|---|
You may need to adjust numbers at the end of |
The ifupdown
system automatically runs scripts installed in "/etc/network/*/
" while exporting environment variables to scripts:
Table 5.14. List of environment variables passed by the ifupdown system
environment variable | value passed |
---|---|
"$IFACE "
|
physical name (interface name) of the interface being processed. |
"$LOGICAL "
|
logical name (configuration name) of the interface being processed. |
"$ADDRFAM "
|
<address_family> of the interface. |
"$METHOD "
|
<method_name> of the interface. (e.g., "static") |
"$MODE "
|
"start" if run from ifup , "stop" if run from ifdown .
|
"$PHASE "
|
as per "$MODE ", but with finer granularity, distinguishing the pre-up , post-up , pre-down and post-down phases.
|
"$VERBOSITY "
|
indicates whether "--verbose " was used; set to 1 if so, 0 if not.
|
"$PATH "
|
the command search path: "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin "
|
"$IF_<OPTION> "
|
the value for the corresponding option under the iface stanza.
|
Here, each environment variable, "$IF_<OPTION>
", is created from the name for the corresponding option such as <option1> and <option2> by prepending "$IF_
", converting the case to the upper case, replacing hyphens to underscores, and discarding non-alphanumeric characters.
![]() |
Tip |
---|---|
See Section 5.3.2, “The basic syntax of "/etc/network/interfaces"” for <address_family>, <method_name>, <option1> and <option2>. |
The ifupdown-extra
package (see Section 5.3.14, “The ifupdown-extra package”) uses these environment variables to extend the functionality of the ifupdown
package. The ifmetric
package (see Section 5.4.2, “The ifmetric package”) installs the "/etc/network/if-up.d/ifmetric
" script which sets the metric via the "$IF_METRIC
" variable. The guessnet
package (see Section 5.4.8, “Mapping with guessnet”), which provides simple and powerful framework for the auto-selection of the network configuration via the mapping mechanism, also uses these.
![]() |
Note |
---|---|
For more specific examples of custom network configuration scripts using these environment variables, you should check example scripts in " |
Instead of manually choosing configuration as described in Section 5.4.6, “The manually switchable network configuration”, you can use the mapping mechanism described in Section 5.4.5, “The mapping stanza” to select network configuration automatically with custom scripts.
The guessnet-ifupdown
(8) command provided by the guessnet
package is designed to be used as a mapping script and provides powerful framework to enhance the ifupdown
system.
guessnet
options for each network configuration under iface
stanza.
iface
with first non-ERROR result as the network configuration.
This dual usage of the "/etc/network/interfaces
" file by the mapping script, guessnet-ifupdown
, and the original network configuration infrastructure, ifupdown
, does not cause negative impacts since guessnet
options only export extra environment variables to scripts run by the ifupdown
system. See details in guessnet-ifupdown
(8).
![]() |
Note |
---|---|
When multiple |
The capability of default GUI network configuration tools for each desktop environments such as GNOME tends to be limited to basic configurations such as static IP or DHCP. They actually overwrite contents of "/etc/network/interfaces
" file behind you. Please check how they change "/etc/network/interfaces
" file by yourself.
![]() |
Caution |
---|---|
They may not understand complicated advanced configuration done manually in " |
There are independent automatic network configuration tools, such as NetworkManager (NM) (network-manager
and associated packages) and Wicd (wicd
package) which manage network connection via daemon independen of the ifupdown
package. They allow easy management of wireless connections. These come with its own nice GUI user interfaces.
![]() |
Warning |
---|---|
Do not use these automatic network configuration tools for servers. These are aimed primarily for mobile desktop users on laptops. |
![]() |
Warning |
---|---|
These automatic network configuration tools are moving targets and documentation here is likely to be incorrect for |
![]() |
Caution |
---|---|
These automatic network configuration tools may not be compatible with esoteric configurations of |
The configuration of NM is described in "/usr/share/doc/network-manager/README.Debian
". Essentially:
Make desktop user, e.g. foo
, belong to group "netdev
" by:
$ sudo adduser foo netdev
Keep configuration of "/etc/network/interfaces
" as simple as:
auto lo iface lo inet loopback auto eth0 iface eth0 inet dhcp
Restart NM by:
$ sudo /etc/init.d/network-manager restart
![]() |
Note |
---|---|
Only interfaces which are not listed in " |
The configuration of Wicd is described in "/usr/share/doc/wicd/README.Debian
". Essentially:
Make configuration in "/etc/network/interfaces
" only as:
auto lo iface lo inet loopback
Restart Wicd.
$ sudo /etc/init.d/wicd restart
Iproute2 commands offer complete low-level network configuration capabilities. Here is a translation table from obsolete net-tools commands to new iproute2 etc. commands.
Table 5.15. Translation table from obsolete net-tools
commands to new iproute2
commands.
obsolete net-tools | new iproute2 etc. | manipulation |
---|---|---|
ifconfig (8)
|
ip addr
|
protocol (IP or IPv6) address on a device. |
route (8)
|
ip route
|
routing table entry. |
arp (8)
|
ip neigh
|
ARP or NDISC cache entry. |
ipmaddr
|
ip maddr
|
multicast address. |
iptunnel
|
ip tunnel
|
tunnel over IP. |
nameif (8)
|
ifrename (8)
|
name network interfaces based on MAC addresses. |
mii-tool (8)
|
ethtool (8)
|
Ethernet device settings. |
See ip
(8) and IPROUTE2 Utility Suite Howto.
You may use low level network commands as follows safely since they do not change network configuration:
Table 5.16. List of low level network commands.
command | description |
---|---|
ifconfig
|
displays the link and address status of active interfaces |
ip addr show
|
displays the link and address status of active interfaces |
route -n
|
displays all the routing table in numerical addresses |
ip route show
|
displays all the routing table in numerical addresses |
arp
|
displays the current content of the ARP cache tables |
ip neigh
|
displays the current content of the ARP cache tables |
plog
|
display ppp daemon log |
ping yahoo.com
|
check Internet connection to "yahoo.com "
|
whois yahoo.com
|
check who registered "yahoo.com " in the domains database
|
traceroute yahoo.com
|
trace Internet connection to "yahoo.com "
|
tracepath yahoo.com
|
trace Internet connection to "yahoo.com "
|
mtr yahoo.com
|
trace Internet connection to "yahoo.com " (repeatedly)
|
dig [@dns-server.com] example.com [{a|mx|any}]
|
check DNS records of "example.com " by "dns-server.com " for a "a ", "mx ", or "any " record
|
iptables -L -n
|
check packet filter |
netstat -a
|
find all open ports |
netstat -l --inet
|
find listening ports |
netstat -ln --tcp
|
find listening TCP ports (numeric) |
dlint example.com
|
check DNS zone information of "examle.org "
|
![]() |
Tip |
---|---|
Some of these low level network configuration tools reside in " |
Generic network optimization is beyond the scope of this documentation. I will touch only subjects pertinent to the consumer grade connection.
Table 5.17. List of network optimization tools.
packages | popcon | size | description |
---|---|---|---|
iftop
|
V:1.0, I:6 | 108 | displays bandwidth usage information on an network interface |
iperf
|
V:0.4, I:2 | 208 | Internet Protocol bandwidth measuring tool |
apt-spy
|
V:0.2, I:1.6 | 204 |
writes a "/etc/apt/sources.list " file based on bandwidth tests
|
ifstat
|
V:0.2, I:1.0 | 88 | InterFace STATistics Monitoring |
bmon
|
V:0.2, I:0.8 | 188 | portable bandwidth monitor and rate estimator |
ethstatus
|
V:0.12, I:0.8 | 84 | script that quickly measures network device throughput |
bing
|
V:0.11, I:0.7 | 96 | Empirical stochastic bandwidth tester |
bwm-ng
|
V:0.2, I:0.9 | 152 | small and simple console-based bandwidth monitor |
ethstats
|
V:0.06, I:0.3 | 52 | console-based Ethernet statistics monitor |
ipfm
|
V:0.04, I:0.15 | 156 | a bandwidth analysis tool |
The Maximum Transmission Unit (MTU) value can be determined experimentally with ping
(8) with "-M do
" option which sends ICMP packets with data size starting from 1500 (with offset of 28 bytes for the IP+ICMP header) and finding the largest size without IP fragmentation. For example:
$ ping -c 1 -s $((1500-28)) -M do www.debian.org PING www.debian.org (194.109.137.218) 1472(1500) bytes of data. From 192.168.11.2 icmp_seq=1 Frag needed and DF set (mtu = 1454) --- www.debian.org ping statistics --- 0 packets transmitted, 0 received, +1 errors
ping
(8) command succeed
This process is Path MTU (PMTU) discovery (RFC1191) and the tracepath
(8) command can automate this.
![]() |
Tip |
---|---|
The above example with PMTU value of 1454 is for my previous FTTP provider which used Asynchronous Transfer Mode (ATM) as its backbone network and served its clients with the PPPoE. The actual PMTU value depends on your environment, e.g., 1500 for the my new FTTP provider. |
Table 5.18. Basic guide lines of the optimal MTU value
network environment | MTU | rationale |
---|---|---|
Dial-up link (IP: PPP) | 576 | standard |
Ethernet link (IP: DHCP or fixed) | 1500 | standard and default |
Ethernet link (IP: PPPoE) | 1492 (=1500-8) | 2 bytes for PPP header and 6 bytes for PPPoE header |
Ethernet link (ISP's backbone: ATM, IP: DHCP or fixed) | 1462 (=48*31-18-8) | author's speculation: 18 for Ethernet header, 8 for SAR trailer. |
Ethernet link (ISP's backbone: ATM, IP: PPPoE) | 1454 (=48*31-8-18-8) | see "Optimal MTU configuration for PPPoE ADSL Connections" for rationale. |
In addtion to these basic guide lines, you should know:
Here are examples for setting the MTU value from its default 1500 to 1454.
For the DHCP (see Section 5.3.4, “The network interface served by the DHCP”), you can replace pertinent iface
stanza lines in the "/etc/network/interfaces
" with, e.g.,:
iface eth0 inet dhcp hostname "mymachine" pre-up /sbin/ifconfig $IFACE mtu 1454
For the static IP (see Section 5.3.5, “The network interface with the static IP”), you can replace pertinent 'iface
' stanza lines in the "/etc/network/interfaces
" with, e.g.,:
iface eth0 inet static address 192.168.11.100 netmask 255.255.255.0 broadcast 192.168.11.255 gateway 192.168.11.1 mtu 1454 dns-domain lan dns-nameservers 192.168.11.1
For the direct PPPoE (see Section 5.2.5, “The PPPoE connection with pppoeconf”), you can replace pertinent "mtu
" line in the "/etc/ppp/peers/dsl-provider
" with:
mtu 1454
The maximum segment size (MSS) is used as an alternative measure of packet size. The relationship between MSS and MTU are:
![]() |
Note |
---|---|
The |
The TCP throughput can be maximized by adjusting TCP buffer size parameters as described in "TCP Tuning Guide" and "TCP tuning" for the modern high-bandwidth and high-latency WAN. So far, the current Debian default settings serve well even for my LAN connected by the fast 1G bps FTTP service.
Netfilter provides infrastructure for stateful firewall and network address translation (NAT) with Linux kernel modules (see Section 3.5.11, “The kernel module initialization”).
Table 5.19. List of firewall tools.
packages | popcon | size | description |
---|---|---|---|
iptables
|
V:25, I:99 | 1368 | administration tools for netfilter |
iptstate
|
V:0.15, I:0.9 | 156 |
Tool to continuously monitor netfilter state. (similar to top (1))
|
shorewall-perl
|
V:0.16, I:0.4 | 608 |
Shoreline Firewall, netfilter configuration file generator (Perl-based, recommended for lenny )
|
shorewall-shell
|
I:1.6 | 348 |
Shoreline Firewall, netfilter configuration file generator (shell-based, alternative for lenny )
|
ipmasq
|
V:0.2, I:0.5 | 612 | Simple set of init script to configure netfilter (old) |
Main user space program of netfilter is iptables
(8). You can manually configure netfilter interactively from shell, save its state with iptables-save
(8), and restore it via init script with iptables-restore
(8) upon system reboot.
Configuration helper scripts such as shorewall ease this process.
See documentation at http://www.netfilter.org/documentation/ (or in "/usr/share/doc/iptables/html/
"):
![]() |
Tip |
---|---|
Although these were written for Linux 2.4, both |
After establishing network connectivity (see Chapter 5, Network setup), you can run verious network applications.
There are many web browser packages to access remote contents with Hypertext Transfer Protocol (HTTP).
Table 6.1. List of web browsers.
package | popcon | size | description |
---|---|---|---|
iceweasel
|
V:35, I:56 | 3908 | Web browser (X) (unbranded Mozilla Firefox, ) |
iceape-browser
|
V:2, I:4 | NOT_FOUND | Web browser (X) (unbranded Mozilla browser, removed due to security concerns bug#505565) |
epiphany-browser
|
V:8, I:41 | 32 | Web browser (X) (GNOME HIG compliant browser, Epiphany) |
galeon
|
V:1.3, I:2 | 1748 | Web browser (X) (GNOME browser, Galeon was superseded by Epiphany) |
konqueror
|
V:11, I:20 | 3652 | Web browser (X) (KDE browser, Konqueror) |
w3m
|
V:23, I:85 | 1968 | Web browser (text) (w3m) |
lynx
|
V:2, I:25 | 48 | , , |
elinks
|
V:2, I:6 | 1452 | , , |
links
|
V:3, I:9 | 1372 | , , |
links2
|
V:1.0, I:4 | 3280 | , , |
You may be able to use following special URL strings for some browsers to confirm their settings.
about:
"
about:config
"
about:plugins
"
Debian offers many free browser plugin packages in the main component which can handle not only Java (software platform) and Flash but also MPEG, MPEG2, MPEG4, DivX, Windows Media Video (.wmv), QuickTime (.mov), MP3 (.mp3), Ogg/Vorbis files, DVDs, VCDs, etc. Debian also offers helper programs to install non-free browser plugin packages as contrib or non-free components.
Table 6.2. List of browser plugin packages.
package | popcon | size | component | description |
---|---|---|---|---|
icedtea-gcjwebplugin
|
V:0.6, I:0.8 | 204 | main | Java plugin using Hotspot JIT |
sun-java6-plugin
|
I:9 | 52 | non-free | Java plugin for Sun's Java SE 6 (i386 only) |
swfdec-mozilla
|
V:11, I:23 | 244 | main | Flash plugin based on libswfdec |
mozilla-plugin-gnash
|
V:0.5, I:1.8 | 108 | main | Flash plugin based on Gnash |
flashplugin-nonfree
|
V:1.4, I:10 | 128 | contrib | Flash plugin helper to install Adobe Flash Player (i386, amd64 only) |
mozilla-bonobo
|
V:0.16, I:0.4 | 168 | main | Mozilla plugin support for GNOME Bonobo components |
mozilla-plugin-vlc
|
V:3, I:5 | 160 | main | Multimedia plugin based on VLC media player |
totem-mozilla
|
V:21, I:41 | 268 | main | Multimedia plugin based on GNOME's Totem media player |
gecko-mediaplayer
|
V:0.19, I:0.2 | 680 | main | Multimedia plugin based on (GNOME) MPlayer |
nspluginwrapper
|
V:1.9, I:3 | 472 | contrib | A wrapper to run i386 Netscape plugins on amd64 architecture |
![]() |
Tip |
---|---|
Although use of above Debian packages are much easier, browser plugins can be still manually enabled by installing "*.so" into plugin directories (e.g., " |
Some web sites refuse to be connected based on the user-agent string of your browser. You can work around this situation by spoofing the user-agent string. For exaple, you can do this by adding:
user_pref{"general.useragent.override","Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)"};
into user configuration files such as "~/.gnome2/epiphany/mozilla/epiphany/user.js
" or "~/.mozilla/firefox/*.default/user.js
". Alternatively, you can add and reset this variable by typing "about:config
" into URL and right clicking its display contents.
![]() |
Caution |
---|---|
Spoofed user-agent string may cause bad side effects with Java. |
![]() |
Caution |
---|---|
If you are to set up the mail server to exchange mail directly with the Internet, you should be better than reading this elementary document. |
In order to contain spam (unwanted and unsolicited e-mail) problems, many ISPs which provide consumer grade Internet connection are implementing counter measures:
When configuring your mail system or resolving mail delivery problems, you must consider these new limitations.
In light of these hostile Internet situation and limitations, some independent Internet mail ISPs such as Yahoo.com and Gmail.com offer the secure mail service which can be connected from anywhere on the Internet using Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL) :
![]() |
Caution |
---|---|
It is not realistic to run SMTP server on consumer grade network to send mail directly to the remote host reliably. They are very likely to be rejected. You must use some smarthost services offered by your connection ISP or independent mail ISPs. For the simplicity, I will assume that the smarthost is located at " |
Table 6.3. List of popular mail system for workstation.
package | popcon | size | function |
---|---|---|---|
exim4-daemon-light
|
V:61, I:67 | 928 | Exim4 mail transport agent (MTA: Debian default) |
exim4-base
|
V:63, I:69 | 1660 | Exim4 documentation (text) and common files |
exim4-doc-html
|
I:0.8 | 5756 | Exim4 documentation (html) |
exim4-doc-info
|
I:0.4 | 596 | Exim4 documentation (info) |
postfix
|
V:16, I:18 | 3436 | Postfix mail transport agent (MTA: alternative) |
postfix-doc
|
I:2 | 3332 | Postfix documentation (html+text) |
sasl2-bin
|
V:1.9, I:5 | 448 | Cyrus SASL API implementation (supplement postfix for SMTP-AUTH) |
cyrus-sasl2-doc
|
I:3 | 284 | Cyrus SASL - documentation |
fetchmail
|
V:2, I:6 | 1812 | Remote mail retrieval and forwarding utility |
procmail
|
V:18, I:86 | 360 | Mail filter utility |
mutt
|
V:22, I:83 | 5772 |
Mail user agent (MUA) to read/write the mail usually used with vim
|
The choice between exim4-*
and postfix
packages is really up to you.
Although the popcon vote count of exim4 looks several times popular than that of postfix, this does not mean postfix is not popular with Debian developers. The Debian server system uses both exim4 and postfix. The mail header analysis of mailing list postings from prominent Debian developers also indicate both of these MTAs are as popular.
The exim4-*
packages are known to have very small memory consumption and very flexible for its configuration. The postfix
package is known to be compact, fast, simple, and secure. Both come with ample documentation and are as good in quality and license.
The most simple mail configuration is that the mail is sent to the ISP's smarthost and received from ISP's POP3 server by the MUA itself. This type of configuration is popular with full featured GUI based mail user agent (MUA) such as icedove
(1), evolution
(1), etc.. If you need to filter mail by their types, you use MUA's filtering function. For this case, the local mail transport agent (MTA) need to do local delivery only.
The alternative mail configuration is that the mail is sent via local MTA to the ISP's smarthost and received from ISP's POP3 by fetchmail
(1) to the local mailbox. If you need to filter mail by their types, you use procmail
(1) to filter mail into separate mailboxes. This type of configuration is popular with simple console based MUA such as mutt
(1), gnus
(1), etc., although this is possible with any MUAs. For this case, the local MTA need to do both smarthost delivery and local delivery.
For Internet via smarthost, you (re)configure exim4-*
packages as follows:
$ sudo /etc/init.d/exim4 stop $ sudo dpkg-reconfigure exim4-conf
Reply to "Keep number of DNS-queries minimal (Dial-on-Demand)?" as:
$ sudo vim /etc/exim4/passwd.client
$ cat /etc/exim4/passwd.client ^smtp.*\.hostname\.dom:username@hostname.dom:password $ sudo /etc/init.d/exim4 start
The host name in "/etc/exim4/passwd.client
" should not be the alias. You check the real host name with:
$ host smtp.hostname.dom smtp.hostname.dom is an alias for smtp99.hostname.dom. smtp99.hostname.dom has address 123.234.123.89
I use regex in "/etc/exim4/passwd.client
" to work around the alias issue so even if the ISP moves host pointed by the alias, SMTP AUTH will likely be working.
![]() |
Caution |
---|---|
You must execute |
![]() |
Caution |
---|---|
Starting |
![]() |
Note |
---|---|
Please read the official guide at: " |
![]() |
Tip |
---|---|
Local customization file " |
For Internet via smarthost, you should first read postfix documentation and key manual pages:
Table 6.4. List of important postfix manual pages
command | function |
---|---|
postfix (1)
|
Postfix control program |
postconf (1)
|
Postfix configuration utility |
postconf (5)
|
Postfix configuration parameters |
postmap (1)
|
Postfix lookup table maintenance |
postalias (1)
|
Postfix alias database maintenance |
You (re)configure postfix
and sasl2-bin
packages as follows:
$ sudo /etc/init.d/postfix stop $ sudo dpkg-reconfigure postfix
[smtp.hostname.dom]:587
"
$ sudo postconf -e 'smtp_sender_dependent_authentication = yes' $ sudo postconf -e 'smtp_sasl_auth_enable = yes' $ sudo postconf -e 'smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd' $ sudo postconf -e 'smtp_sasl_type = cyrus' $ sudo vim /etc/postfix/sasl_passwd
$ cat /etc/postfix/sasl_passwd [smtp.hostname.dom]:587 username:password $ sudo postmap hush:/etc/postfix/sasl_passwd $ sudo /etc/init.d/postfix start
Here the use of "[
" and "]
" in the dpkg-reconfigure dialogue and "/etc/postfix/sasl_passwd
" ensures not to check MX record but directly use exact hostname specified. Read more for "Enabling SASL authentication in the Postfix SMTP client" in "usr/share/doc/postfix/html/SASL_README.html
".
There are a few mail address configuration files for mail transport, delivery and user agents.
Table 6.5. List of mail address related configuration files.
file | function | application |
---|---|---|
/etc/mailname
|
default host name for (outgoing) mail |
Debian specific, mailname (5)
|
/etc/email-addresses
|
host name spoofing for outgoing mail |
exim (8) specific, exim4-config_files (5)
|
/etc/postfix/generic
|
host name spoofing for outgoing mail |
postfix (1) specific, activated after postmap (1) command execution.
|
/etc/aliases
|
account name alias for incoming mail |
general, activated after newaliases (1) command execution.
|
The mailname in the "/etc/mailname
" file is usually a fully qualified domain name (FQDN) that resolves to one of the host's IP addresses. The mobile workstation which does not have a hostname with resolvable IP address, set this mailname to the value of "hostname -f
". (This is safe choice and works for both exim4-*
and postfix
.)
![]() |
Tip |
---|---|
The contents of " |
When setting the mailname to "hostname -f
", the spoofing of the source mail address via MTA can be realized by:
/etc/email-addresses
" file for exim4
(8) as explained in the exim4-config_files
(5), and
/etc/postfix/generic
" file for postfix
(1) as explained in the generic
(5).
For postfix
, the following extra steps are needed:
# postmap hash:/etc/postfix/generic # postconf -e 'smtp_generic_maps = hash:/etc/postfix/generic' # postfix reload
You check filters using:
exim
(8) with -brw, -bf, -bF, -bV, …
options.
postmap
(1) with -q
option.
![]() |
Tip |
---|---|
Exim comes with several utility programs such as |
There are several basic MTA operations. Some may be performed via sendmail
(1) compatibility interface.
Table 6.6. List of basic MTA operation.
exim command | postfix command | description |
---|---|---|
sendmail
|
sendmail
|
Read mail from standard input and arrange for delivery. (-bm )
|
mailq
|
mailq
|
List the mail queue with status and queue ID. (-bp )
|
newaliases
|
newaliases
|
Initialize alias database. (-I )
|
exim4 -q
|
postqueue -f
|
flush waiting mail (-q )
|
exim4 -qf
|
postsuper -r ALL deferred; postqueue -f
|
flush all mail |
exim4 -qff
|
postsuper -r ALL; postqueue -f
|
flush even frozen mail |
exim4 -Mg queue_id
|
postsuper -h queue_id
|
freeze one message by its queue ID |
exim4 -Mrm queue_id
|
postsuper -d queue_id
|
remove one message by its queue ID |
--- |
postsuper -d ALL
|
remove all messages |
For the script in "/etc/ppp/ip-up.d/*
", "flush all mail" may be good idea.
Use mutt
as the mail user agent (MUA) in combination with vim
. Customize with "~/.muttrc
"; for example:
# use visual mode and "gq" to reformat quotes set editor="vim -c 'set tw=72 et ft=mail'" # # header weeding taken from the manual (Sven's Draconian header weeding) # ignore * unignore from: date subject to cc unignore user-agent x-mailer hdr_order from subject to cc date user-agent x-mailer set hostname=spoof.example.org set from="First Last <username@example.org>" ....
Add the following to "/etc/mailcap
" or "~/.mailcap
" to display HTML mail and MS Word attachments inline:
text/html; lynx -force_html %s; needsterminal; application/msword; /usr/bin/antiword '%s'; copiousoutput; description="Microsoft Word Text"; nametemplate=%s.doc
You need to manually deliver mails to the sorted mailboxes in your home directory from "/var/mail/<username>
" if your home directory became full and procmail
(1) failed. After making disk space in the home directory, run:
# /etc/init.d/${MAILDAEMON} stop # formail -s procmail </var/mail/<username> # /etc/init.d/${MAILDAEMON} start
For mail system programs, there are many alternatives developed with different priority. Here is the overview.
There are many choices for MTA (mail transfer agent).
Table 6.7. List of MTA.
package | popcon | size | capability |
---|---|---|---|
exim4-daemon-light
|
V:61, I:67 | 928 | full |
postfix
|
V:16, I:18 | 3436 | full (security) |
exim4-daemon-heavy
|
V:1.8, I:2 | 1040 | full (flexible) |
sendmail-bin
|
V:2, I:2 | 2080 | full (only if you are already familiar) |
nullmailer
|
V:0.6, I:0.8 | 452 | strip down, no local mail |
ssmtp
|
V:0.9, I:1.4 | 0 | strip down, no local mail |
nbsmtp
|
V:0.2, I:0.2 | 120 | ? |
courier-mta
|
V:0.2, I:0.2 | 4000 | very full (web interface etc.) |
xmail
|
V:0.19, I:0.2 | 824 | light |
masqmail
|
V:0.04, I:0.06 | 556 | light |
esmtp
|
V:0.11, I:0.2 | 156 | light |
esmtp-run
|
V:0.08, I:0.12 | 8 |
light (sendmail compatibility extension to esmtp )
|
msmtp
|
V:0.2, I:0.6 | 324 | light |
msmtp-mta
|
V:0.08, I:0.12 | 32 |
light (sendmail compatibility extension to msmtp )
|
If you subscribe to Debian related mailing list, it may be a good idea to use such MUA as mutt
and gnus
which are the de facto standard for the participant and known to behave as expected.
Table 6.8. List of MUA.
package | popcon | size | type |
---|---|---|---|
iceweasel
|
V:35, I:56 | 3908 | X GUI (unbranded Mozilla Firefox) |
evolution
|
V:23, I:41 | 10200 | X GUI (part of a groupware suite) |
icedove
|
V:11, I:15 | 38108 | X GUI (unbranded Mozilla Thunderbird) |
mutt
|
V:22, I:83 | 5772 |
character terminal probably with vim
|
gnus
|
V:0.04, I:0.5 | 6272 |
character terminal under (x)emacs
|
Although fetchmail
(1) has been de facto standard for the remote mail retrieval on GNU/Linux, the authour likes getmail
(1) now. If you want to reject mail before downloading to save bandwidth, mailfilter
or mpop
may be useful. Whichever mail retriever utilities are used, it is good idea to configure system to deliver retrieved mails to MDA, such as maildrop
, via pipe.
Table 6.9. List of remote mail retrieval and forward utilities.
package | popcon | size | capability |
---|---|---|---|
fetchmail
|
V:2, I:6 | 1812 | mail retriever (POP3, APOP, IMAP) (old) |
getmail4
|
V:0.3, I:0.7 | 632 | mail retriever (POP3, IMAP4, and SDPS) (simple, secure, and reliable) |
mailfilter
|
V:0.01, I:0.07 | 352 | mail retriever (POP3) with with regex filtering capability |
mpop
|
V:0.01, I:0.06 | 364 | mail retriever (POP3) and MDA with filtering capability |
getmail
(1) configuration is described in getmail documentation. Here is my set up to access multiple POP3 accounts as user:
/usr/local/bin/getmails
" as:
#!/bin/sh set -e rcfiles="/usr/bin/getmail" for file in $HOME/.getmail/config/* ; do rcfiles="$rcfiles --rcfile $file" done exec $rcfiles $@
$ sudo chmod 755 /usr/local/bin/getmails $ mkdir -m 0700 $HOME/.getmail $ mkdir -m 0700 $HOME/.getmail/config $ mkdir -m 0700 $HOME/.getmail/log
$HOME/.getmail/config/pop3_name
" for each POP3 acconts as:
[retriever] type = SimplePOP3SSLRetriever server = pop.example.com username = pop3_name@example.com password = secret [destination] type = MDA_external path = /usr/bin/maildrop unixfrom = True 'Spam' [options] verbose = 0 delete = True delivered_to = False message_log = ~/.getmail/log/pop3_name.log
$ chmod 0600 $HOME/.getmail/config/*
/usr/local/bin/getmails
" to run every 15 minutes with cron
(8) by executing "sudo crontab -e -u <user_name>
" and adding following entry:
5,20,35,50 * * * * /usr/local/bin/getmails --quiet
![]() |
Tip |
---|---|
Problems of POP3 access may not come from |
Most MTA programs, such as postfix
and exim4
, function as MDA (mail delivery agent). There are specialized MDA with filtering capabilities.
Although procmail
(1) has been de facto standard for MDA with filter on GNU/Linux, authour likes maildrop
(1) now. Whichever filtering utilities are used, it is good idea to configure system to deliver filtered mails to a qmail-style Maildir.
Table 6.10. List of MDA with filter.
package | popcon | size | description |
---|---|---|---|
procmail
|
V:18, I:86 | 360 | MDA with filter (old) |
mailagent
|
V:0.5, I:6 | 1688 | MDA with perl filter |
maildrop
|
V:0.3, I:0.8 | 1040 | MDA with structured filtering language |
maildrop
(1) configuration is described in maildropfilter documentation. Here is a configuration example for "$HOME/.mailfilter
":
logfile $HOME/.maildroplog # clearly bad looking mails: drop them into X-trash and exit if ( /^X-Advertisement/ ||\ /^Subject:.*BUSINESS PROPOSAL/ ||\ /^Subject:.*URGENT.*ASISSTANCE/ ||\ /^Subject: *I NEED YOUR ASSISTANCE/ ) to "$HOME/Maildir/X-trash/" # Delivering mailinglist messages if ( /^Precedence:.*list/ ||\ /^Precedence:.*bulk/ ||\ /^List-/ ||\ /^X-Distribution:.*bulk/ ) { if ( /^Resent-Sender.*debian-user-request@lists.debian.org/) to "$HOME/Maildir/debian-user/" if ( /^Resent-Sender.*debian-devel-request@lists.debian.org/) to "$HOME/Maildir/debian-devel/" if ( /^Resent-Sender.*debian-announce-request@lists.debian.org/) to "$HOME/Maildir/debian-announce/" to "$HOME/Maildir/mailing-list/" } to "$HOME/Maildir/Inbox/" exit
![]() |
Warning |
---|---|
Unlike |
Equivalent configurartion can be done with procmail
(1) with "$HOME/.procmailrc
" as:
MAILDIR=$HOME/Maildir DEFAULT=$MAILDIR/Inbox/ LOGFILE=$MAILDIR/Maillog # clearly bad looking mails: drop them into X-trash and exit :0 * 1^0 ^X-Advertisement * 1^0 ^Subject:.*BUSINESS PROPOSAL * 1^0 ^Subject:.*URGENT.*ASISSTANCE * 1^0 ^Subject: *I NEED YOUR ASSISTANCE X-trash/ # Delivering mailinglist messages :0 * 1^0 ^Precedence:.*list * 1^0 ^Precedence:.*bulk * 1^0 ^List- * 1^0 ^X-Distribution:.*bulk { :0 * 1^0 ^Return-path:.*debian-devel-admin@debian.or.jp jp-debian-devel/ :0 * ^Resent-Sender.*debian-user-request@lists.debian.org debian-user/ :0 * ^Resent-Sender.*debian-devel-request@lists.debian.org debian-devel/ :0 * ^Resent-Sender.*debian-announce-request@lists.debian.org debian-announce :0 mailing-list/ } :0 Inbox/
If you are to run a private server on LAN, you may consider to run POP3 / IMAP4 server for delivering mail to LAN clients.
Table 6.11. List of POP3/IMAP4 servers.
package | popcon | size | type | description |
---|---|---|---|---|
qpopper
|
V:1.2, I:5 | 644 | POP3 | Qualcomm enhanced version |
courier-pop
|
V:1.4, I:2 | 232 | POP3 | support only the maildir format |
ipopd
|
V:0.12, I:0.2 | 204 | POP3 | formerly part of the University of Washington IMAP package |
cyrus-pop3d-2.2
|
V:0.16, I:0.3 | 856 | POP3 | part of the Cyrus IMAPd suite |
xmail
|
V:0.19, I:0.2 | 824 | POP3 | ESMTP/POP3 mail server |
courier-imap
|
V:3, I:4 | 1604 | IMAP | This provides access to email stored in Maildirs |
uw-imapd
|
V:1.2, I:5 | 272 | IMAP | the University of Washington IMAP |
cyrus-imapd-2.2
|
V:0.5, I:0.7 | 2636 | IMAP | part of the Cyrus IMAPd suite |
In the old Unix-like system, the BSD Line printer daemon was the standard. Since the standard print out format of the free software is PostScript on the Unix like system, some filter system was used along with Ghostscript to enable printing to the non-PostScript printer.
Recently, Common UNIX Printing System (CUPS) is the new de facto standard. The CUPS uses Internet Printing Protocol (IPP). The IPP is now supported by other OSs such as Windows XP and Mac OS X and has became new cross-platform de facto standard for remote printing with bi-directional communication capability.
The standard printable data format for the application on the Debian system is the PostScript (PS) which is a page description language. The data in PS format is fed into the Ghostscript PostScript interpreter to produce the printable data specific to the printer. See Section 11.3.1, “Ghostscript”.
Thanks to the file format dependent auto-conversion feature of the CUPS system, simply feeding any data to the lpr
command should generate the expected print output. (In CUPS, lpr
can be enabled by installing the cups-bsd
package.)
The Debian system has few notable packages for the print servers and utilities:
Table 6.12. List of print servers and utilities.
package | popcon | size | function | port |
---|---|---|---|---|
lpr
|
V:3, I:3 | 440 | BSD lpr/lpd (Line printer daemon) | printer (515) |
lprng
|
V:0.9, I:1.2 | 3020 | , , (Enhanced) | , , |
cups
|
V:29, I:40 | 11164 | Internet Printing CUPS server | IPP (631) |
cups-client
|
V:8, I:41 | 440 |
System V printer commands for CUPS: lp (1), lpstat (1), lpoptions (1), cancel (1), lpmove (8), lpinfo (8), lpadmin (8), …
|
, , |
cups-bsd
|
V:6, I:37 | 180 |
BSD printer commands for CUPS: lpr (1), lpq (1), lprm (1), lpc (8)
|
, , |
cups-driver-gutenprint
|
V:8, I:32 | 1264 | printer drivers for CUPS | Not applicable |
![]() |
Tip |
---|---|
You can configure CUPS system by pointing your web browser to "http://localhost:631/" . |
The Secure SHell (SSH) is the secure way to connect over the Internet. A free version of SSH called OpenSSH is available as the ssh
package in Debian.
Table 6.13. List of remote access server and utilities.
package | popcon | size | tool | comment |
---|---|---|---|---|
openssh-client
|
V:55, I:98 | 2084 | ssh | Secure shell client |
openssh-server
|
V:65, I:77 | 812 | sshd | Secure shell server |
ssh-askpass-fullscreen
|
V:0.11, I:0.5 | 92 | ssh-askpass-fullscreen | asks user for a pass phrase for ssh-add (GNOME2) |
ssh-askpass
|
V:0.7, I:4 | 156 | ssh-askpass | asks user for a pass phrase for ssh-add (plain X) |
![]() |
Caution |
---|---|
See Section 4.7.3, “Extra security measures for the Internet” if your SSH is accessible from Internet. |
![]() |
Tip |
---|---|
Please use the |
/etc/ssh/sshd_not_to_be_run
must not be present if one wishes to run the OpenSSH server.
SSH has two authentication protocols:
Table 6.14. List of SSH authentication protocols and methods.
SSH protocol | SSH method | description |
---|---|---|
SSH-1 | RSAAuthentication | RSA identity key based user authentication |
, , | RhostsAuthentication |
.rhosts based host authentication (insecure, disabled)
|
, , | RhostsRSAAuthentication |
.rhosts authentication combined with RSA host key (disabled)
|
, , | ChallengeResponseAuthentication | RSA challenge-response authentication |
, , | PasswordAuthentication | password based authentication |
SSH-2 | PubkeyAuthentication | public key based user authentication |
, , | HostbasedAuthentication |
"~/.rhosts " or "/etc/hosts.equiv " authentication combined with public key client host authentication (disabled)
|
, , | ChallengeResponseAuthentication | challenge-response authentication |
, , | PasswordAuthentication | password based authentication |
Be careful about these differences if you are using a non-Debian system.
See "/usr/share/doc/ssh/README.Debian.gz
", ssh
(1), sshd
(8), ssh-agent
(1), and ssh-keygen
(1) for details.
Following are the key configuration files:
Table 6.15. List of SSH configuration files.
configuration file | function |
---|---|
/etc/ssh/ssh_config
|
SSH client defaults. See ssh_config (5).
|
/etc/ssh/sshd_config
|
SSH server defaults. See sshd_config (5).
|
~/.ssh/authorized_keys
|
the lists of the default public SSH keys that clients use to connect to this account on this host. |
~/.ssh/identity
|
secret SSH-1 RSA key of the user. |
~/.ssh/id_rsa
|
secret SSH-2 RSA key of the user. |
~/.ssh/id_dsa
|
secret SSH-2 DSA key of the user. |
![]() |
Tip |
---|---|
See |
The following will start an ssh
(1) connection from a client.
Table 6.16. List of SSH client startup examples.
command | description |
---|---|
ssh username@hostname.domain.ext
|
connect with default mode |
ssh -v username@hostname.domain.ext
|
connect with default mode with debugging messages |
ssh -1 username@hostname.domain.ext
|
force to connect with SSH version 1 |
ssh -1 -o RSAAuthentication=no -l username hostname.domain.ext
|
force to use password with SSH version 1 |
ssh -o PreferredAuthentications=password -l username hostname.domain.ext
|
force to use password with SSH version 2 |
If you use the same user name on the local and the remote host, you can eliminate typing "username@
". Even if you use different user name on the local and the remote host, you can eliminate it using "~/.ssh/config
". For Debian Alioth service with account name "foo-guest
", you set "~/.ssh/config
" to contain:
Host alioth.debian.org svn.debian.org git.debian.org User foo-guest
For the user, ssh
(1) functions as a smarter and more secure telnet
(1). Unlike telnet
command, ssh
command does not bomb on the telnet
escape character (initial default CTRL-]).
To establish a pipe to connect to port 25 of remote-server from port 4025 of localhost, and to port 110 of remote-server from port 4110 of localhost through ssh
, execute on the local machine:
# ssh -q -L 4025:remote-server:25 4110:remote-server:110 username@remote-server
This is a secure way to make connections to SMTP/POP3 servers over the Internet. Set the "AllowTcpForwarding
" entry to "yes
" in "/etc/ssh/sshd_config
" of the remote host.
One can avoid having to remember a password for each remote system by using "RSAAuthentication
" (SSH-1 protocol) or PubkeyAuthentication (SSH-2 protocol).
On the remote system, set the respective entries, "RSAAuthentication yes
" or "PubkeyAuthentication yes
", in "/etc/ssh/sshd_config
".
Then generate authentication keys locally and install the public key on the remote system:
RSAAuthentication
": RSA1 key for SSH-1 (deprecated because superseded.)
$ ssh-keygen $ cat .ssh/identity.pub | ssh user1@remote "cat - >>.ssh/authorized_keys"
$ ssh-keygen -t rsa $ cat .ssh/id_rsa.pub | ssh user1@remote "cat - >>.ssh/authorized_keys"
$ ssh-keygen -t dsa $ cat .ssh/id_dsa.pub | ssh user1@remote "cat - >>.ssh/authorized_keys"
![]() |
Note |
---|---|
There are no more reasons to work around RSA patent using DSA since it has been expired. DSA stands for Digital Signature Algorithm and slow. |
One can change the pass phrase later with "ssh-keygen -p
". Make sure to verify settings by testing the connection. In case of any problem, use "ssh -v
".
You can add options to the entries in "~/.ssh/authorized_keys
" to limit hosts and to run specific commands. See sshd
(8) for details.
Note that SSH-2 has "HostbasedAuthentication
". For this to work, you must adjust the settings of "HostbasedAuthentication
" to "yes
" in both "/etc/ssh/sshd_config
" on the server machine and "/etc/ssh/ssh_config
" or "~/.ssh/config
" on the client machine.
There are a few free SSH clients available for other platforms.
Table 6.17. List of free SSH clients for other platforms.
environment | free SSH program |
---|---|
Windows | puTTY (http://www.chiark.greenend.org.uk/~sgtatham/putty/) (GPL) |
Windows (cygwin) | SSH in cygwin (http://www.cygwin.com/) (GPL) |
Macintosh Classic | macSSH (http://www.macssh.com/) (GPL) |
Mac OS X |
OpenSSH; use ssh in the Terminal application (GPL)
|
It is safer to protect your SSH authentication key with a pass phrase. If it was not set, use "ssh-keygen -p
" to set it.
Place your public key (e.g. "~/.ssh/id_rsa.pub
") into "~/.ssh/authorized_keys
" on a remote host using a password-based connection to the remote host as described above.
$ ssh-agent bash $ ssh-add ~/.ssh/id_rsa Enter passphrase for /home/<username>/.ssh/id_rsa: Identity added: /home/<username>/.ssh/id_rsa (/home/<username>/.ssh/id_rsa)
$ scp foo <username>@remote.host:foo
For the X server, the normal Debian startup script executes ssh-agent
as the parent process. So you only need to execute ssh-add
once. For more, read ssh-agent
(1)and ssh-add
(1).
If you have problems, check the permissions of configuration files and run ssh
with the "-v
" option.
Use the "-P
" option if you are root and have trouble with a firewall; this avoids the use of server ports 1--1023.
If ssh
connections to a remote site suddenly stop working, it may be the result of tinkering by the sysadmin, most likely a change in "host_key
" during system maintenance. After making sure this is the case and nobody is trying to fake the remote host by some clever hack, one can regain a connection by removing the "host_key
" entry from "~/.ssh/known_hosts
" on the local machine.
Table 6.18. List of other network application servers.
package | popcon | size | protocol | focus |
---|---|---|---|---|
telnetd
|
V:0.5, I:1.4 | 156 | TELNET | TELNET server |
telnetd-ssl
|
V:0.16, I:0.4 | 204 | , , | , , (SSL support) |
nfs-kernel-server
|
V:14, I:23 | 324 | NFS | Unix file sharing |
samba
|
V:22, I:34 | 13464 | SMB | windows file and printer sharing |
netatalk
|
V:6, I:10 | 2448 | ATP | apple/mac file and printer sharing (AppleTalk) |
proftpd-basic
|
V:4, I:4 | 2060 | FTP | general file download |
wu-ftpd
|
V:0.5, I:0.7 | 820 | , , | , , |
apache2-mpm-prefork
|
V:36, I:42 | 56 | HTTP | general web server |
apache2-mpm-worker
|
V:5, I:6 | 56 | , , | , , |
squid
|
V:6, I:7 | 1816 | , , | general web proxy server |
squid3
|
V:1.0, I:1.3 | 2404 | , , | , , |
slpd
|
V:0.2, I:0.4 | 228 | SLP | OpenSLP Server as LDAP server |
bind9
|
V:11, I:17 | 840 | DNS | IP address for other hosts |
dhcp3-server
|
V:5, I:9 | 808 | DHCP | IP address of client itself |
Common Internet File System Protocol (CIFS) is the same protocol as Server Message Block (SMB).
![]() |
Tip |
---|---|
Use of proxy server such as |
Table 6.19. List of network application clients.
package | popcon | size | protocol | focus |
---|---|---|---|---|
netcat
|
V:2, I:57 | 36 | TCP/IP | TCP/IP swiss army knife |
stunnel4
|
V:0.5, I:1.7 | 508 | SSL | Universal SSL Wrapper |
telnet
|
V:15, I:90 | 200 | TELNET | TELNET client |
telnet-ssl
|
V:0.3, I:1.2 | 244 | , , | , , (SSL support) |
nfs-common
|
V:52, I:82 | 504 | NFS | Unix file sharing |
smbclient
|
V:7, I:41 | 25116 | SMB | MS windows file and printer sharing client |
smbfs
|
V:6, I:26 | 4656 | , , | Mount and umount commands for remote MS windows file |
ftp
|
V:10, I:87 | 160 | FTP | FTP client |
lftp
|
V:1.4, I:6 | 1724 | , , | , , |
ncftp
|
V:1.7, I:8 | 1212 | , , | Full screen FTP client |
wget
|
V:29, I:99 | 1944 | HTTP and FTP | Web downloader |
curl
|
V:5, I:19 | 304 | , , | , , |
dog
|
V:0.07, I:0.3 | 76 | HTTP |
Web uploader (cat with URL support)
|
bind9-host
|
V:47, I:90 | 172 | DNS |
The host command from bind9, priority standard
|
dnsutils
|
V:14, I:91 | 388 | , , |
The dig command from bind, priority standard
|
host
|
V:1.5, I:3 | 180 | , , |
The host command from dnsutils, priority extra
|
dhcp3-client
|
V:50, I:93 | 608 | DHCP | Obtain IP address |
ldap-utils
|
V:1.6, I:7 | 608 | LDAP | Obtain data from LDAP server |
The telnet
program enables manual connection and diagnosis of the system daemons. E.g.:
$ telnet mail.ispname.net pop3
The following RFCs provide required knowledge to text each system daemon.
The port usage is described in "/etc/services
".
The X window system on the Debian system is based on the source from X.Org. As of January 2009, they are X11R7.1(etch), X11R7.3(lenny) and X11R7.3(sid).
There are a few (meta)packages provided to ease installation.
Table 7.1. List of key (meta)packages for X window.
(meta)package | popcon | size | description |
---|---|---|---|
xorg
|
I:52 | 32 | This metapackage provides the X libraries, an X server, a set of fonts, and a group of basic X clients and utilities. |
xserver-xorg
|
V:31, I:58 | 204 | This package provides the full suits of the X server and its configuration. |
xbase-clients
|
V:13, I:56 | 184 | This package provides a miscellaneous assortment of X clients. |
x11-common
|
V:51, I:90 | 756 | This package contains the filesystem infrastructure for the X window system. |
xorg-docs
|
I:12 | 5008 | This package contains miscellaneous documentation for the X.Org software suite. |
xspecs
|
I:1.5 | 6504 | This package contains X protocol, extension, and library technical specifications. |
menu
|
V:31, I:58 | 1956 | This package generates the Debian menu for all menu-aware applications. |
gksu
|
V:27, I:53 | 176 |
This package provides a Gtk+ frontend to su (1) or sudo (8).
|
menu-xdg
|
I:56 | 76 | This package converts the Debian menu structure to the freedesktop.org xdg menu structure. |
xdg-utils
|
V:12, I:50 | 256 | This package provides utilities to integrate desktop environment provided by the freedesktop.org. |
gnome-desktop-environment
|
I:34 | 20 | metapackage for the stadard GNOME desktop environment. |
kde-core
|
I:11 | NOT_FOUND | metapackage for the core KDE desktop environment. |
xfce4
|
I:5 | 48 | metapackage for the Xfce lightweight desktop environment. |
lxde-core
|
I:1.6 | 40 | metapackage for the LXDE lightweight desktop environment. |
fluxbox
|
V:1.3, I:3 | 4332 | Fluxbox: package for highly configurable and low resource X window manager. |
For the basics of X, refer to X
(7), the LDP XWindow-User-HOWTO.
A desktop environment is usually a combination of a X window manager, a file manager, and a suite of compatible utility programs.
You can setup a full desktop environment such as GNOME, KDE, Xfce, or LXDE, from the aptitude
under the task menu.
![]() |
Tip |
---|---|
Task menu may be out of sync with the latest package transition state under Debian |
You may alternatively setup a simple environment manually just with a X window manager such as Fluxbox.
See Window Managers for X for the guide to the X window manager and the desktop environment.
Debian menu system provides a general interface for both text- and X-oriented programs with update-menus
(1) from the menu
package. Each package installs its menu data in the "/usr/share/menu/
" directory. See "/usr/share/menu/README
".
Each package which is compliant to Freedesktop.org's xdg menu system installs its menu data provided by "*.desktop
" under "/usr/share/applications/
". Modern desktop environments which are compliant to Freedesktop.org standard use these data to generate their menu using the xdg-utils
package. See "/usr/share/doc/xdg-utils/README
".
In order to obtain access to the traditional Debian menu under GNOME desktop environment, you must install the menu-xdg
package, click "System" → "Preference" → "Main Menu", and check the box for "Debian".
![]() |
Tip |
---|---|
You may need to do the similar for other modern desktop environments which are compliant to Freedesktop.org standard. |
The X window system is activated as a combination of the server and client programs. The meaning for the words server and client with respect to the words local and remote requires attention here:
Table 7.2. List of server/client terminology.
type | description |
---|---|
X server | a program run on a local host connected to the user's display and input devices. |
X client | a program run on a remote host that processes data and talks to the X server. |
application server | a program run on a remote host that processes data and talks to the clients. |
application client | a program run on a local host connected to the user's display and input devices. |
See xorg
(1) for X server information.
![]() |
Note |
---|---|
X server (post- |
To (re)configure an X server,
# dpkg-reconfigure --priority=low x11-common # dpkg-reconfigure --priority=low xserver-xorg
will generate a new "/etc/X11/xorg.conf
" file using dexconf
(1).
If you have manually edited this "/etc/X11/xorg.conf
" file but would like it to be automatically updated again, run the following command:
# sudo dpkg-reconfigure -phigh xserver-xorg
Please check your X configuration with respect to the specification of your monitor carefully. For the large high resolution CRT monitor, it is a good idea to set the refresh rate as high as your monitor can handle (85 Hz is great, 75 Hz is OK) to reduce flicker. For the LCD monitor, slower standard refresh rate (60Hz) is usually fine due to its slow response.
![]() |
Note |
---|---|
Be careful not to use too high refresh rate which may cause fatal hardware failure of your monitor system. |
There are several ways of getting the "X server" (display side) to accept connections from an "X client" (application side):
Table 7.3. List of connection methods to the X server.
method | package | popcon | size | user | encryption | pertinent use |
---|---|---|---|---|---|---|
xhost command
|
xbase-clients
|
V:13, I:56 | 184 | unchecked | no | deprecated |
xauth command
|
xbase-clients
|
V:13, I:56 | 184 | checked | no | for local connection via pipe |
ssh -X command
|
openssh-client
|
V:55, I:98 | 2084 | checked | yes | for remote network connection |
GNOME display manager |
gdm
|
V:32, I:45 | 15207 | checked | no(XDMCP) | for local connection via pipe |
KDE display manager |
kdm
|
V:10, I:13 | 3772 | checked | no(XDMCP) | for local connection via pipe |
X display manager |
xdm
|
V:0.8, I:2 | 688 | checked | no(XDMCP) | for local connection via pipe |
WindowMaker display manager |
wdm
|
V:23, I:85 | 1968 | checked | no(XDMCP) | for local connection via pipe |
LTSP display manager |
ldm
|
V:0.02, I:0.11 | 284 | checked | yes | for remote SSH network connection (thin client) |
![]() |
Warning |
---|---|
Do not use remote TCP/IP connection over unsecured network for X connection unless you have very good reason such as use of encryption. A remote TCP/IP socket connection without encryption is prone to the eavesdropping attack and is disabled by default on the Debian system. Use " |
![]() |
Warning |
---|---|
Do not use XDMCP connection over unsecured network either. It sends data via UDP/IP without encryption and prone to the eavesdropping attack. |
![]() |
Tip |
---|---|
You can dare to enable remote TCP/IP connection by setting " |
![]() |
Tip |
---|---|
LTSP stands for Linux Terminal Server Project. |
The X Window system is usually started as an X session which is the combination of an X server and connecting X clients. For normal desktop system, both of them are executed on the workstation.
To start the X Window system,
startx
command started from the command line, or
*dm
started from the end of the start up script in "/etc/rc?.d/
" directory ("?
" corresponding to the runlevel)
are used to start the X session. (The start up script for the display manager daemons checks the content of the "/etc/X11/default-display-manager
" file before actually executing themselves.)
![]() |
Tip |
---|---|
See Section 8.3.5, “Specific locale only under X Window” for initial environment variables of the X display manager. |
Essentially, all these programs execute the "/etc/X11/Xsession
" script. Then the "/etc/X11/Xsession
" script performs run-parts like action to execute scripts in the "/etc/X11/Xsession.d/
" directory. This is essentially an execution of a first program which is found in the following order with the exec
builtin command:
/etc/X11/Xsession
" by the X display manager, if it is defined.
~/.xsession
" or "~/.Xsession
" script, if it is defined.
/usr/bin/x-session-manager
" command, if it is defined.
/usr/bin/x-window-manager
" command, if it is defined.
/usr/bin/x-terminal-emulator
" command, if it is defined.
This process is affected by the content of "/etc/X11/Xsession.options
". The exact programs to which these "/usr/bin/x-*
" commands point, are determined by the Debian alternative system and changed by "update-alternatives --config x-session-manager
", etc.
gdm
(1) lets you select the session type (or desktop environment: Section 7.2, “Setting up desktop environment”), and language (or locale: Section 8.3, “The locale”) of the X session from its menu. It keeps the selected default value in "~/.dmrc
" as, e.g.:
[Desktop] Session=default Language=ja_JP.UTF-8
On a system where "/etc/X11/Xsession.options
" contains a line "allow-user-xsession
" without preceding "#
" characters, any user who defines "~/.xsession
" or "~/.Xsession
" will be able to customize the action of "/etc/X11/Xsession
" by completely overiding the system code. The last command in the "~/.xsession
" file should use form of "exec some-window/session-manager
" to start your favorite X window/session managers.
Here are new methods to customize the X session without completely overiding the system code as above.
gdm
can select a specific session and set it as the argument of "/etc/X11/Xsession
".
~/.xsessionrc
" file is executed as a part of start up process (desktop independent.)
~/.gnomerc
" file is executed as a part of start up process. (GNOME desktop only)
~/.gnome2/session
" file etc..
The use of "ssh -X
" enables a secure connection from a local X server to a remote application server.
X11Forwarding
" entries to "yes
" in "/etc/ssh/sshd_config
" of the remote host, if you want to avoid "-X
" command-line option.
xterm
in the local host.
ssh
(1) to establish a connection with the remote site.
localname @ localhost $ ssh -q -X loginname@remotehost.domain Password: .....
gimp
", on the remote site.
loginname @ remotehost $ gimp &
This method allows the display of the remote X client output as if it were locally connected through a local UNIX domain socket.
Secure X terminal via Internet, which displays remotely run entire X desktop environment, can easily achieved by using specialized package such as ldm
. Your local machine becomes a secure thin client to the remote application server connected via SSH.
If you want to add similar feature to your normal display manager gdm
, create executable shell script at "/usr/local/bin/ssh-session
" as:
#!/bin/sh -e # Based on gdm-ssh-session in gdm source (GPL) ZENITY=$(type -p zenity) TARGETHOST=$($ZENITY --width=600 \ --title "Host to connect to" --entry \ --text "Enter the name of the host you want to log in to as user@host.dom:") TARGETSESSION=$($ZENITY --width=600 --height=400 \ --title "Remote session name" --list --radiolist --text "Select one" \ --column " " --column "Session" --column "description" --print-column 2 \ TRUE "/etc/X11/Xsession" "Debian" \ FALSE "/etc/X11/xinit/Xclients" "RH variants" \ FALSE "gnome-session" "GNOME session" \ FALSE "xterm" "Safe choice" \ FALSE "rxvt" "Safe choice" \ FALSE "gnome-terminal" "Safe choice") echo "Connecting to "$TARGETHOST" with $TARGETSESSION" /usr/bin/ssh -A -X -T -n "$TARGETHOST" "$TARGETSESSION" #SSH_ASKPASS=/usr/bin/ssh-askpass /usr/bin/ssh -A -X -T -n "$TARGETHOST" "$TARGETSESSION"
Then add followings to "/etc/dm/Sessions/ssh.desktop
":
[Desktop Entry] Encoding=UTF-8 Name=SSH Comment=This session logs you into a remote host using ssh Exec=/usr/local/bin/ssh-session Type=Application
X window on the Debian system support two mechanisms for font management:
The core X11 font system provides backward compatibility with older applications such as Xterm with bitmap fonts. It is supported by installing pertinent font packages which trigger defoma
(1) scripts to generate required files such as "fonts.dir
".
The Xft2 font system is used by all modern applications such as ones from GNOME, KDE, OpenOffice.org, etc.. It supports all fonts listed below (Section 7.6.1, “Basic fonts”, Section 7.6.2, “Additional fonts”, and Section 7.6.3, “CJK fonts”) with advanced features such as anti-aliasing. It has no configuration mechanism itself, rather it relies upon the fontconfig library to configure and customize fonts as described in fonts.conf
(5). Actual rasterization is supported by the FreeType 2 font engine. These new X clients using Xft2 font system can talk to modern X server via the X Rendering Extension.
Table 7.4. Table of packages to support X window font systems.
package | popcon | size | description |
---|---|---|---|
xfonts-utils
|
V:36, I:70 | 456 | X Window System font utility programs |
libxft2
|
V:44, I:75 | 148 | Xft: FreeType-based font drawing library for X |
libfreetype6
|
V:58, I:88 | 780 | FreeType 2 font engine, shared library files |
fontconfig
|
V:37, I:74 | 460 | generic font configuration library - support binaries |
fontconfig-config
|
V:21, I:82 | 416 | generic font configuration library - configuration |
defoma
|
V:30, I:84 | 449 | Debian Font Manager — automatic font configuration framework |
x-ttcidfont-conf
|
I:40 | 156 | TrueType configuration for X (for CJK support) |
You can check actual font path for:
xset q
"
fc-match
"
![]() |
Tip |
---|---|
"The Penguin and Unicode" is a good overview of modern X Window system. Other documentations at http://unifont.org/ should provide good information on Unicode fonts, Unicode-enabled software, internationalization, and Unicode usability issues on free/libre/open source (FLOSS) operating systems. |
![]() |
Tip |
---|---|
You should rely on fontconfig infrastructure to configure fonts on the Debian system. Debian Font Manager ( |
There are 2 major types of computer fonts:
While scaling of bitmap fonts causes jugged image, scaling of outline/stroke fonts produces smooth image.
Bitmap fonts on the Debian system are provided by compressed X11 pcf bitmap font files having their file extension ".pcf.gz
".
Outline fonts on the Debian system are provided by:
.pfb
" (binary font file) and ".afm
" (font metrics file).
.ttf
".
Table 7.5. Table of corresponding PostScript Type 1 fonts.
font package | popcon | size | sans-serif font | serif font | monospace font | source of font |
---|---|---|---|---|---|---|
PostScript | N/A | N/A | Helvetica | Times | Courier | Adobe |
gsfonts | V:18, I:69 | 4792 | Nimbus Sans L | Nimbus Roman No9 L | Nimbus Mono L | URW (Adobe compatible size) |
gsfonts-x11 | I:29 | 116 | Nimbus Sans L | Nimbus Roman No9 L | Nimbus Mono L | X font support with PostScript Type 1 fonts. |
t1-cyrillic | I:1.9 | 4996 | Free Helvetian | Free Times | Free Courier | URW extended (Adobe compatible size) |
lmodern | V:3, I:16 | 46180 | LMSans* | LMRoman* | LMTypewriter* | scalable PostScript and OpenType fonts based on Computer Modern (from TeX) |
Table 7.6. Table of corresponding TrueType fonts.
font package | popcon | size | sans-serif font | serif font | monospace font | source of font |
---|---|---|---|---|---|---|
ttf-mscorefonts-installer | I:11 | 196 | Arial | Times New Roman | Courier New | Microsoft (Adobe compatible size) (This installs non-free data) |
ttf-liberation | I:38 | 1696 | Liberation Sans | Liberation Serif | Liberation Mono | Liberation Fonts project (Microsoft compatible size) |
ttf-freefont | I:20 | 4212 | FreeSans | FreeSerif | FreeMono | GNU freefont (Microsoft compatible size) |
ttf-bitstream-vera | I:18 | NOT_FOUND | Bitstream Vera Sans | Bitstream Vera Serif | Bitstream Vera Sans Mono | Bitstream, Inc. |
ttf-dejavu | I:81 | 68 | DejaVu Sans | DejaVu Serif | DejaVu Sans Mono | DejaVu, Bitstream with extended character code support |
ttf-dejavu-core | I:55 | 2584 | DejaVu Sans | DejaVu Serif | DejaVu Sans Mono | DejaVu, basic font style variants |
ttf-dejavu-extra | I:55 | 5768 | DejaVu Sans | DejaVu Serif | DejaVu Sans Mono | DejaVu, extra font style variants |
ttf-unifont | I:3 | 16060 | N/A | N/A | unifont | GNU Unifont, with all printable character code in Unicode 5.1 Basic Multilingual Plane (BMP) |
aptitude
(8) will help you find additional fonts easily:
defoma
package list,
~Gmade-of::data:font
",
~nxfonts-
", or
~nttf-
".
Since Free fonts are sometimes limited, installing or sharing some commercial TrueType fonts is an option for a Debian users. In order to make this process easy for the user, some convenience packages have been created:
ttf-mathematica4.1
ttf-mscorefonts-installer
You'll have a really good selection of TrueType fonts at the expense of contaminating your Free system with non-Free fonts.
Here are some key points focused on CJK issues.
Table 7.7. Table of key words used in CJK font names to indicate font types.
font type | Japanese font name | Chinese font name | Korean font name |
---|---|---|---|
sans-serif | gothic, ゴチック hei, | gothic dodu | m, gulim, gothic |
serif | mincho, 明朝 so | ng, ming ba | tang |
Font name such as "VL PGothic" with "P" is a proportional font which corresponds to the fixed width "VL Gothic" font.
For example, Shift_JIS code table comprises 7070 characters. They can be grouped into:
.hbf
" may be deployed for fonts containing single-byte and double-byte characters.
In order to save space for TrueType font files, TrueType font collection file with file extension ".ttc
" may be used.
I order to cover complicated code space of characters, CID keyed Type 1 PostScript font is used with CMap files starting themselves with "%!PS-Adobe-3.0 Resource-CMap
". This is rarely used for normal X display but used for PDF rendering etc. (see Section 7.7.2, “X utility applications”).
![]() |
Tip |
---|---|
The multiple glyphs are expected for some Unicode code points due to Han unification. One of the most annoying ones are "U+3001 IDEOGRAPHIC COMMA" and "U+3002 IDEOGRAPHIC FULL STOP" whose character positions differ among CJK countries. Configuring priority of Japanese centric fonts over Chinese ones using " |
Here is a list of basic office applications (OO is OpenOffice.org):
Table 7.8. List of basic X office applications
package | popcon | package size | description | type |
---|---|---|---|---|
openoffice.org-writer
|
V:25, I:48 | 25020 | word processor | OO |
openoffice.org-calc
|
V:25, I:47 | 17416 | spreadsheet | OO |
openoffice.org-impress
|
V:22, I:47 | 2896 | presentation | OO |
openoffice.org-base
|
V:20, I:46 | 9052 | database management | OO |
openoffice.org-draw
|
V:22, I:47 | 8808 | vector graphics editor (draw) | OO |
openoffice.org-math
|
V:20, I:46 | 1484 | mathematical equation/formula editor | OO |
abiword
|
V:4, I:7 | 8348 | word processor | GNOME |
gnumeric
|
V:4, I:8 | 8248 | spreadsheet | GNOME |
gimp
|
V:14, I:50 | 13468 | bitmap graphics editor (paint) | GTK |
inkscape
|
V:12, I:28 | 61584 | vector graphics editor (draw) | GNOME |
dia-gnome
|
V:1.6, I:4 | 596 | flowchart and diagram editor | GNOME |
mergeant
|
V:0.17, I:0.2 | 1412 | database management | GNOME |
planner
|
V:0.8, I:6 | 7468 | project management | GNOME |
kword
|
V:1.1, I:2 | NOT_FOUND | word processor | KDE |
kspread
|
V:1.0, I:2 | NOT_FOUND | spreadsheet | KDE |
kpresenter
|
V:0.8, I:2 | NOT_FOUND | presentation | KDE |
kexi
|
V:0.4, I:2 | NOT_FOUND | database management | KDE |
kivio
|
V:1.0, I:2 | NOT_FOUND | flowchart and diagram editor | KDE |
karbon
|
V:0.9, I:2 | NOT_FOUND | vector graphics editor (draw) | KDE |
krita
|
V:1.1, I:2 | NOT_FOUND | bitmap graphics editor (paint) | KDE |
kplato
|
V:0.3, I:2 | NOT_FOUND | project management | KDE |
kchart
|
V:0.8, I:2 | NOT_FOUND | graph and chart drawing program | KDE |
kformula
|
V:0.7, I:1.9 | NOT_FOUND | mathematical equation/formula editor | KDE |
kugar
|
V:0.7, I:1.8 | NOT_FOUND | business quality report generator | KDE |
Here is a list of basic utility applications which caught my eyes:
Table 7.9. List of basic X utility applications
package | popcon | package size | description | type |
---|---|---|---|---|
evince
|
V:27, I:44 | 1036 | document(pdf) viewer | GNOME |
kpdf
|
V:7, I:15 | NOT_FOUND | document(pdf) viewer | KDE3 |
okular
|
V:2, I:4 | 3208 | document(pdf) viewer | KDE4 |
evolution
|
V:23, I:41 | 10200 | Personal information Management (groupware and email) | GNOME |
kontact
|
V:2, I:13 | 1504 | Personal information Management (groupware and email) | KDE |
scribus
|
V:0.7, I:3 | 26864 | desktop page layout editor | KDE |
glabels
|
V:0.19, I:0.8 | 1045 | label editor | GNOME |
kbarcode
|
V:0.07, I:0.4 | 2180 | barcode and label printing application | KDE |
gnucash
|
V:0.9, I:2 | 5840 | personal accounting | GNOME |
homebank
|
V:0.08, I:0.4 | 896 | personal accounting | GTK |
kmymoney2
|
V:0.2, I:0.9 | 9488 | personal accounting | KDE |
xsane
|
V:7, I:42 | 744 | scanner frontend | GTK |
kooka
|
V:1.4, I:11 | NOT_FOUND | scanner frontend | KDE |
![]() |
Caution |
---|---|
The |
![]() |
Note |
---|---|
Installing softwares such as |
xmodmap
(1) is a utility for modifying keymaps and pointer button mappings in the X window system.
To get the keycode, run xev
(1) in the X and press keys. To get the meaning of keysym, look into the MACRO definition in "/usr/include/X11/keysymdef.h
" file. All "#define
" statements in this file are named as "XK_
" prepended to keysym names.
Most traditional X client programs, such as xterm
(1), can be started with a set of standard command line options to specify geometry, font, and display.
They also use the X resource database to configure their appearance. The system-wide defaults of X resources are stored in "/etc/X11/Xresources/*
" and application defaults of them are stored in "/etc/X11/app-defaults/*
". Use these settings as the starting points.
The file "~/.Xresources
" is used to store user resource specifications. This file is automatically merged into the default X resources upon login. To make changes to these settings and make them effective immediately, merge them into the database using the command:
$ xrdb -merge ~/.Xresources
See x
(7) and xrdb
(1).
Learn everything about xterm
(1) at http://dickey.his.com/xterm/xterm.faq.html.
![]() |
Warning |
---|---|
Never start the X display/session manager under the root account by typing in |
The easiest way to run a particular X client, e.g. "foo
" as root is to use sudo
(8):
$ sudo foo &
or
$ sudo -s # foo &
or
$ gksu foo &
or
$ ssh -X root@localhost # foo &
![]() |
Caution |
---|---|
Use of |
Please note, in order for the X client to connect to the X server,
$XAUTHORITY
" and "$DISPLAY
" environment variables must be copied to the new user's ones, and
$XAUTHORITY
" environment variable must be readable by the new user.
The gksu
package (popcon: V:27, I:53) is a specialized GTK+ GUI package for gaining the root privileges. It can be configured to use su
(1) or sudo
(8) as its backend depending on the "/apps/gksu/sudo-mode
" gconf key. You can edit gconf key using gconf-editor
(1) (menu: "Applications" → "System Tools" → "Configuration Editor").
Multilingualization (M17N) for an application software is done in 2 steps:
![]() |
Tip |
---|---|
There are 17, 18, or 10 letters between "m" and "n", "i" and "n", or "l" and "n". |
The modern software such as GNOME and KDE are multilingualized. They are internationalized by making them handle UTF-8 data and localized by providing their translated messages through the gettext
(1) infrastructure. Translated messages may be provided as separate localization packages. They can be selected simply by setting pertinent environment variables to the appropriate locale.
The simplest representation of the text data is ASCII which is sufficient for English and uses less than 127 characters (representable with 7 bits). In order to support much more characters for the international support, many character encoding systems have been invented. The modern and sensible encoding system is UTF-8 which can handle practically all the characters known to the human (see Section 8.3.1, “Basics of encoding”).
See Introduction to i18n for details.
The international hardware support is enabled with localized hardware configuration data.
The Debian system can be configured to work with many international keyboard arrangements:
Table 8.1. List of keyboard reconfiguration methods.
environment | command |
---|---|
Linux console |
dpkg-reconfigure --priority=low console-data
|
X Window |
dpkg-reconfigure --priority=low xserver-xorg
|
This will support keyboard input for accented characters of many European languages with its dead-key function. For Asian languages, you need more complicated input method support such as SCIM discussed next.
Setup of multilingual input for the Debian system is simplified by using the SCIM family of packages with the im-switch
package. The list of SCIM packages are:
Table 8.2. List of input method supports with scim.
package | popcon | size | suported locale |
---|---|---|---|
scim-anthy | V:0.2, I:0.6 | 3252 | Japanese |
scim-canna | V:0.06, I:0.3 | 376 | , , |
scim-skk | V:0.04, I:0.3 | 1060 | , , |
scim-prime | V:0.03, I:0.3 | 912 | , , |
scim-tables-ja | I:0.6 | 380 | , , (not very useful) |
scim-tables-zh | I:0.9 | 11676 | Chinese (for zh_*) |
scim-pinyin | V:0.3, I:0.9 | 5324 | , , (for zh_CN) |
scim-chewing | V:0.12, I:0.6 | 264 | , , (for zh_TW) |
scim-hangul | V:0.05, I:0.10 | 288 | Korean |
scim-tables-ko | I:0.08 | 472 | , , |
scim-thai | V:0.02, I:0.04 | 140 | Thai |
scim-m17n | V:0.07, I:0.14 | 92 | Multilingual: Indic, Arabic and others |
scim-tables-additional | I:0.19 | 336 | , , |
scim-uim | V:0.06, I:0.2 | 140 | , , |
The kinput2 method and other locale dependent Asian classic input methods still exist but are not recommended for the modern UTF-8 X environment. The uim tool chain is an alternative approach for the international input method for the modern UTF-8 X environment which is also capable for non-X environment.
I find the Japanese input method started under English environment ("en_US.UTF-8
") very useful. Here is how I did it with SCIM.
scim-anthy
with its recommended packages such as im-switch
.
im-switch -c
" from user's shell and select "scim".
im-switch -l
".
![]() |
Note |
---|---|
In order to start SCIM under the non-CJK and non-en_US locale, you need to add list of those locales in UTF-8 to the " |
/SupportedUnicodeLocales = en_US.UTF-8,en_GB.UTF_8,fr_FR.UTF-8
Please note:
im-switch
(8) behaves differently if command is executed from root or not.
im-switch
depends on the locale.
$GTK_IM_MODULE
") may cause instability during the library transition in unstable
.
For the detail of setup, see "/usr/share/doc/im-switch/README.Debian.gz
", "/usr/share/doc/scim/README.Debian.gz
" or "/usr/share/doc/uim/README.Debian.gz
". Here key points are described.
If you wish to input without going through XIM, set "$XMODIFIERS
" value to "none" while starting a program. This may be the case if you use Japanese input infrastructure egg
on emacs
(1). From shell, execute as:
$ XMODIFIERS=none emacs
In order to adjust the command executed by the Debian menu, place customized configuration in "/etc/menu/
" following method described in "/usr/share/doc/menu/html
".
Linux console can only display limited characters. (You need to use special terminal program such as jfbterm
(1) to display non-European languages on the non-X console.)
X Window can display any characters in the UTF-8 as long as required font data exists. (The encoding of the original font data is taken care by the X Window system and transparent to the user.)
The following will focus on the locale for applications run under X Window environment started from gdm
(1).
The environment variable "LANG=xx_YY.ZZZZ
" sets the locale to language code "xx
", country code "yy
", and encoding "ZZZZ
" (see Section 1.5.1.1, “"$LANG" variable”).
Current Debian system normally sets the locale as "LANG=xx_YY.UTF-8
". This uses the UTF-8 encoding with the Unicode character set. This UTF-8 encoding system is a multibyte code system and uses code points smartly. The ASCII data, which consist only with 7-bit range codes, are always valid UTF-8 data consisting only with 1 byte per character.
Previous Debian system used to set the locale as "LANG=C
" or "LANG=xx_YY
" (without ".UTF-8
").
LANG=C
" or "LANG=POSIX
".
LANG=xx_YY
".
Actual traditional encoding system used for "LANG=xx_YY
" can be identified by checking "/usr/share/i18n/SUPPORTED
". For example, "en_US
" uses "ISO-8859-1
" encoding and "fr_FR@euro
" uses "ISO-8859-15
" encoding.
![]() |
Tip |
---|---|
For meaning of encoding values, see Table 11.2, “List of encoding values and their usage.”. |
The UTF-8 encoding is the modern and sensible text encoding system for I18N and enables to represent Unicode characters, i.e., practically all characters known to human. UTF stands for Unicode Transformation Format (UTF) schemes.
I recommend to use UTF-8 locale for your desktop, e.g., "LANG=en_US.UTF-8
". The first part of the locale determines messages presented by applications. For example, gedit
(1) (text editor for the GNOME Desktop) under "LANG=fr_FR.UTF-8
" locale can display and edit Chinese character text data while presenting menus in French, as long as required fonts and input methods are installed.
I also recommend to set the locale only using the "$LANG
" environment variable. I do not see much benefit of setting a complicated combination of "LC_*
" variables (see locale
(1)) under UTF-8 locale.
Even plain English text may contain non-ASCII characters, e.g. left and right quotation marks are not available in ASCII:
“double quoted text” ‘single quoted text’
When ASCII plain text data is converted to UTF-8 one, it has exactly the same content and size as the original ASCII one. So you loose nothing by deploying UTF-8 locale.
Some programs consume more memory after supporting I18N. This is because they are coded to use UTF-32(UCS4) internally to support Unicode for speed optimization and consume 4 bytes per each ASCII character data independent of locale selected. Again, you loose nothing by deploying UTF-8 locale.
The vendor specific old non-UTF-8 encoding systems tend to have minor but annoying differences on some characters such as graphic ones for many countries. The deployment of the UTF-8 system by the modern OSs practically solved these conflicting encoding issues.
In order for the system to access a particular locale, the locale data must be compiled from the locale database. (The Debian system does not come with all available locales pre-compiled unless you installed the locales-all
package.) The full list of supported locales available for compiling are listed in "/usr/share/i18n/SUPPORTED
". This lists all the proper locale names. The following will list all the available UTF-8 locales already compiled to the binary form:
$ locale -a
The following command execution will reconfigure the locale
package:
/etc/defaults/locale
" for use by PAM (see Section 4.5, “PAM and NSS”).
# dpkg-reconfigure locales
The list of available locale should include "en_US.UTF-8
" and all the interesting languages with "UTF-8
".
The recommended default locale is "en_US.UTF-8
" for US English. For other languages, please make sure to chose locale with "UTF-8
". Any one of these settings can handle any international characters.
![]() |
Note |
---|---|
Although setting locale to " |
The environment variable "$LANG
" is:
gdm
(1) for all X programs,
~/.xsessionrc
" for all X programs (lenny
feature),
login
(1) for the local Linux console programs,
ssh
(1) for the remote console programs, or
~/.bashrc
", for all console programs.
![]() |
Tip |
---|---|
It is good idea to install system wide default locale as " |
You can chose specific locale only under X Window irrespective of your system wide default locale. This should provide your best desktop experience with stability.
This way, you can always access functioning character terminal with readable messages even when X Window system is not working. This becomes essential for languages which use non-roman characters such as Chinese, Japanese, and Korean.
For gdm
(1), you can select different locale for the X session from its menu independent of the system default locale value in the "/etc/defaults/locale
".
You can set the locale of the X session manager and the value for the default locale permanently using PAM customization (see Section 4.5, “PAM and NSS”) as follows. (There may be another way available as the improvement of X session manager package but please read following as the generic and basic method of setting the locale.)
First, change the following line defining language environment variable in its PAM configuration file, such as "/etc/pam.d/gdm
":
auth required pam_env.so read_env=1 envfile=/etc/default/locale
into
auth required pam_env.so read_env=1 envfile=/etc/default/locale-x
Then create a "/etc/defaults/locale-gdm
" file with "-rw-r--r-- 1 root root
" permission containing, eg. for Japanese message:
LANG="ja_JP.UTF-8"
and keep the default "/etc/defaults/locale
" file for other programs being:
LANG="en_US.UTF-8"
This is the most generic technique to customize locale.
Alternatively for this case, you may simply change locale using the "~/.xsessionrc
" file.
For cross platform data exchanges (see Section 10.1.10, “Removable mass storage device”), you may need to mount some file system with particular encodings. For example, mount
(8) for vfat filesystem assumes CP437 if used without option. You need to provide
explicit mount option to use UTF-8 or CP932 for filenames.
![]() |
Note |
---|---|
When auto-mounting a hot-pluggable USB memory stick under modern desktop environment such as GNOME, you may provide such mount option by right clicking the icon on the desktop, click "Drive" tab, click to expand "Setting", and entering "utf8" to "Mount options:". The next time this memory stick is mounted, mount with UTF-8 is enabled. |
![]() |
Note |
---|---|
If you are upgrading system or moving disk drives from older non-UTF-8 system, file names with non-ASCII characters may be encoded in the historic and deprecated encodings such as ISO-8859-1 or eucJP. Please seek help of text conversion tools to convert them to UTF-8. See Section 11.1, “Text data conversion tools”. |
Samba uses Unicode for newer clients (Windows NT, 200x, XP) but uses CP850 as default for older clients (DOS and Windows 9x/Me clients). This default for older clients can be changed using "dos charset
" in the "/etc/samba/smb.conf
" file, e.g., to CP932 for Japanese.
Translations exist for many of the text messages and documents that are displayed in the Debian system, such as error messages, standard program output, menus, and manual pages. GNU gettext(1) command tool chain is used as the backend tool for most translation activities.
aptitude
(8) lists under "Tasks" → "Localization" provide extensive list of useful binary packages which add localized messages to applications and provide translated documentation.
For example, you can obtain the localized message for manpage by installing the manpages-<LANG>
package. To read the Italian-language manpage for <programname>, execute
LANG=it_IT.UTF-8 man <programname>
to read it from "/usr/share/man/it/
".
The sort order of characters with sort
(1) is affected by the language choice of the locale. Spanish and English locale sort differently.
The date format of ls
(1) is affected by the locale. The date format of "LANG=C ls -l
" and "LANG=en_US.UTF-8
" are different (see Section 9.2.5, “Customized display of time and date”).
Number punctuation are different for locales. For example, in English locale, one thousand one point one is displayed as "1,000.1
" while in German locale, it is displayed as "1.000,1
". You see this difference in spreadsheet program.
Here, I will describe basic tips to configure and manage systems, mostly from the console.
screen
(1) is a very useful tool for people to access remote sites via unreliable or intermittent connections since it support interrupted network connections.
Table 9.1. List of programs to support interrupted network connections.
package | popcon | size | description |
---|---|---|---|
screen
|
V:11, I:31 | 1036 | terminal multiplexer with VT100/ANSI terminal emulation |
screen
(1) not only allows one terminal window to work with multiple processes, but also allows remote shell process to survive interrupted connections. Here is a typical use scenario of screen
(1).
screen
on a single console.
screen
windows created with ^A c
("Control-A" followed by "c").
screen
windows by ^A n
("Control-A" followed by "n").
You detach the screen
session by any methods such as:
^A d
("Control-A" followed by "d") and manually logging out from the remote connection, or
^A DD
("Control-A" followed by "DD") to have screen
detach and log you out.
screen
as "screen -r
".
screen
will magically reattach all previous screen
windows with all actively running programs.
![]() |
Tip |
---|---|
You can save connection fees for metered network connections such as dial-up and packet ones, because you can leave a process active while disconnected, and then re-attach it later when you connect again. |
In a screen
session, all keyboard inputs are sent to your current window except for the command keystroke, by default ^A
("Control-A"). All screen
commands are entered by typing ^A
plus a single key [plus any parameters]. Here are important ones to remember:
Table 9.2. List of key bindings for screen.
key binding | meaning |
---|---|
^A ?
|
show a help screen (display key bindings) |
^A c
|
create a new window and switch to it |
^A n
|
go to next window |
^A p
|
go to previous window |
^A 0
|
go to window number 0 |
^A 1
|
go to window number 1 |
^A w
|
show a list of windows |
^A a
|
send a Ctrl-A to current window as keyboard input |
^A h
|
write a hardcopy of current window to file |
^A H
|
begin/end logging current window to file |
^A ^X
|
lock the terminal (password protected) |
^A d
|
detach screen session from the terminal |
^A DD
|
detach screen session and log out |
See screen
(1) for details.
Many programs record their activities under the "/var/log/
" directory.
klogd
(8)
syslogd
(8)
See Section 3.5.8, “The system message” and Section 3.5.9, “The kernel message”.
Here are notable log analyzers ("~Gsecurity::log-analyzer
" in aptitude
(8)).
Table 9.3. List of system log analyzers.
package | popcon | size | description |
---|---|---|---|
logwatch
|
V:2, I:3 | 2312 | log analyser with nice output written in Perl |
fail2ban
|
V:3, I:3 | 616 | bans IPs that cause multiple authentication errors |
analog
|
V:1.4, I:17 | 4612 | web server log analyzer |
awstats
|
V:1.7, I:3 | 5100 | powerful and featureful web server log analyzer |
sarg
|
V:1.6, I:1.8 | 1448 | squid analysis report generator |
pflogsumm
|
V:0.3, I:0.7 | 164 | Postfix log entry summarizer |
syslog-summary
|
V:0.2, I:1.0 | 80 | summarize the contents of a syslog log file |
lire
|
V:0.17, I:0.2 | 5056 | full-featured log analyzer and report generator |
fwlogwatch
|
V:0.13, I:0.2 | 432 | Firewall log analyzer |
squidview
|
V:0.11, I:0.6 | 260 | monitors and analyses squid access.log files |
visitors
|
V:0.10, I:0.3 | 224 | fast web server log analyzer |
swatch
|
V:0.08, I:0.2 | 112 | Log file viewer with regexp matching, highlighting, & hooks |
crm114
|
V:0.07, I:0.2 | 1164 | The Controllable Regex Mutilator and Spam Filter (CRM114) |
icmpinfo
|
V:0.06, I:0.3 | 84 | Interpret ICMP messages |
![]() |
Note |
---|---|
CRM114 provides language infrastructure to write fuzzy filters with the TRE regex library. Its popular use is spam mail filter but it can be used as log analyzer. |
The simple use of script
(1) (see Section 1.4.9, “Recording the shell activities”) to record shell activity produces a file with control characters. This can be avoided by using col
(1):
$ script Script started, file is typescript
Ctrl-D
to exit script
$ col -bx <typescript >cleanedfile $ vim cleanedfile
If you don't have script
(for example, during the boot process in the initramfs), you can use following instead:
$ sh -i 2>&1 | tee typescript
![]() |
Tip |
---|---|
Some |
![]() |
Tip |
---|---|
You may use |
![]() |
Tip |
---|---|
You may use |
Although pager tools such as more
(1) and less
(1) (see Section 1.4.5, “The pager”) and custom tools for highlighting and formatting Section 11.1.7, “Highlighting and formatting plain text data” can display text data nicely, general purpose editors (see Section 1.4.6, “The text editor”) are most versatile and customizable.
![]() |
Tip |
---|---|
For |
The default display format of time and date by the "ls -l
" command depends on the locale (see value Section 1.2.6, “Timestamps”). The "$LANG
" variable is referred first and it can be overridden by the "$LC_TIME
" variable.
The actual default display format for each locale depends on the version of the standard C library (the libc6
package) used. I.e., different releases of Debian had different defaults.
If you really wish to customize this display format of time and date beyond the locale, you should set the time style value by the "--time-style
" argument or by the "$TIME_STYLE
" value (see ls
(1), date
(1), "info coreutils 'ls invocation'
").
Table 9.4. Display examples of time and date for the "ls -l
" command for lenny
.
time style value | locale | display of time and date |
---|---|---|
iso
|
any |
01-19 00:15
|
long-iso
|
any |
2009-01-19 00:15
|
full-iso
|
any |
2009-01-19 00:15:16.000000000 +0900
|
locale
|
C
|
Jan 19 00:15
|
locale
|
en_US.UTF-8
|
2009-01-19 00:15
|
locale
|
es_ES.UTF-8
|
ene 19 00:15
|
+%d.%m.%y %H:%M
|
any |
19.01.09 00:15
|
+%d.%b.%y %H:%M
|
C or en_US.UTF-8
|
19.Jan.09 00:15
|
+%d.%b.%y %H:%M
|
es_ES.UTF-8
|
19.ene.09 00:15
|
![]() |
Tip |
---|---|
You can eliminate typing long option on commandline using command alias, e.g. " |
![]() |
Tip |
---|---|
ISO 8601 is followed for these iso-formats. |
Shell echo to most modern terminals can be colorized using ANSI escape code (see "/usr/share/doc/xterm/ctlseqs.txt.gz
"). E.g.:
$ RED=$(printf "\x1b[31m") $ NORMAL=$(printf "\x1b[0m") $ REVERSE=$(printf "\x1b[7m") $ echo "${RED}RED-TEXT${NORMAL} ${REVERSE}REVERSE-TEXT${NORMAL}"
Colorized commands are handy for inspecting their output in the interactive environment. I include following in my "~/.bashrc
".
if [ "$TERM" != "dumb" ]; then eval "`dircolors -b`" alias ls='ls --color=always' alias ll='ls --color=always -l' alias la='ls --color=always -A' alias less='less -R' alias ls='ls --color=always' alias grep='grep --color=always' alias egrep='egrep --color=always' alias fgrep='fgrep --color=always' alias zgrep='zgrep --color=always' else alias ll='ls -l' alias la='ls -A' fi
The use of alias limits color effects to the interactive command usage. It has advantage over exporting environment variable "export GREP_OPTIONS='--color=auto'
" since color can be seen under pager programs such as less
(1). If you wish to surpress color when piping to other programs, use "--color=auto
" instead in the above example for "~/.bashrc
".
![]() |
Tip |
---|---|
You can turn off these colorizing aliases in the interactive environment by invoking shell with " |
There are few ways to record the graphic image of an X application, including an xterm
display.
Table 9.5. List of graphic image manipulation tools.
package | popcon | size | command |
---|---|---|---|
xbase-clients
|
V:13, I:56 | 184 |
xwd (1)
|
gimp
|
V:14, I:50 | 13468 | GUI menu |
imagemagick
|
V:15, I:32 | 304 |
import (1)
|
scrot
|
V:0.2, I:1.2 | 80 |
scrot (1)
|
There are specialized tools to record changes in configuration files with help of DVCS system.
Table 9.6. List of packages to record configuration history in VCS.
package | popcon | size | description |
---|---|---|---|
etckeeper
|
V:0.4, I:0.7 | 372 | store configuration files and its metadata with Git (default), Mercurial, or Bazaar. (new) |
changetrack
|
V:0.06, I:0.08 | 152 | store configuration files with RCS. (old) |
I recommend to use the etckeeper
package with git
(1) which put entire "/etc
" under VCS control. Its installation guide and tutorial are found in "/usr/share/doc/etckeeper/README.gz
".
Essentially, running "sudo etckeeper init
" initializes the git repository for "/etc
" just like the process explained in Section 10.5.4.4, “Git for recording configuration history”) but with special hook scripts for more thorough setups.
As you change your configuration, you can use git
(1) normally to record them. It will automatically record changes nicely every time you run package management commands, too.
![]() |
Tip |
---|---|
You can browse the change history of " |
Booting your system with Linux live CDs or debian-installer CDs in rescue mode make it easy for you to reconfigure data storage on your boot device. See also Section 10.2, “The binary data”.
For partition configuration, although fdisk
(8) has been considered standard, parted
(8) deserves some attention. "Disk partitioning data", "partition table", "partition map", and "disk label" are all synonyms.
Most PCs use the classic Master Boot Record (MBR) scheme to hold disk partitioning data in the first sector, i.e., LBA sector 0 (512 bytes).
![]() |
Note |
---|---|
Some new PCs with Extensible Firmware Interface (EFI), including Intel-based Macs, use GUID Partition Table (GPT) scheme to hold disk partitioning data not in the first sector. |
Although fdisk
(8) has been standard for the disk partitioning tool, parted
(8) is replacing it.
Table 9.7. List of disk partition management packages
package | pocon | size | description | GUID Partition Table |
---|---|---|---|---|
util-linux
|
V:90, I:99 | 1848 |
Miscellaneous system utilities including fdisk (8) and cfdisk (8)
|
Not supported |
parted
|
V:1.0, I:8 | 164 | The GNU Parted disk partition resizing program | Supported |
gparted
|
V:4, I:42 | 3168 |
GNOME partition editor based on libparted
|
Supported |
qtparted
|
V:0.17, I:1.2 | 764 |
KDE partition editor based on libparted
|
Supported |
gptsync
|
V:0.01, I:0.15 | 72 | Synchronize classic MBR partition table with the GPT one | Supported |
![]() |
Caution |
---|---|
Although |
![]() |
Note |
---|---|
In order to switch between GPT and MBR, you need to erase first few blocks of disk contents directly (see Section 10.2.11, “Clear file contents”) and use " |
Although reconfiguration of your partition may yield different names for partitions, you can access them consistently. This is also helpful if you have multiple disks and your BIOS doesn't give them consistent device names.
mount
(8) with "-U
" options can mount a block device using UUID, instead of using its file name such as "/dev/sda3
".
/etc/fstab
" (see fstab
(5)) can use UUID.
![]() |
Tip |
---|---|
You can probe UUID of a block special device with |
For ext3 filesystem, the e2fsprogs
package provides:
The mkfs
(8) and fsck
(8) commans are provided by the e2fsprogs
package as front-ends to various filesystem dependent programs (mkfs.fstype
and fsck.fstype
). For ext3 filesystem, they are mkfs.ext3
(8) and fsck.ext3
(8) (they are hardlinked to mke2fs
(8) and e2fsck
(8)).
Similar commands are available for each filesystem supported by Linux.
Table 9.8. List of filesystem management packages
package | popcon | size | description |
---|---|---|---|
e2fsprogs
|
V:66, I:99 | 1884 | Utilities for the ext2/ext3/ext4 filesystems. |
reiserfsprogs
|
V:3, I:10 | 1200 | Utilities for the Reiserfs filesystem. |
dosfstools
|
V:3, I:23 | 224 | Utilities for the FAT filesystem. (Microsoft: MS-DOS, Windows) |
xfsprogs
|
V:2, I:10 | 3044 | Utilities for the XFS filesystem. (SGI: IRIX) |
ntfsprogs
|
V:1.5, I:6 | 632 | Utilities for the NTFS filesystem. (Microsoft: Windows NT, …) |
jfsutils
|
V:0.6, I:3 | 1116 | Utilities for the JFS filesystem. (IBM: AIX, OS/2) |
reiser4progs
|
V:0.08, I:0.7 | 1292 | Utilities for the Reiser4 filesystem. |
hfsprogs
|
V:0.04, I:0.5 | 324 | Utilities for HFS and HFS Plus filesystem. (Apple: Mac OS) |
btrfs-tools
|
V:0.01, I:0.13 | 968 | Utilities for the btrfs filesystem. |
![]() |
Tip |
---|---|
Ext3 filesystem is the default filesystem for the Linux system and strongly recommended to use it unless you have some specific reasons not to. After Linux kernel 2.6.28 (Debian |
![]() |
Warning |
---|---|
You might face some limtations with ext4 since it is new. For example, you must have Linux kernel 2.6.30 or later if you wish to resizean ext4 partition. |
![]() |
Tip |
---|---|
Some tools allow access to filesystem without Linux kernel support (see Section 10.2.5, “Manipulating files without mounting disk”). |
The mkfs
(8) command creates the filesystem on a Linux system. The fsck
(8) command provides the filesystem integrity check and repair on a Linux system.
![]() |
Caution |
---|---|
It is generally not safe to run |
![]() |
Tip |
---|---|
Check files in " |
![]() |
Tip |
---|---|
Use " |
Performance and characteristics of a filesystem can be optimized by mount options used on it (see fstab
(5) and mount
(8)). For example:
defaults
" option implies default options: "rw,suid,dev,exec,auto,nouser,async
". (general)
noatime
" or "relatime
" option is very effective for speeding up the read access. (general)
user
" option allows an ordinary user to mount the file system. This option implies "noexec,nosuid,nodev
" option combination. (general, used for CD and floppy)
noexec,nodev,nosuid
" option combination is used to enhance security. (general)
noauto
" option limits mounting by explicit operation only. (general)
data=journal
" option for ext3fs can enhance data integrity against power failure with some loss of write speed.
![]() |
Tip |
---|---|
You need to provide kernel boot parameter " |
Characteristics of a filesystem can be optimized via its superblock using the tune2fs
(8) command. For example on "/dev/hda1
":
sudo tune2fs -l /dev/hda1
" will display the contents of its filesystem superblock.
sudo tune2fs -c 50 /dev/hda1
" will change frequency of filesystem checks (fsck
execution during boot-up) to every 50 boots.
sudo tune2fs -j /dev/hda1
" will add journaling capability to the filesystem, i.e. filesystem conversion from ext2 to ext3. (Do this on the unmounted filesystem.)
sudo tune2fs -O extents,uninit_bg,dir_index /dev/hda1 && fsck -pf /dev/hda1
" will convert it from ext3 to ext4. (Do this on the unmounted filesystem.)
![]() |
Warning |
---|---|
Filesystem conversion for the boot device to the ext4 filesystem should be avoided until GRUB boot loader supports the ext4 filesystem well and installed Linux Kernel version is newer than 2.6.28. |
![]() |
Warning |
---|---|
Please check your hardware and read manpage of |
You can test disk access speed of a harddisk, e.g. "/dev/hda
", by "hdparm -tT /dev/hda
". For some harddisk connected with (E)IDE, you can speed it up with "hdparm -q -c3 -d1 -u1 -m16 /dev/hda
" by enabling the "(E)IDE 32-bit I/O support", enabling the "using_dma flag", setting "interrupt-unmask flag", and setting the "multiple 16 sector I/O" (dangerous!).
You can test write cache feature of a harddisk, e.g. "/dev/sda
", by "hdparm -W /dev/sda
". You can disable its write cache feature with "hdparm -W 0 /dev/sda
".
You may be able to read badly pressed CDROMs on modern high head CD-ROM drive by slowing it down with "setcd -x 2
.
You can monitor and log your harddisk which is compliant to SMART with the smartd
(8) daemon.
smartmontools
package.
Identify your harddisk drives by listing them with df
(1).
/dev/hda
".
Check the output of "smartctl -a /dev/hda
" to see if SMART feature is actually enabled.
smartctl -s on -a /dev/hda
".
Enable smartd
(8) daemon to run by:
start_smartd=yes
" in the "/etc/default/smartmontools
" file.
smartd
(8) daemon by "sudo /etc/init.d/smartmontools restart
".
![]() |
Tip |
---|---|
The |
For partitions created on Logical Volume Manager (Linux) at install time, they can be resized easily by concatenating extents onto them or truncating extents from them over multiple storage devices without major system reconfiguration.
![]() |
Caution |
---|---|
Deployment of the current LVM system may degrade guarantee against filesystem corruption offered by journaled file systems such as ext3fs unless their system performance is sacrificed by disabling write cache of harddisk. |
If you have an empty partition (e.g., "/dev/sdx
"), you can format it with mkfs.ext3
(1) and mount
(8) it to a directory where you need more space. (You need to copy original data contents.)
$ sudo mv work-dir old-dir $ sudo mkfs.ext3 /dev/sdx $ sudo mount -t ext3 /dev/sdx work-dir $ sudo cp -a old-dir/* work-dir $ sudo rm -rf old-dir
If you have an empty directory (e.g., "/path/to/emp-dir
") in another partition with usable space, you can create a symlink to the directory with ln
(8).
$ sudo mv work-dir old-dir $ sudo mkdir -p /path/to/emp-dir $ sudo ln -sf /path/to/emp-dir work-dir $ sudo cp -a old-dir/* work-dir $ sudo rm -rf old-dir
![]() |
Caution |
---|---|
Some software may not function well with "symlink to a directory". |
If you have usable space in another partition (e.g., "/path/to/
"), you can create a directory in it and stack that on to a directory where you need space with aufs.
$ sudo mv work-dir old-dir $ sudo mkdir -p /path/to/emp-dir $ sudo mount -t aufs -o br:/path/to/emp-dir:old-dir none work-dir
![]() |
Caution |
---|---|
Use of aufs for long term data storage is not good idea since it is under development and its design change may introduce issues. |
With physical access to your PC, anyone can easily gain root privilege and access all the files on your PC (see Section 4.7.4, “Securing the root password”). This means that login password system can not secure your private and sensitive data against possible theft of your PC. You must deploy data encryption technology to do it. Although GNU privacy guard (see Section 10.3, “Data security infrastructure”) can encrypt files, it takes some user efforts.
dm-crypt and eCryptfs facilitates automatic data encryption natively via Linux kernel modules with minimal user efforts.
Table 9.9. List of data encryption utilities.
package | popcon | size | function |
---|---|---|---|
cryptsetup
|
V:3, I:4 | 904 | Utilities for encrypted block device (dm-crypt / LUKS) |
cryptmount
|
V:0.09, I:0.5 | 304 | Utilities forencrypted block device (dm-crypt / LUKS) with focus on mount/unmount by normal users |
ecryptfs-utils
|
V:0.09, I:0.2 | 444 | Utilities for encrypted stacked filesystem (eCryptfs) |
Dm-crypt is a cryptographic filesystem using device-mapper. Device-mapper maps one block device to another.
eCryptfs is another cryptographic filesystem using stacked filesystem. Stacked filesystem stacks itself on top of an existing directory of a mounted filesystem.
![]() |
Caution |
---|---|
Data encryption costs CPU time etc. Please weigh its benefits and costs. |
![]() |
Note |
---|---|
Entire Debian system can be installed on a encrypted disk by the debian installer (lenny or newer) using dm-crypt/LUKS and initramfs. |
![]() |
Tip |
---|---|
See Section 10.3, “Data security infrastructure” for user space encryption utility: GNU Privacy Guard. |
You can encrypt contents of removable mass storage devices, e.g. USB memory stick on "/dev/sdx
", using dm-crypt/LUKS. You simply formatting it as:
# badblocks -c 10240 -s -w -t random -v /dev/sdx # shred -v -n 1 /dev/sdx # fdisk /dev/sdx ... "n" "p" "1" "return" "return" "w" # cryptsetup luksFormat /dev/sdx1 ... # cryptsetup luksOpen /dev/sdx1 sdx1 ... # ls -l /dev/mapper/ total 0 crw-rw---- 1 root root 10, 60 2008-10-04 18:44 control brw-rw---- 1 root disk 254, 0 2008-10-04 23:55 sdx1 # mkfs.vfat /dev/mapper/sdx1 ... # cryptsetup luksClose sdx1
Then, it can be mounted just like normal one on to "/media/<disk_label>
", except for asking password (see Section 10.1.10, “Removable mass storage device”) under modern desktop environment, such as GNOME using gnome-mount
(1). The difference is that every data written to it is encrypted. You may alternatively format media in different file format, e.g., ext3 with "mkfs.ext3 /dev/sdx1
".
![]() |
Note |
---|---|
If you are really paranoid for the security of data, you may need to overwrite multiple times in the above example. This operation is very time consuming though. |
If your original "/etc/fstab
" contains:
/dev/sda7 swap sw 0 0
then you can enable encrypted swap partition using dm-crypt by as
# aptitude install cryptsetup # swapoff -a # echo "cswap /dev/sda7 /dev/urandom swap" >> /etc/crypttab # perl -i -p -e "s/\/dev\/sda7/\/dev\/mapper\/cswap/" /etc/fstab # /etc/init.d/cryptdisks restart ... # swapon -a
You can encrypt files written under "~/Private/
" automatically using eCryptfs and the ecryptfs-utils
package.
ecryptfs-setup-private
(1) and set up "~/Private/
" by following prompts.
~/Private/
" by running ecryptfs-mount-private
(1).
move sensitive data files to "~/Private/
" and make symlinks as needed.
~/.fetchmailrc
", "~/.ssh/identity
", "~/.ssh/id_rsa
", "~/.ssh/id_dsa
" and other files with "go-rwx
".
move sensitive data directories to a subdirectory in "~/Private/
" and make symlinks as needed.
~/.gnupg
" and other directories with "go-rwx
".
~/Desktop/Private/
" to "~/Private/
" for easier desktop operations.
~/Private/
" by running ecryptfs-umount-private
(1).
~/Private/
" by issuing "ecryptfs-mount-private
" as you need encrypted data.
If you use your login password for wrapping encryption keys, you can automate mounting eCryptfs via
Pluggable Authentication Module by having a following line just before "pam_permit.so
" in "/etc/pam.d/common-auth
" as:
auth required pam_ecryptfs.so unwrap
and the last line in "/etc/pam.d/common-session
" as:
session optional pam_ecryptfs.so unwrap
and the first active line in "/etc/pam.d/common-password
" as:
password required pam_ecryptfs.so
This is quite convienient.
![]() |
Warning |
---|---|
Configuration errors of PAM may lock you out of your own system. See Chapter 4, Authentication. |
![]() |
Caution |
---|---|
If you use your login password for wrapping encryption keys, your encrypted data are as secure as your user login password (see Section 4.3, “Good password”). Unless you are careful to set up a strong password, your data will be at risk when someone runs password cracking software after stealing your laptop (see Section 4.7.4, “Securing the root password”). |
Program activities can be monitored and controlled using specialized tools.
Table 9.10. List of tools for monitoring and controlling program activities
package | popcon | size | description |
---|---|---|---|
time
|
V:7, I:85 | 152 |
time (1) runs a program to report system resource usages with respect to time.
|
coreutils
|
V:91, I:99 | 12868 |
nice (1) runs a program with modified scheduling priority.
|
bsdutils
|
V:71, I:99 | 180 |
renice (1) modifies the scheduling priority of a running process.
|
powertop
|
V:0.6, I:11 | 424 |
powertop (1) gives information about system power use on Intel-based laptops.
|
procps
|
V:87, I:99 | 752 |
The "/proc " file system utilities: ps (1), top (1), kill (1), watch (1), …
|
psmisc
|
V:52, I:87 | 536 |
The "/proc " file system utilities: killall (1), fuser (1), pstree (1)
|
cron
|
V:91, I:99 | 324 |
The cron (8) daemon runs processes according to a schedule (in background).
|
at
|
V:54, I:83 | 220 |
at (1) or batch (1) commands run a job at a specified time or below certain load level.
|
lsof
|
V:16, I:91 | 444 |
lsof (8) lists open files by a running process using "-p " option.
|
strace
|
V:7, I:63 | 420 |
strace (1) traces system calls and signals.
|
ltrace
|
V:0.3, I:2 | 228 |
ltrace (1) traces library calls.
|
xtrace
|
V:0.02, I:0.15 | 204 |
xtrace (1) traces communication between X11 client and server.
|
Display time used by the process invoked by the command.
# time some_command >/dev/null real 0m0.035s # time on wall clock (elapsed real time) user 0m0.000s # time in user mode sys 0m0.020s # time in kernel mode
A nice value is used to control the scheduling priority for the process.
Table 9.11. List of nice values for the scheduling priority.
nice value | scheduling priority |
---|---|
19 | lowest priority process (nice) |
0 | very high priority process for user. |
-20 | very high priority process for root. (not-nice) |
# nice -19 top # very nice # nice --20 wodim -v -eject speed=2 dev=0,0 disk.img # very fast
Sometimes an extreme nice value does more harm than good to the system. Use this command carefully.
The ps
(1) command on the Debian support both BSD and SystemV features and helps to identify the process activity statically.
Table 9.12. List of ps command styles.
style | typical command | feature |
---|---|---|
BSD |
ps aux
|
display %CPU %MEM |
System V |
ps -efH
|
display PPID |
For the zombie (defunct) children process, you can kill them by the parent process ID identified in the (PPID
) field.
The pstree
(1) command display a tree of processes.
top
(1) on the Debian has rich features and helps to identify what process is acting funny dynamically.
Table 9.13. List of commands for top.
command key | response |
---|---|
h or ?
|
To show help. |
f
|
To set/reset display field. |
o
|
To reorder display field. |
F
|
To set sort key field. |
k
|
To kill a process. |
r
|
To renice a process. |
q
|
To quit the top command.
|
You can list all files opened by a process with a process ID (PID), e.g. 1 as:
$ sudo lsof -p 1
PID=1 is usually init
program.
You can trace program activity with strace
(1), ltrace
(1), or xtrace
(1) for system calls and signals, library calls, or communication between X11 client and server. For example:
$ sudo strace ls ...
You can also identify processes using files or sockets by fuser
(1). For example:
$ sudo fuser -v /var/log/mail.log USER PID ACCESS COMMAND /var/log/mail.log: root 2946 F.... syslogd
You see that file "/var/log/mail.log
" is open for writing by the syslogd
(8) command.
$ sudo fuser -v smtp/tcp USER PID ACCESS COMMAND smtp/tcp: Debian-exim 3379 F.... exim4
Now you know your system runs exim4
(8) to handle TCP connections to SMTP port (25).
watch
(1) executes a program repeatedly with a constant interval while showing its output in fullscreen.
$ watch w
This will display who is logged on to the system updated every 2 seconds.
There are several ways to repeat a command looping over files matching some condition, e.g. matching glob pattern "*.ext
".
for x in *.ext; do if [ -f "$x"]; then command "$x" ; fi; done
find
(1) and xargs
(1) combination:
find . -type f -maxdepth 1 -name '*.ext' -print0 | xargs -0 -n 1 command
find
(1) with "-exec
" option with a command:
find . -type f -maxdepth 1 -name '*.ext' -exec command '{}' \;
find
(1) with "-exec
" option with a short shell script:
find . -type f -maxdepth 1 -name '*.ext' -exec sh -c "command '{}' && echo 'successful'" \;
The above examples are written to ensure proper handling of funny file names such as ones containing spaces. See Section 10.1.5, “Idioms for the selection of files” for more advance uses of find
(1).
You can set up to start a process from graphical user interface (GUI).
Under GNOME desktop environment, a program program can be started with proper argument by drag-and-drop of an icon to the launcher icon or by "Open with …" menu with right clicking. KDE can do the equivalent, too. Here is an example for GNOME to set up mc
(1) started in gnome-terminal
(1):
mc-term
" as:
# cat >/usr/local/bin/mc-term <<EOF #!/bin/sh gnome-terminal -e "mc \$1" EOF # chmod 755 /usr/local/bin/mc-term
create a desktop launcher
Create Launcher …
"
Application
"
mc
"
mc-term %f
"
create an open-with association
Open with Other Application …
"
mc-term %f
"
![]() |
Tip |
---|---|
Launcher is a file at " |
Some programs start another program automatically. Here are check points for customizing this process:
application configuration menu:
mc
(1): "/etc/mc/mc.ext
"
$BROWSER
", "$EDITOR
", "$VISUAL
", and "$PAGER
" (see eviron
(7)).
update-alternatives
(8) system for programs such as "editor
", "view
", "x-www-browser
", "gnome-www-browser
", and "www-browser
" (see Section 1.4.7, “Setting a default text editor”).
~/.mailcap
" and "/etc/mailcap
" file contents which associate MIME type with program (see mailcap
(5)).
~/.mime.types
" and "/etc/mime.types
" file contents which associate file name extension with MIME type (see run-mailcap
(1)).
![]() |
Tip |
---|---|
|
![]() |
Tip |
---|---|
The |
![]() |
Tip |
---|---|
In order to run a console application such as |
# cat /usr/local/bin/mutt-term <<EOF #!/bin/sh gnome-terminal -e "mutt \$@" EOF chmod 755 /usr/local/bin/mutt-term
Use kill
(1) to kill (or send a signal to) a process by the process ID.
Use killall
(1) or pkill
(1) to do the same by the process command name and other attributes.
Table 9.14. List of frequently used signals for kill command.
signal value | signal name | function |
---|---|---|
1 | HUP | restart daemon |
15 | TERM | normal kill |
9 | KILL | kill hard |
Run the at
(1) command to schedule a one-time job:
$ echo 'command -args'| at 3:40 monday
Use cron
(8) to schedule tasks regularly. See crontab
(1) and crontab
(5).
Run the command "crontab -e
" to create or edit a crontab file to set up regularly scheduled events.
Example of a crontab file:
# use /bin/sh to run commands, no matter what /etc/passwd says SHELL=/bin/sh # mail any output to paul, no matter whose crontab this is MAILTO=paul # Min Hour DayOfMonth Month DayOfWeek command (Day... are OR'ed) # run at 00:05, every day 5 0 * * * $HOME/bin/daily.job >> $HOME/tmp/out 2>&1 # run at 14:15 on the first of every month -- output mailed to paul 15 14 1 * * $HOME/bin/monthly # run at 22:00 on weekdays(1-5), annoy Joe. % for newline, last % for cc: 0 22 * * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?%.%% 23 */2 1 2 * echo "run 23 minutes after 0am, 2am, 4am ..., on Feb 1" 5 4 * * sun echo "run at 04:05 every sunday" # run at 03:40 on the first Monday of each month 40 3 1-7 * * [ "$(date +%a)" == "Mon" ] && command -args
![]() |
Tip |
---|---|
For the system not running continuously, install the |
Insurance against system malfunction is provided by the kernel compile option "Magic SysRq key" (SAK key) which is now the default for the Debian kernel. Pressing Alt-SysRq followed by one of the following keys does the magic of rescuing control of the system:
Table 9.15. List of SAK command keys.
key following Alt-SysRq | function |
---|---|
r
|
Unraw restores the keyboard after things like X crashes. |
0
|
Changing the console loglevel to 0 reduces error messages. |
k
|
SAK (system attention key) kills all processes on the current virtual console. |
e
|
Send a SIGTERM to all processes, except for init (8).
|
i
|
Send a SIGKILL to all processes, except for init (8).
|
s
|
Sync all mounted filesystems. |
u
|
Remount all mounted filesystems read-only (umount). |
b
|
Reboot the system without syncing or unmounting. |
The combination of "Alt-SysRq s", "Alt-SysRq u", and "Alt-SysRq r" is good for getting out of really bad situations.
See "/usr/share/doc/linux-doc-2.6.*/Documentation/sysrq.txt.gz
".
![]() |
Caution |
---|---|
The Alt-SysRq feature may be considered a security risk by allowing users access to root-privileged functions. Placing " |
![]() |
Tip |
---|---|
From SSH terminal etc., you can use the Alt-SysRq feature by writing to the " |
You can send message to everyone who is logged on to the system with wall
(1):
$ echo "We are shutting down in 1 hour" | wall
For the PCI-like devices (AGP, PCI-Express, CardBus, ExpressCard, etc.), lspci
(8) (probably with "-nn
" option) is a good start for the hardware identification
Alternatively, you can identify the hardware by reading contents of "/proc/bus/pci/devices
" or browsing directory tree under "/sys/bus/pci
" (see Section 1.2.12, “procfs and sysfs”).
Table 9.16. List of hardware identification tools.
package | popcon | size | description |
---|---|---|---|
pciutils
|
V:16, I:92 | 780 |
Linux PCI Utilities, lspci (8)
|
usbutils
|
V:38, I:97 | 548 |
Linux USB utilities, lsusb (8)
|
pcmciautils
|
V:0.9, I:14 | 172 |
PCMCIA utilities for Linux 2.6, pccardctl (8)
|
scsitools
|
V:0.2, I:1.3 | 484 |
Collection of tools for SCSI hardware management, lsscsi (8)
|
pnputils
|
V:0.02, I:0.2 | 108 |
Plug and Play BIOS utilities, lspnp (8)
|
procinfo
|
V:0.5, I:4 | 164 |
Displays system information from "/proc ", lsdev (8)
|
lshw
|
V:1.1, I:6 | 804 |
Information about hardware configuration, lshw (1)
|
discover
|
V:4, I:14 | 928 |
Hardware identification system, discover (8)
|
Although most of the hardware configuration on modern GUI desktop systems such as GNOME and KDE can be managed through accompanying GUI configuration tools, it is a good idea to know some basics methods to configure them.
Table 9.17. List of hardware configuration tools.
package | popcon | size | description |
---|---|---|---|
hal
|
V:45, I:58 | 1844 |
Hardware Abstraction Layer, lshal (1)
|
console-tools
|
V:59, I:95 | 948 | Linux console font and keytable utilities. |
x11-xserver-utils
|
V:32, I:46 | 648 |
X server utilities. xset (1) and xmodmap (1).
|
acpid
|
V:55, I:90 | 200 | Daemon to manage events delivered by the Advanced Configuration and Power Interface (ACPI) |
acpi
|
V:3, I:33 | 92 | Utilities for ACPI devices |
apmd
|
V:1.2, I:12 | 144 | Daemon to manage events delivered by the Advanced Power Management (APM) |
powersaved
|
V:0.7, I:0.9 | 1800 | Daemon to manage battery, temperature, ac, cpufreq (SpeedStep, Powernow!) control and monitor with ACPI and APM supports. |
noflushd
|
V:0.07, I:0.14 | 244 | Allow idle hard disks to spin down |
sleepd
|
V:0.07, I:0.11 | 92 | Puts a laptop to sleep during inactivity |
hdparm
|
V:13, I:34 | 284 |
Hard disk access optimization. Very effective but dangerous. You must read hdparm (8) first.
|
smartmontools
|
V:6, I:18 | 828 | Control and monitor storage systems using S.M.A.R.T. |
setserial
|
V:2, I:4 | 176 | Collection of tools for serial port management. |
memtest86+
|
V:0.5, I:4 | 384 | Collection of tools for memory hardware management. |
scsitools
|
V:0.2, I:1.3 | 484 | Collection of tools for SCSI hardware management. |
tpconfig
|
V:0.4, I:0.5 | 208 | A program to configure touchpad devices |
setcd
|
V:0.08, I:0.4 | 28 | Compact disc drive access optimization. |
big-cursor
|
I:0.19 | 68 | Larger mouse cursors for X |
Here, ACPI is a newer framework for the power management system than APM.
The following will set system and hardware time to MM/DD hh:mm, CCYY.
# date MMDDhhmmCCYY # hwclock --utc --systohc # hwclock --show
Times are normally displayed in the local time on the Debian system but the hardware and system time usually use UTC.
If the hardware (BIOS) time is set to GMT, change the setting to "UTC=yes
" in the "/etc/default/rcS
".
If you wish to update system time via network, consider to use the NTP service with the packages such as ntp
, ntpdate
, and chrony
. See:
ntp-doc
package
![]() |
Tip |
---|---|
|
There are several components to configure character console and ncurses
(3) system features:
/etc/terminfo/*/*
" file (terminfo
(5))
$TERM
" environment variable (term
(7))
setterm
(1), stty
(1), tic
(1), and toe
(1)
If the terminfo
entry for xterm
doesn't work with a non-Debian xterm
, change your terminal type, "$TERM
", from "xterm
" to one of the feature-limited versions such as "xterm-r6
" when you log in to a Debian system remotely. See "/usr/share/doc/libncurses5/FAQ
" for more. "dumb
" is the lowest common denominator for "$TERM
".
Device drivers for sound cards for current Linux 2.6 are provided by Advanced Linux Sound Architecture (ALSA). ALSA provides emulation mode for previous Open Sound System (OSS) for compatibility.
Run "dpkg-reconfigure linux-sound-base
" to select the sound system to use ALSA via blacklisting of kernel modules. Unless you have very new sound hardware, udev infrastructure should configure your sound system.
![]() |
Tip |
---|---|
Use " |
![]() |
Tip |
---|---|
If you can not get sound, your speaker may be connected to a muted output. Modern sound system has many outputs. |
Application softwares may be configured not only to access sound devices directly but also to access them via some standardized sound server system.
Table 9.18. List of sound packages
There is usually a common sound engine for each popular desktop environment. Each sound engine used by the application can choose to connect to different sound servers.
For disabling the screen saver, use following commands.
Table 9.19. List of commands for disabling the screen saver.
environment | command |
---|---|
The Linux console |
setterm -powersave off
|
The X Window by turning off screensaver |
xset s off
|
The X Window by disabling dpms |
xset -dpms
|
The X Window by GUI configuration of screen saver |
xscreensaver-command -prefs
|
One can always unplug the PC speaker. ;-) Removing pcspkr
kernel module does this for you.
The following will prevent the readline
(3) program used by bash
(1) to beep when encountering "\a
" (ASCII=7):
$ echo "set bell-style none">> ~/.inputrc
The kernel boot message in the "/var/log/dmesg
" contains the total exact size of available memory.
free
(1) and top
(1) display information on memory resources on the running system.
$ grep '\] Memory' /var/log/dmesg [ 0.004000] Memory: 990528k/1016784k available (1975k kernel code, 25868k reserved, 931k data, 296k init) $ free -k total used free shared buffers cached Mem: 997184 976928 20256 0 129592 171932 -/+ buffers/cache: 675404 321780 Swap: 4545576 4 4545572
For my MacBook with 1GB=1048576k DRAM (video system steals some of this):
Table 9.20. List of memory sizes reported.
report | size |
---|---|
Total size in dmesg | 1016784k = 1GB - 31792k |
Free in dmesg | 990528k |
Total under shell | 997184k |
Free under shell | 20256k |
Do not worry about the large size of "used
" and the small size of "free
" in the "Mem:
" line, but read the one under them (675404 and 321780 in the example below) and relax.
Poor system maintenance may expose your system to external exploitation.
For system security and integrity check, you should start with:
debsums
package: See debsums
(1) and Section 2.5.2, “Top level "Release" file and authenticity”.
chkrootkit
package: See chkrootkit
(1).
clamav
package family: See clamscan
(1) and freahclam
(1).
Table 9.21. List of tools for system security and integrity check
package | popcon | size | description |
---|---|---|---|
logcheck
|
V:3, I:4 | 264 | This mails anomalies in the system logfiles to the administrator |
debsums
|
V:2, I:3 | 264 | This verifies installed package files against MD5 checksums. |
chkrootkit
|
V:2, I:6 | 864 | Rootkit detector. |
clamav
|
V:2, I:11 | 504 | Anti-virus utility for Unix - command-line interface. |
tiger
|
V:0.8, I:1.0 | 3088 | Report system security vulnerabilities |
tripwire
|
V:0.6, I:0.8 | 5020 | File and directory integrity checker |
john
|
V:0.5, I:2 | 476 | Active password cracking tool |
aide
|
V:0.3, I:0.5 | 1112 | Advanced Intrusion Detection Environment - static binary |
bastille
|
V:0.19, I:0.5 | 1960 | Security hardening tool |
integrit
|
V:0.10, I:0.2 | 440 | A file integrity verification program |
crack
|
V:0.05, I:0.2 | 204 | Password guessing program |
Here is a simple script to check for typical world writable incorrect file permissions.
# find / -perm 777 -a \! -type s -a \! -type l -a \! \( -type d -a -perm 1777 \)
![]() |
Caution |
---|---|
Since the |
Debian distributes modularized Linux kernel as packages for supported architectures.
There are few notable features on Linux kernel 2.6 compared to 2.4.
ide-scsi
module.
iptable
kernel modules.
Most normal programs don't need kernel headers and in fact may break if you use them directly for compiling. They should be compiled against the headers in "/usr/include/linux
" and "/usr/include/asm
" provided by the libc6-dev
package (created from the glibc
source package) on the Debian system.
![]() |
Note |
---|---|
For compiling some kernel-specific programs such as the kernel modules from the external source and the automounter daemon ( |
Debian has its own method of compiling the kernel and related modules.
Table 9.22. List of key packages to be installed for the kernel recompilation on the Debian system
package | popcon | size | description |
---|---|---|---|
build-essential
|
I:45 | 48 |
essential packages for building Debian packages: make , gcc , …
|
bzip2
|
V:57, I:80 | 132 | compress and decompress utilities for bz2 files |
libncurses5-dev
|
V:4, I:27 | 6724 | developer's libraries and docs for ncurses |
git-core
|
V:6, I:10 | 14344 | git: distributed revision control system used by the Linux kernel |
fakeroot
|
V:4, I:27 | 444 | provide fakeroot environment for building package as non-root |
initramfs-tools
|
V:35, I:97 | 420 | tool to build an initramfs (Debian specific) |
kernel-package
|
V:2, I:18 | 2340 | tool to build Linux kernel packages (Debian specific) |
module-assistant
|
V:4, I:22 | 540 | tool to help build module packages (Debian specific) |
devscripts
|
V:2, I:13 | 1656 | helper scripts for a Debian Package maintainer (Debian specific) |
linux-tree-2.6.*
|
N/A | N/A | Linux kernel source tree meta package (Debian specific) |
If you use initrd
in Section 3.3, “Stage 2: the boot loader”, make sure to read the related information in initramfs-tools
(8), update-initramfs
(8), mkinitramfs
(8) and initramfs.conf
(5).
![]() |
Warning |
---|---|
Do not put symlinks to the directories in the source tree (e.g. " |
![]() |
Note |
---|---|
When compiling the latest Linux kernel on the Debian |
The Debian standard method for compiling kernel source to create a custom kernel package uses make-kpkg
(1). The official documentation is in (the bottom of) "/usr/share/doc/kernel-package/README.gz
". See kernel-pkg.conf
(5) and kernel-img.conf
(5) for customization.
Here is an example for amd64 system:
# aptitude install linux-tree-<version> $ cd /usr/src $ tar -xjvf linux-source-<version>.tar.bz2 $ cd linux-source-<version> $ cp /boot/config-<oldversion> .config $ make menuconfig ... $ make-kpkg clean $ fakeroot make-kpkg --append_to_version -amd64 --initrd --revision=rev.01 kernel_image modules_image $ cd .. # dpkg -i linux-image*.deb
shutdown -r now
" .
![]() |
Caution |
---|---|
When you intend to create a non-modularized kernel compiled only for one machine, invoke |
The Debian standard method for creating and installing a custom module package for a custom kernel package uses module-assistant
(8) and module-source packages. For example, following will build the unionfs
kernel module package and installs it.
$ sudo aptitude install module-assistant ... $ sudo aptitude install unionfs-source unionfs-tools unionfs-utils $ sudo m-a update $ sudo m-a prepare $ sudo m-a auto-install unionfs ... $ sudo apt-get autoremove
You can still build Linux kernel from the pristine sources with the classic method. You must take care the details of the system configuration manually.
$ cd /usr/src $ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-<version>.tar.bz2 $ tar -xjvf linux-<version>.tar.bz2 $ cd linux-<version> $ cp /boot/config-<version> .config $ make menuconfig ... $ make dep; make bzImage $ make modules # cp ./arch/x86_64/boot/bzImage /boot/vmlinuz-<version> # make modules_install # depmod -a # update-initramfs -c -k <version>
set up bootloader
/etc/lilo.conf
" and run "/sbin/lilo
", if you use lilo
.
/boot/grub/menu.lst
", if you use grub
.
shutdown -r now
".
Although most of hardware drivers are available as free software and as a part of the Debian system, you may need to load some non-free external drivers to support some hardwares, such as Winmodem, on your system.
Check pertinent resources:
Use of virtualized system enables us to run multiple instances of system simultaneously on a single hardware.
Virturization involves 2 steps:
debootstrap
and cdebootstrap
helps this process.
There are several system virtualization and emulation related packages in Debian beyond simple chroot. Some packages also help you to setup such system.
Table 9.23. List of virtualization tools
package | pocon | size | description |
---|---|---|---|
schroot
|
V:1.0, I:1.6 | 1988 | Specialized tool for executing Debian binary packages in chroot |
sbuild
|
V:0.07, I:0.3 | 408 | Tool for building Debian binary packages from Debian sources |
pbuilder
|
V:0.4, I:2 | 1112 | Personal package builder for Debian packages |
debootstrap
|
V:1.7, I:12 | 260 | Bootstrap a basic Debian system (written in sh) |
cdebootstrap
|
V:0.5, I:2 | 112 | Bootstrap a Debian system (written in C) |
rootstrap
|
V:0.03, I:0.2 | 156 | A tool for building complete Linux filesystem images |
user-mode-linux
|
V:0.10, I:0.5 | 17828 | User-mode Linux (kernel) |
xen-tools
|
V:0.3, I:2 | 996 | Tools to manage debian XEN virtual server |
bochs
|
V:0.09, I:0.5 | 3360 | Bochs: IA-32 PC emulator |
qemu
|
V:1.2, I:6 | 43984 | Qemu: fast generic processor emulator |
virtualbox-ose
|
V:2, I:3 | 22944 | VirtualBox: x86 virtualization solution on i386 and amd64 |
wine
|
V:1.6, I:16 | 64 | Wine: Windows API Implementation (standard suite) |
dosbox
|
V:0.6, I:3 | 2240 | DOSBox: x86 emulator with Tandy/Herc/CGA/EGA/VGA/SVGA graphics, sound and DOS |
util-vserver
|
V:0.8, I:1.1 | 2388 | Linux-VServer virtual private servers - user-space tools |
vzctl
|
V:0.5, I:1.0 | 1148 | OpenVZ server virtualization solution - control tools |
vzquota
|
V:0.5, I:1.0 | 272 | OpenVZ server virtualization solution - quota tools |
See Wikipedia article Comparison of virtual machines for detail comparison of different virtualization solutions.
chroot
(8) offers most basic way to run different instances of the GNU/Linux environment on a single system simultaneously without rebooting.
![]() |
Caution |
---|---|
Examples below assumes both parent system and chroot system share the same CPU architecture. |
You can learn how to setup and use chroot
(8) by running pbuilder
(8) program under script
(1) as follows.
$ sudo mkdir /sid-root $ sudo pbuilder --create --no-targz --debug --buildplace /sid-root
You will see how debootstrap
(8) or debootstrap
(1) populate system data for sid
environment under "/sid-root
".
![]() |
Tip |
---|---|
These |
$ sudo pbuilder --login --no-targz --debug --buildplace /sid-root
You will see how a system shell running under sid
environment is created:
"/etc/hosts
", "/etc/hostname
", "/etc/resolv.conf
")
/proc
" filesystem
/dev/pts
" filesystem
/usr/sbin/policy-rc.d
" created (this always exits with 101)
chroot /sid-root bin/bash -c 'exec -a -bash bin/bash'
"
![]() |
Note |
---|---|
Some programs under chroot may require access to more files from the parent system to function than |
![]() |
Note |
---|---|
The " |
![]() |
Tip |
---|---|
The original purpose of the specialized chroot package, |
![]() |
Tip |
---|---|
Similar |
![]() |
Tip |
---|---|
By installing a system into a separate partition using the installer of other distributions and using this system for |
You can run another login process on a separate virtual terminal where you can log in to the chroot system directly. Since on default Debian systems tty1
to tty6
run Linux consoles and tty7
runs the X Window System, let's set up tty8
for a chrooted console as an example. After creating a sid
chroot system under "/sid-root
" by following steps you learned from pbuilder
(8), type from the root shell of the main system:
main # echo "8:23:respawn:/usr/sbin/chroot /sid-root /sbin/getty 38400 tty8" >> /etc/inittab main # init q
Tools and tips for managing binary and text data on the Debian system are described.
The security of the data and its controlled sharing have several aspects:
These can be realized by using some combination of:
Here is a summary of archive and compression tools available on the Debian system:
Table 10.1. List of archive and compression tools.
package | popcon | size | command | comment | extension |
---|---|---|---|---|---|
tar
|
V:63, I:99 | 2456 |
tar (1)
|
the standard archiver (de facto standard) |
.tar
|
cpio
|
V:34, I:99 | 664 |
cpio (1)
|
Unix System V style archiver, use with find (1)
|
.cpio
|
binutils
|
V:48, I:79 | 9036 |
ar (1)
|
archiver for the creation of static libraries |
.ar
|
fastjar
|
V:4, I:39 | 220 |
fastjar (1)
|
archiver for Java (zip like) |
.jar
|
pax
|
V:1.5, I:5 | 156 |
pax (1)
|
new POSIX standard archiver, compromise between tar and cpio
|
.pax
|
afio
|
V:0.3, I:1.6 | 240 |
afio (1)
|
extended cpio with per-file compression etc.
|
.afio
|
par2
|
V:0.5, I:1.7 | 284 |
par2 (1)
|
Parity Archive Volume Set, for checking and repair of files |
.par2
|
gzip
|
V:91, I:99 | 292 |
gzip (1), zcat (1), …
|
GNU LZ77 compression utility (de facto standard) |
.gz
|
bzip2
|
V:57, I:80 | 132 |
bzip2 (1), bzcat (1), …
|
Burrows-Wheeler block-sorting compression utility with higher compression ratio than gzip (1) (slower than gzip with similar syntax)
|
.bz2
|
lzma
|
V:9, I:64 | 172 |
lzma (1)
|
LZMA compression utility with higher compression ratio than gzip (1) (slower than gzip with similar syntax)
|
.lzma
|
p7zip
|
V:3, I:25 | 996 |
7zr (1), p7zip (1)
|
7-Zip file archiver with high compression ratio (LZMA compression) |
.7z
|
p7zip-full
|
V:10, I:20 | 3400 |
7z (1), 7za (1)
|
7-Zip file archiver with high compression ratio (LZMA compression and others) |
.7z
|
lzop
|
V:1.0, I:8 | 144 |
lzop (1)
|
LZO compression utility with higher compression and decompression speed than gzip (1) (lower compression ratio than gzip with similar syntax)
|
.lzo
|
zip
|
V:9, I:61 | 628 |
zip (1)
|
InfoZIP: DOS archive and compression tool |
.zip
|
unzip
|
V:23, I:72 | 384 |
unzip (1)
|
InfoZIP: DOS unarchive and decompression tool |
.zip
|
![]() |
Warning |
---|---|
Do not set the " |
![]() |
Note |
---|---|
The gzipped |
![]() |
Note |
---|---|
|
![]() |
Note |
---|---|
|
![]() |
Note |
---|---|
|
![]() |
Note |
---|---|
Internal structure of OpenOffice data files are " |
Here is a summary of simple copy and backup tools available on the Debian system:
Table 10.2. List of copy and synchronization tools.
package | popcon | size | tool | function |
---|---|---|---|---|
coreutils
|
V:91, I:99 | 12868 | GNU cp | Locally copy files and directories ("-a" for recursive). |
openssh-client
|
V:55, I:98 | 2084 | scp |
Remotely copy files and directories (client). "-r " for recursive.
|
openssh-server
|
V:65, I:77 | 812 | sshd | Remotely copy files and directories (remote server). |
rsync
|
V:16, I:40 | 640 | - | 1-way remote synchronization and backup. |
unison
|
V:0.9, I:3 | 1644 | - | 2-way remote synchronization and backup. |
pdumpfs
|
V:0.06, I:0.18 | 148 | - |
Daily local backup using hardlinks, similar to Plan9's dumpfs .
|
![]() |
Tip |
---|---|
Execution of the |
![]() |
Tip |
---|---|
Version control system (VCS) tools in Table 10.15, “List of version control system tools.” can function as the multi-way copy and synchronization tools. |
Here are several ways to archive and unarchive the entire contents of the directory "/source
".
With GNU tar
(1):
$ tar cvzf archive.tar.gz /source $ tar xvzf archive.tar.gz
With cpio
(1):
$ find /source -xdev -print0 | cpio -ov --null > archive.cpio; gzip archive.cpio $ zcat archive.cpio.gz | cpio -i
With afio
(1):
$ find /source -xdev -print0 | afio -ovZ0 archive.afio $ afio -ivZ archive.afio
Here are several ways to copy the entire contents of the directory
/source
" to "/dest
", and
/source
" at local to "/dest
" at "user@host.dom
".
With GNU cp
(1) and openSSH scp
(1):
# cp -a /source /dest # scp -pr /source user@host.dom:/dest
With GNU tar
(1):
# (cd /source && tar cf - . ) | (cd /dest && tar xvfp - ) # (cd /source && tar cf - . ) | ssh user@host.dom '(cd /dest && tar xvfp - )'
With cpio
(1):
# cd /source; find . -print0 | cpio -pvdm --null --sparse /dest
With afio
(1):
# cd /source; find . -print0 | afio -pv0a /dest
scp
(1) can even copy files between remote hosts:
# scp -pr user1@host1.dom:/source user2@host2.dom:/dest
find
(1) is used to select files for archive and copy commands (see Section 10.1.3, “Idioms for the archive” and Section 10.1.4, “Idioms for the copy”) or for xargs
(1) (see Section 9.5.9, “Repeating a command looping over files”). This can be enhanced by using its command arguments.
Basic syntax of find
(1) can be summarized as:
-o
" between conditionals) has lower precedence than "logical AND" (specified by "-a
" or nothing between conditionals).
!
" before a conditional) has higher precedence than "logical AND".
-prune
" always returns logical TRUE and, if it is a directory, searching of file is stopped beyond this point.
-name
" matches the base of the filename with shell glob (see Section 1.5.3, “Shell glob”) but it also matches its initial ".
" with metacharacters such as "*
" and "?
". (New POSIX feature)
-regex
" matches the full path with emacs style BRE (see Section 1.6.2, “Regular expressions”) as default.
-size
" matches the file based on the file size (value precedented with "+
" for larger, precedented with "-
" for smaller)
-newer
" matches the file newer than the one specified in its argument.
-print0
" always returns logical TRUE and print the full filename (null terminated) on the standard output.
find
(1) is often used with an idiomatic style. For example:
# find /path/to \ -xdev -regextype posix-extended \ -type f -regex ".*\.afio|.*~" -prune -o \ -type d -regex ".*/\.git" -prune -o \ -type f -size +99M -prune -o \ -type f -newer /path/to/timestamp -print0
This means to do following actions:
/path/to
"
.*\.afio
" or ".*~
" from search by stop processing,
.*/\.git
" from search by stop processing,
/path/to/timestamp
".
Please note the idiomatic use of "-prune -o
" to exclude files in the above example.
![]() |
Note |
---|---|
For non-Debian Unix-like system, some options may not be supported for |
We all know that computers fail sometime or human errors cause system and data damages. Backup and recovery operations are the essential part of successful system administration. All possible failure modes will hit you some day.
There are 3 key factors which determine actual backup and recovery policy:
Knowing what to backup and recover.
~/
"
/var/
" (except "/var/cache/
", "/var/run/
", and "/var/tmp/
").
/etc/
"
/usr/local/
" or "/opt/
"
Knowing how to backup and recover.
Assessing risks and costs involved.
As for secure storage of data, data should be at least on different disk partitions preferably on different disks and machines to withstand the filesystem corruption. Important data are best stored on a write-once media such as CD/DVD-R to prevent overwrite accidents. (See Section 10.2, “The binary data” for how to write to the storage media from the shell commandline. GNOME desktop GUI environment gives you easy access via menu: "Places→CD/DVD Creator".)
![]() |
Note |
---|---|
You may wish to stop some application daemons such as MTA (see Section 6.2.5.1, “MTA”) while backing up data. |
![]() |
Note |
---|---|
You should pay extra care to the backup and restoration of identity related data files such as " |
![]() |
Note |
---|---|
If you run a cron job as a user process, you need to restart it after the system restoration. See Section 9.5.14, “Schedule tasks regularly” for |
Here is a select list of notable backup utility suites available on the Debian system:
Table 10.3. List of backup suite utilities.
package | popcon | size | description |
---|---|---|---|
rdiff-backup
|
V:1.3, I:3 | 764 | (remote) incremental backup |
dump
|
V:0.4, I:1.6 | 620 |
4.4BSD dump (8) and restore (8) for ext2/ext3 filesystems
|
xfsdump
|
V:0.3, I:1.8 | 684 |
Dump and restore with xfsdump (8) and xfsrestore (8) for XFS filesystem on GNU/Linux and IRIX
|
backupninja
|
V:0.4, I:0.5 | 408 | lightweight, extensible meta-backup system |
mondo
|
V:0.13, I:0.8 | 1172 | Mondo Rescue: disaster recovery backup suite |
sbackup
|
V:0.09, I:0.2 | 488 | Simple Backup Suite for GNOME desktop |
keep
|
V:0.2, I:0.5 | 1196 | backup system for KDE |
bacula-common
|
V:1.1, I:2 | 832 | Bacula: network backup, recovery and verification - common support files |
bacula-client
|
I:0.9 | 60 | Bacula: network backup, recovery and verification - client meta-package |
bacula-console
|
V:0.3, I:1.2 | 340 | Bacula: network backup, recovery and verification - text console |
bacula-server
|
I:0.6 | 60 | Bacula: network backup, recovery and verification - server meta-package |
amanda-common
|
V:0.4, I:0.9 | 3120 | Amanda: Advanced Maryland Automatic Network Disk Archiver (Libs) |
amanda-client
|
V:0.3, I:0.8 | 560 | Amanda: Advanced Maryland Automatic Network Disk Archiver (Client) |
amanda-server
|
V:0.14, I:0.3 | 1264 | Amanda: Advanced Maryland Automatic Network Disk Archiver (Server) |
cdrw-taper
|
V:0.00, I:0.06 | 172 | taper replacement for Amanda to support backups to CD-RW or DVD+RW |
backuppc
|
V:0.7, I:0.8 | 2082 | BackupPC is a high-performance, enterprise-grade system for backing up PCs (disk based) |
backup-manager
|
V:0.4, I:0.5 | 640 | command-line backup tool |
backup2l
|
V:0.2, I:0.3 | 140 | low-maintenance backup/restore tool for mountable media (disk based) |
faubackup
|
V:0.2, I:0.2 | 156 | backup system using a filesystem for storage (disk based) |
Basic tools such as ones descrived in Section 10.1.1, “Archive and compression tools” and Section 10.1.2, “Copy and synchronization tools” are used to facilitate system backup. In addition to these, rdiff-backup
and dump
packages also facilitate system backup.
rdiff-backup
package facilitates backs up of one directory to another, possibly over a network.
dump
package facilitates restoration of complete system by archiving filesystems themselves. This can perform incremental backup of filesystems efficiently. See files in "/usr/share/doc/dump/
" and "Is dump really deprecated?".
xfsdump
package works similarly to the dump
package.
There are programs for backup using these basic tools as their backends.
sbackup
and keep
packages provide easy GUI frontend to regular backups of user data for desktop users. An equivalent function can be realized by a simple script (Section 10.1.8, “An example script for the system backup”) and cron
(8).
For a personal Debian desktop system running unstable
suite, I only need to protect personal and critical data. I reinstall system once a year anyway. Thus I see no reason to backup the whole system or to install a full featured backup utility.
I use a simple script to make a backup archive and burn it into CD/DVD using GUI. Here is an example script for this.
#!/bin/sh -e # Copyright (C) 2007-2008 Osamu Aoki <osamu@debian.org>, Public Domain BUUID=1000; USER=osamu # UID and name of a user who accesses backup files BUDIR="/var/backups" XDIR0=".+/Mail|.+/Desktop" XDIR1=".+/\.thumbnails|.+/\.?Trash|.+/\.?[cC]ache|.+/\.gvfs|.+/sessions" XDIR2=".+/CVS|.+/\.git|.+/\.svn|.+/Downloads|.+/Archive|.+/Checkout|.+/tmp" XSFX=".+\.iso|.+\.tgz|.+\.tar\.gz|.+\.tar\.bz2|.+\.afio|.+\.tmp|.+\.swp|.+~" SIZE="+99M" DATE=$(date --utc +"%Y%m%d-%H%M") [ -d "$BUDIR" ] || mkdir -p "BUDIR" umask 077 dpkg --get-selections \* > /var/lib/dpkg/dpkg-selections.list debconf-get-selections > /var/cache/debconf/debconf-selections { find /etc /usr/local /opt /var/lib/dpkg/dpkg-selections.list \ /var/cache/debconf/debconf-selections -xdev -print0 find /home/$USER /root -xdev -regextype posix-extended \ -type d -regex "$XDIR0|$XDIR1" -prune -o -type f -regex "$XSFX" -prune -o \ -type f -size "$SIZE" -prune -o -print0 find /home/$USER/Mail/Inbox /home/$USER/Mail/Outbox -print0 find /home/$USER/Desktop -xdev -regextype posix-extended \ -type d -regex "$XDIR2" -prune -o -type f -regex "$XSFX" -prune -o \ -type f -size "$SIZE" -prune -o -print0 } | cpio -ov --null -O $BUDIR/BU$DATE.cpio chown $BUUID $BUDIR/BU$DATE.cpio touch $BUDIR/backup.stamp
This is meant to be a script example executed from root:
find … -print0
" with "find … -newer $BUDIR/backup.stamp -print0
" to make a differential backup.
scp
(1) or rsync
(1) or burn them to CD/DVD for extra data security. (I use GNOME desktop GUI for burning CD/DVD. See Section 12.1.8, “Shell script example with zenity” for extra redundancy.)
![]() |
Tip |
---|---|
You can recover debconf configuration data with " |
For the set of data under a directory tree, the copy with "cp -a
" provides the normal backup.
For the set of large non-overwritten static data under a directory tree such as the data under the "/var/cache/apt/packages/
" directory, hardlinks with "cp -al
" provide an alternative to the normal backup with efficient use of the disk space.
Here is a copy script, which I named as bkup
, for the data backup. This script copies all (non-VCS) files under the current directory to the dated directory on the parent directory or on a remote host.
#!/bin/sh -e # Copyright (C) 2007-2008 Osamu Aoki <osamu@debian.org>, Public Domain function fdot(){ find . -type d \( -iname ".?*" -o -iname "CVS" \) -prune -o -print0;} function fall(){ find . -print0;} function mkdircd(){ mkdir -p "$1";chmod 700 "$1";cd "$1">/dev/null;} FIND="fdot";OPT="-a";MODE="CPIOP";HOST="localhost";EXTP="$(hostname -f)" BKUP="$(basename $(pwd)).bkup";TIME="$(date +%Y%m%d-%H%M%S)";BU="$BKUP/$TIME" while getopts gcCsStrlLaAxe:h:T f; do case $f in g) MODE="GNUCP";; # cp (GNU) c) MODE="CPIOP";; # cpio -p C) MODE="CPIOI";; # cpio -i s) MODE="CPIOSSH";; # cpio/ssh S) MODE="AFIOSSH";; # afio/ssh t) MODE="TARSSH";; # tar/ssh r) MODE="RSYNCSSH";; # rsync/ssh l) OPT="-alv";; # hardlink (GNU cp) L) OPT="-av";; # copy (GNU cp) a) FIND="fall";; # find all A) FIND="fdot";; # find non CVS/ .???/ x) set -x;; # trace e) EXTP="${OPTARG}";; # hostname -f h) HOST="${OPTARG}";; # user@remotehost.example.com T) MODE="TEST";; # test find mode \?) echo "use -x for trace." esac; done shift $(expr $OPTIND - 1) if [ $# -gt 0 ]; then for x in $@; do cp $OPT $x $x.$TIME; done elif [ $MODE = GNUCP ]; then mkdir -p "../$BU";chmod 700 "../$BU";cp $OPT . "../$BU/" elif [ $MODE = CPIOP ]; then mkdir -p "../$BU";chmod 700 "../$BU" $FIND|cpio --null --sparse -pvd ../$BU elif [ $MODE = CPIOI ]; then $FIND|cpio -ov --null | ( mkdircd "../$BU"&&cpio -i ) elif [ $MODE = CPIOSSH ]; then $FIND|cpio -ov --null|ssh -C $HOST "( mkdircd \"$EXTP/$BU\"&&cpio -i )" elif [ $MODE = AFIOSSH ]; then $FIND|afio -ov -0 -|ssh -C $HOST "( mkdircd \"$EXTP/$BU\"&&afio -i - )" elif [ $MODE = TARSSH ]; then (tar cvf - . )|ssh -C $HOST "( mkdircd \"$EXTP/$BU\"&& tar xvfp - )" elif [ $MODE = RSYNCSSH ]; then rsync -rlpt ./ "${HOST}:${EXTP}-${BKUP}-${TIME}" else echo "Any other idea to backup?" $FIND |xargs -0 -n 1 echo fi
This is meant to be command examples. Please read script and test it by yourself.
![]() |
Tip |
---|---|
I keep this |
![]() |
Tip |
---|---|
For making snapshot history of a source file tree or a configuration file tree, it is easier and space efficient to use |
Removable mass storage devices may be any one of
These removable mass storage devices can be automatically mounted as a user under modern desktop environment, such as GNOME using gnome-mount
(1).
Mount point under GNOME is chosen as "/media/<disk_label>
" which can be customized
mlabel
(1) for FAT filesystem,
genisoimage
(1) with "-V
" option for ISO9660 filesystem, and
tune2fs
(1) with "-L
" option for ext2/ext3 filesystem.
![]() |
Note |
---|---|
Automounting under modern desktop environment happens only when those removable media devices are not listed in " |
![]() |
Tip |
---|---|
When providing wrong mount option causes problem, erase its corresponding setting under " |
Table 10.4. List of packages which permit normal users to mount removable devices without a matching "/etc/fstab
" entry.
package | popcon | size | description |
---|---|---|---|
gnome-mount
|
V:23, I:38 | 968 | wrapper for (un)mounting and ejecting storage devices (used by GNOME) |
pmount
|
V:12, I:37 | 868 | mount removable devices as normal user (used by KDE) |
cryptmount
|
V:0.09, I:0.5 | 304 | Management and user-mode mounting of encrypted file systems |
usbmount
|
I:1.9 | 108 | automatically mount and unmount USB mass storage devices |
When sharing data with other system via removable mass storage device, you should format it with common filesystem supported by both systems. Here is a list of filesystem choices.
Table 10.5. List of filesystem choices for removable storage devices with typical usage scenarios.
filesystem | typical usage scenario |
---|---|
FAT12 | Cross platform sharing of data on the floppy disk. (⇐32MiB) |
FAT16 | Cross platform sharing of data on the small harddisk like device. (⇐2GiB) |
FAT32 | Cross platform sharing of data on the large harddisk like device. (⇐8TiB, supported by newer than MS Windows95 OSR2) |
NTFS | Cross platform sharing of data on the large harddisk like device. (supported natively on MS Windows NT and later version, and supported by NTFS-3G via FUSE on Linux) |
ISO9660 | Cross platform sharing of static data on CD-R and DVD+/-R |
UDF | Incremental data writing on CD-R and DVD+/-R (new) |
MINIX filesystem | Space efficient unix file data storage on the floppy disk. |
ext2 filesystem | Sharing of data on the harddisk like device with older Linux systems. |
ext3 filesystem | Sharing of data on the harddisk like device with current Linux systems. (Journaling file system) |
![]() |
Tip |
---|---|
See Section 9.4.1, “Removable disk encryption with dm-crypt/LUKS” for cross platform sharing of data using device level encryption. |
The FAT filesystem is supported by almost all modern operating systems and is quite useful for the data exchange purpose via removable harddisk like media (.
When formatting removable harddisk like devices for cross platform sharing of data with the FAT filesystem, the following should be safe choices:
Partitioning them with fdisk
(8), cfdisk
(8) or parted
(8) (see Section 9.3.1, “Partition configuration”) into a single primary partition and to mark it as:
Formatting the primary partition with mkfs.vfat
(8)
/dev/sda1
" for FAT16, or
-F 32 /dev/sda1
" for FAT32.
When using the FAT or ISO9660 filesystems for sharing data, the following should be the safe considerations:
tar
(1), cpio
(1), or afio
(1) to retain the long filename, the symbolic link, the original Unix file permission and the owner information.
split
(1) command to protect it from the file size limitation.
![]() |
Note |
---|---|
For FAT filesystems by its design, the maximum file size is |
![]() |
Note |
---|---|
Microsoft itself does not recommend to use FAT for drives or partitions of over 200 MB. Microsoft highlights its short comings such as inefficient disk space usage in their "Overview of FAT, HPFS, and NTFS File Systems". Of course for the Linux, we should normally use the ext3 filesystem. |
![]() |
Tip |
---|---|
For more on filesystems and accessing filesystems, please read "Filesystems HOWTO". |
When sharing data with other system via network, you should use common service. Here are some hints.
Table 10.6. List of the network service to chose with the typical usage scenario.
network service | typical usage scenario |
---|---|
SMB/CIFS network mounted filesystem with Samba |
Sharing files via "Microsoft Windows Network". See smb.conf (5) and The Official Samba 3.2.x HOWTO and Reference Guide or the samba-doc package.
|
NFS network mounted filesystem with the Linux kernel |
Sharing files via "Unix/Linux Network". See exports (5) and Linux NFS-HOWTO.
|
HTTP service | Sharing file between the web server/client. |
HTTPS service | Sharing file between the web server/client with encrypted Secure Sockets Layer (SSL) or Transport Layer Security (TLS). |
FTP service | Sharing file between the FTP server/client. |
Although these filesystems mounted over network or file transfer methods over network are quite convenient for sharing data, these may be insecure. Their network connection must be secured by:
When choosing computer data storage media for important data archive, you should be careful about their limitations. For small personal data back up, I use CD-R and DVD-R by the brand name company and store in a cool, dry, clean environment. (Tape archive media seem to be popular for professional use.)
![]() |
Note |
---|---|
A fire-resistant safe are usually meant for paper documents. Most of the computer data storage media have less temperature tolerance than paper. I usually rely on multiple secure encrypted copies stored in multiple secure locations. |
Optimistic storage life of archive media seen on the net (mostly from vendor info):
Optimistic write cycle of archive media seen on the net (mostly from vendor info):
![]() |
Caution |
---|---|
Figures of storage life and write cycle here should not be used for decisions on any critical data storage. Please consult the specific product information provided by the manufacture. |
![]() |
Tip |
---|---|
Since CD/DVD-R and paper have only 1 write cycle, they inherently prevent accidental data loss by overwriting. This is advantage! |
![]() |
Tip |
---|---|
If you need fast and frequent backup of large amount of data, a harddisk on a remote host linked by a fast network connection, may be the only realistic option. |
Here, we discuss direct manipulation of the binary data on storage media. See Section 9.3, “Data storage tips”, too.
The disk image file, "disk.img
", of an unmounted device, e.g., the second SCSI drive "/dev/sdb
", can be made using cp
(1) or dd
(1):
# cp /dev/sdb disk.img # dd if=/dev/sdb of=disk.img
The disk image of the traditional PC's master boot record (MBR) (see Section 9.3.1, “Partition configuration”) which reside on the first sector on the primary IDE disk partial disk can be made by using dd
(1):
# dd if=/dev/hda of=mbr.img bs=512 count=1 # dd if=/dev/hda of=mbr-nopart.img bs=446 count=1 # dd if=/dev/hda of=mbr-part.img skip=446 bs=1 count=66
mbr.img
": the MBR with the partition table.
mbr-nopart.img
": the MBR without the partition table.
part.img
": the partition table of the MBR only..
If you have a SCSI device (including the new serial ATA drive) as the boot disk, substitute "/dev/hda
" with "/dev/sda
".
If you are making an image of a disk partition of the original disk, substitute "/dev/hda
" with "/dev/hda1
" etc.
The disk image file, "disk.img
" can be written to an unmounted device, e.g., the second SCSI drive "/dev/sdb
" with matching size, by dd
(1):
# dd if=disk.img of=/dev/sdb
Similarly, the disk partition image file, "disk.img
" can be written to an unmounted partition, e.g., the first partition of the second SCSI drive "/dev/sdb1
" with matching size, by dd
(1):
# dd if=disk.img of=/dev/sdb1
The most basic viewing method of binary data is to use "od -t x1
" command.
Table 10.7. List of packages which view and edit binary data.
package | popcon | size | description |
---|---|---|---|
coreutils
|
V:91, I:99 | 12868 |
This basic package has od (1) to dump files in octal and other formats.
|
bsdmainutils
|
V:65, I:99 | 644 |
This utility package has hd (1) to dump files in ASCII, decimal, hexadecimal, and octal formats.
|
hexedit
|
V:0.3, I:1.9 | 108 | View and edit files in hexadecimal or in ASCII |
bless
|
V:0.07, I:0.3 | 1240 | Full featured hexadecimal editor (GNOME) |
khexedit
|
V:1.6, I:11 | NOT_FOUND | Full featured hexadecimal editor (KDE). |
okteta
|
V:0.3, I:3 | 1252 | Full featured hexadecimal editor (KDE4). |
ncurses-hexedit
|
V:0.08, I:0.6 | 192 | Edit files/disks in HEX, ASCII and EBCDIC |
lde
|
V:0.04, I:0.5 | 992 | Linux Disk Editor |
beav
|
V:0.04, I:0.3 | 164 | Binary editor and viewer for HEX, ASCII, EBCDIC, OCTAL, DECIMAL, and BINARY formats. |
hex
|
V:0.01, I:0.11 | 84 | Hexadecimal dumping tool for Japanese |
![]() |
Tip |
---|---|
HEX is used as an acronym for hexadecimal format. |
If "disk.img
" contains an image of the disk contents and the original disk had a disk configuration which gives xxxx = (bytes/sector) * (sectors/cylinder)
, then the following will mount it to "/mnt
":
# mount -o loop,offset=xxxx disk.img /mnt
Note that most hard disks have 512 bytes/sector. This offset is to skip MBR of the hard disk. You can skip offset in the above example, if "disk.img
" contains
There are tools to read and write files without mounting disk.
There are tools for data file recovery and forensic analysis.
Table 10.9. List of packages for data file recovery and forensic analysis.
package | popcon | size | description |
---|---|---|---|
testdisk
|
V:0.3, I:3 | 4616 | Utilities for partition scan and disk recovery. |
magicrescue
|
V:0.09, I:0.5 | 336 | Recovers files by looking for magic bytes. |
scalpel
|
V:0.03, I:0.2 | 124 | A Frugal, High Performance File Carver. |
recover
|
V:0.09, I:0.9 | 104 | Undelete files on ext2 partitions. |
e2undel
|
V:0.08, I:0.6 | 240 | Undelete utility for the ext2 file system. |
ext3grep
|
V:0.07, I:0.5 | 300 | Tool to help recover deleted files on ext3 filesystems. |
scrounge-ntfs
|
V:0.03, I:0.4 | 44 | Data recovery program for NTFS filesystems. |
gzrt
|
V:0.02, I:0.17 | 68 | Gzip recovery toolkit. |
sleuthkit
|
V:0.14, I:0.6 | 4072 | Tools for forensics analysis. (Sleuthkit) |
autopsy
|
V:0.07, I:0.4 | 1372 | Graphical interface to SleuthKit. |
foremost
|
V:0.09, I:0.6 | 140 | Forensics application to recover data. |
tct
|
V:0.04, I:0.2 | 548 | Forensics related utilities. |
dcfldd
|
V:0.03, I:0.15 | 124 | Enhanced version of dd for forensics and security. |
rdd
|
V:0.02, I:0.13 | 200 | A forensic copy program. |
The ISO9660 image file, "cd.iso
", from the source directory tree at "source_directory
" can be made using genisoimage
(1):
# genisoimage -r -J -T -V volume_id -o cd.iso source_directory
Similary, the bootable ISO9660 image file, "cdboot.iso
", can be made from debian-installer
like directory tree at "source_directory
":
# genisoimage -r -o cdboot.iso -V volume_id \ -b isolinux/isolinux.bin -c isolinux/boot.cat \ -no-emul-boot -boot-load-size 4 -boot-info-table source_directory
Here Isolinux boot loader (see Section 3.3, “Stage 2: the boot loader”) is used for booting.
To make the disk image directly from the CD-ROM device using cp
(1) or dd
(1) has a few problems. The first run of dd
(1) may cause an error message and may yield a shorter disk image with a lost tail-end. The second run of dd
(1) may yield a larger disk image with garbage data attached at the end on some systems if the data size is not specified. Only the second run of dd
(1) with the correct data size specified, and without ejecting the CD after an error message, seems to avoid these problems. If for example the image size displayed by df
(1) is 46301184 blocks, use the following command twice to get the right image (this is my empirical information):
# dd if=/dev/cdrom of=cd.iso bs=2048 count=$((46301184/2))
![]() |
Tip |
---|---|
DVD is only a large CD to |
You can find a usable device by:
# wodim --devices
Then the blank CD-R is inserted to the device, and the ISO9660 image file, "cd.iso
" is written to this device, e.g., "/dev/hda
", by wodim
(1):
# wodim -v -eject dev=/dev/hda cd.iso
If CD-RW is used instead of CD-R, do this instead:
# wodim -v -eject blank=fast dev=/dev/hda cd.iso
![]() |
Tip |
---|---|
If your desktop system mounts CD automatically, unmount it by " |
If "cd.iso
" contains an ISO9660 image, then the following will manually mount it to "/cdrom
":
# mount -t iso9660 -o ro,loop cd.iso /cdrom
![]() |
Tip |
---|---|
Modern desktop system mounts removable media automatically (see Section 10.1.10, “Removable mass storage device”). |
When a data is too big to backup, you can back up a large file into, e.g. 2000MiB chunks and merge those files into a large file.
$ split -b 2000m large_file $ cat x* >large_file
![]() |
Caution |
---|---|
Please make sure you do not have any file starting with " |
In order to clear the contents of a file such as a log file, do not use rm
(1) to delete the file and then create a new empty file, because the file may still be accessed in the interval between commands. The following is the safe way to clear the contents of the file.
$ :>file_to_be_cleared
The following commands will create dummy or empty files:
$ dd if=/dev/zero of=5kb.file bs=1k count=5 $ dd if=/dev/urandom of=7mb.file bs=1M count=7 $ touch zero.file $ : > alwayszero.file
5kb.file
" is 5KB of zeros.
7mb.file
" is 7MB of random data.
zero.file
" is 0 byte file (if file exists, the file contents are kept while updating mtime.)
alwayszero.file
" is always 0 byte file (if file exists, the file contents are not kept while updating mtime.)
There are several ways to completely erase data from an entire harddisk-like device, e.g., USB memory stick at "/dev/sda
".
![]() |
Caution |
---|---|
Check your USB memory stick location with |
dd if=/dev/zero of=/dev/sda
# dd if=/dev/urandom of=/dev/sda
# shred -v -n 1 /dev/sda
Since dd
(1) is available from the shell of many bootable Linux CDs such as Debian installer CD, you can erase your installed system completely by running an erase command from such media on the system hard disk, e.g., "/dev/hda
", "/dev/sda
", etc.
Even if you have accidentally deleted a file, as long as that file is still being used by some application (read or write mode), it is possible to recover such a file.
$ echo foo > bar $ less bar
$ ps aux | grep ' less[ ]' bozo 4775 0.0 0.0 92200 884 pts/8 S+ 00:18 0:00 less bar $ rm bar $ ls -l /proc/4775/fd | grep bar lr-x------ 1 bozo bozo 64 2008-05-09 00:19 4 -> /home/bozo/bar (deleted) $ cat /proc/4775/fd/4 >bar $ ls -l -rw-r--r-- 1 bozo bozo 4 2008-05-09 00:25 bar $ cat bar foo
lsof
package installed, on another terminal:
$ ls -li bar 2228329 -rw-r--r-- 1 bozo bozo 4 2008-05-11 11:02 bar $ lsof |grep bar|grep less less 4775 bozo 4r REG 8,3 4 2228329 /home/bozo/bar $ rm bar $ lsof |grep bar|grep less less 4775 bozo 4r REG 8,3 4 2228329 /home/bozo/bar (deleted) $ cat /proc/4775/fd/4 >bar $ ls -li bar 2228302 -rw-r--r-- 1 bozo bozo 4 2008-05-11 11:05 bar $ cat bar foo
Files with hardlinks can be identified by "ls -li
", e.g.:
$ ls -li total 0 2738405 -rw-r--r-- 1 root root 0 2008-09-15 20:21 bar 2738404 -rw-r--r-- 2 root root 0 2008-09-15 20:21 baz 2738404 -rw-r--r-- 2 root root 0 2008-09-15 20:21 foo
Both "baz
" and "foo
" have link count of "2" (>1) showing them to have hardlinks. Their inode numbers are common "2738404". This means they are the same hardlinked file. If you do not happen to find all hardlinked files by chance, you can search it by the inode, e.g., "2738404":
# find /path/to/mount/point -xdev -inum 2738404
The data security infrastructure is provided by the combination of data encryption tool, message digest tool, and signature tool.
Table 10.10. List of data security infrastructure tools.
package | popcon | size | function |
---|---|---|---|
gnupg
|
V:39, I:99 | 5072 |
GNU privacy guard - OpenPGP encryption and signing tool. gpg (1)
|
gnupg-doc
|
I:1.5 | 4124 | GNU Privacy Guard documentation |
gpgv
|
V:61, I:98 | 392 | GNU privacy guard - signature verification tool |
cryptsetup
|
V:3, I:4 | 904 | Utilities for dm-crypto block device encryption supporting LUKS |
ecryptfs-utils
|
V:0.09, I:0.2 | 444 | Utilities for ecryptfs stacked filesystem encryption |
coreutils
|
V:91, I:99 | 12868 |
The md5sum command computes and checks MD5 message digest
|
coreutils
|
V:91, I:99 | 12868 |
The sha1sum command computes and checks SHA1 message digest
|
openssl
|
V:29, I:90 | 2360 |
The "openssl dgst " command computes message digest (OpenSSL). dgst (1ssl)
|
See Section 9.4, “Data encryption tips” on dm-crypto and ecryptfs which implement automatic data encryption infrastructure via Linux kernel modules.
Here are GNU Privacy Guard commands for the basic key management:
Table 10.11. List of GNU Privacy Guard commands for the key management
command | description |
---|---|
gpg --gen-key
|
generate a new key |
gpg --gen-revoke my_user_ID
|
generate revoke key for my_user_ID |
gpg --edit-key user_ID
|
edit key interactively, "help" for help |
gpg -o file --exports
|
export all keys to file |
gpg --imports file
|
import all keys from file |
gpg --send-keys user_ID
|
send key of user_ID to keyserver |
gpg --recv-keys user_ID
|
recv. key of user_ID from keyserver |
gpg --list-keys user_ID
|
list keys of user_ID |
gpg --list-sigs user_ID
|
list sig. of user_ID |
gpg --check-sigs user_ID
|
check sig. of user_ID |
gpg --fingerprint user_ID
|
check fingerprint of "user_ID" |
gpg --refresh-keys
|
update local keyring |
Here is the meaning of trust code:
Table 10.12. List of the meaning of trust code.
code | trust |
---|---|
-
|
No owner trust assigned / not yet calculated. |
e
|
Trust calculation has failed. |
q
|
Not enough information for calculation. |
n
|
Never trust this key. |
m
|
Marginally trusted. |
f
|
Fully trusted. |
u
|
Ultimately trusted. |
The following will upload my key "A8061F32
" to the popular keyserver "hkp://subkeys.pgp.net
":
$ gpg --keyserver hkp://subkeys.pgp.net --send-keys A8061F32
A good default keyserver set up in "~/.gnupg/gpg.conf
" (or old location "~/.gnupg/options
") contains:
keyserver hkp://subkeys.pgp.net
The following will obtain unknown keys from the keyserver:
$ gpg --list-sigs | \ sed -n '/^sig.*\[User ID not found\]/s/^sig..........\(\w\w*\)\W.*/\1/p' |\ sort | uniq | xargs gpg --recv-keys
There was a bug in OpenPGP Public Key Server (pre version 0.9.6) which corrupted key with more than 2 sub-keys. The newer gnupg
(>1.2.1-2) package can handle these corrupted subkeys. See gpg
(1) under "--repair-pks-subkey-bug
" option.
File handling:
Table 10.13. List of gnu privacy guard commands on files
command | description |
---|---|
gpg -a -s file
|
sign file into ascii armored file.asc |
gpg --armor --sign file
|
, , |
gpg --clearsign file
|
clear-sign message |
gpg --clearsign --not-dash-escaped patchfile
|
clear-sign patchfile |
gpg --verify file
|
verify clear-signed file |
gpg -o file.sig -b file
|
create detached signature |
gpg -o file.sig --detach-sig file
|
, , |
gpg --verify file.sig file
|
verify file with file.sig |
gpg -o crypt_file.gpg -r name -e file
|
public-key encryption intended for name from file to binary crypt_file.gpg |
gpg -o crypt_file.gpg --recipient name --encrypt file
|
, , |
gpg -o crypt_file.asc -a -r name -e file
|
public-key encryption intended for name from file to ASCII armored crypt_file.asc |
gpg -o crypt_file.gpg -c file
|
symmetric encryption from file to crypt_file.gpg |
gpg -o crypt_file.gpg --symmetric file
|
, , |
gpg -o crypt_file.asc -a -c file
|
symmetric encryption intended for name from file to ASCII armored crypt_file.asc |
gpg -o file -d crypt_file.gpg -r name
|
decryption |
gpg -o file --decrypt crypt_file.gpg
|
, , |
Add the following to "~/.muttrc
" to keep a slow GnuPG from automatically
starting, while allowing it to be used by typing "S
" at the index menu.
macro index S ":toggle pgp_verify_sig\n" set pgp_verify_sig=no
The gnupg
plugin let you run GnuPG transparently for files with extension ".gpg
", ".asc
", and ".ppg
".
# aptitude install vim-scripts vim-addon-manager $ vim-addons install gnupg
md5sum
(1) provides utility to make a digest file using the method in rfc1321 and verifying each file with it.
$ md5sum foo bar >baz.md5 $ cat baz.md5 d3b07384d113edec49eaa6238ad5ff00 foo c157a79031e1c40f85931829bc5fc552 bar $ md5sum -c baz.md5 foo: OK bar: OK
![]() |
Note |
---|---|
The computation for the MD5 sum is less CPU intensive than the one for the cryptographic signature by GNU Privacy Guard (GnuPG). Usually, only the top level digest file is cryptographically signed to ensure data integrity. |
There are many merge tools for the source code. Following commands caught my eyes.:
Table 10.14. List of source code merge tools.
command | package | popcon | size | description |
---|---|---|---|---|
diff (1)
|
diff
|
V:90, I:99 | 764 | This compares files line by line. |
diff3 (1)
|
diff
|
V:90, I:99 | 764 | This compares and merges three files line by line. |
vimdiff (1)
|
vim
|
V:14, I:30 | 1740 | This compares 2 files side by side in vim. |
patch (1)
|
patch
|
V:11, I:93 | 204 | This applies a diff file to an original. |
dpatch (1)
|
dpatch
|
V:2, I:15 | 344 | This manage series of patches for Debian package. |
diffstat (1)
|
diffstat
|
V:2, I:14 | 84 | This produces a histogram of changes by the diff. |
combinediff (1)
|
patchutils
|
V:2, I:15 | 292 | This creates a cumulative patch from two incremental patches. |
dehtmldiff (1)
|
patchutils
|
V:2, I:15 | 292 | This extracts a diff from an HTML page. |
filterdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This extracts or excludes diffs from a diff file. |
fixcvsdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This fixes diff files created by CVS that "patch" mis-interprets. |
flipdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This exchanges the order of two patches. |
grepdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This shows which files are modified by a patch matching a regex. |
interdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This shows differences between two unified diff files. |
lsdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This shows which files are modified by a patch. |
recountdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This recomputes counts and offsets in unified context diffs. |
rediff (1)
|
patchutils
|
V:2, I:15 | 292 | This fixes offsets and counts of a hand-edited diff. |
splitdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This separates out incremental patches. |
unwrapdiff (1)
|
patchutils
|
V:2, I:15 | 292 | This demangles patches that have been word-wrapped. |
wiggle (1)
|
wiggle
|
V:0.02, I:0.11 | 204 | This applies rejected patches. |
quilt (1)
|
quilt
|
V:1.0, I:6 | 856 | This manage series of patches. |
meld (1)
|
meld
|
V:0.4, I:2 | 2304 | This is a GTK graphical file comparator and merge tool. |
xxdiff (1)
|
xxdiff
|
V:0.2, I:1.1 | 1352 | This is a plain X graphical file comparator and merge tool. |
dirdiff (1)
|
dirdiff
|
V:0.09, I:0.5 | 212 | This displays and merges changes between directory trees. |
docdiff (1)
|
docdiff
|
V:0.02, I:0.18 | 688 | This compares two files word by word / char by char. |
imediff2 (1)
|
imediff2
|
V:0.02, I:0.11 | 76 | This is an interactive full screen 2-way merge tool. |
makepatch (1)
|
makepatch
|
V:0.03, I:0.2 | 148 | This generates extended patch files. |
applypatch (1)
|
makepatch
|
V:0.03, I:0.2 | 148 | This applies extended patch files. |
wdiff (1)
|
wdiff
|
V:1.8, I:14 | 124 | This displays word differences between text files. |
Following one of these procedures will extract differences between two source files and create unified diff files "file.patch0
" or "file.patch1
" depending on the file location:
$ diff -u file.old file.new > file.patch0 $ diff -u old/file new/file > file.patch1
The diff file (alternatively called patch file) is used to send a program update. The receiving party will apply this update to another file by:
$ patch -p0 file < file.patch0 $ patch -p1 file < file.patch1
Here is a summary of the version control systems (VCS) on the Debian system:
![]() |
Note |
---|---|
If you are new to VCS systems, you should start learning with Git, which is growing fast in popularity. |
Table 10.15. List of version control system tools.
package | popcon | size | tool | VCS type | comment |
---|---|---|---|---|---|
cssc
|
V:0.01, I:0.05 | 2168 | CSSC | local | Clone of the Unix SCCS (deprecated) |
rcs
|
V:1.7, I:9 | 772 | RCS | local | "Unix SCCS done right" |
cvs
|
V:4, I:25 | 3660 | CVS | remote | The previous standard remote VCS |
subversion
|
V:11, I:32 | 4228 | Subversion | remote | "CVS done right", the new de facto standard remote VCS |
git-core
|
V:6, I:10 | 14344 | Git | distributed | fast DVCS in C (used by the Linux kernel and others) |
mercurial
|
V:1.0, I:4 | 324 | Mercurial | distributed | DVCS in python and some C. |
bzr
|
V:0.5, I:2 | 19652 | Bazaar | distributed |
DVCS influenced by tla written in python (used by Ubuntu)
|
darcs
|
V:0.3, I:1.6 | 7872 | Darcs | distributed | DVCS with smart algebra of patches (slow). |
tla
|
V:0.19, I:1.4 | 1100 | GNU arch | distributed | DVCS mainly by Tom Lord. (Historic) |
monotone
|
V:0.05, I:0.4 | 4656 | Monotone | distributed | DVCS in C++ |
VCS is sometimes known as revision control system (RCS), or software configuration management (SCM).
Distributed VCS such as Git is the tool of choice these days. CVS and Subversion may still be useful to join some existing open source program activities.
Debian provides free VCS services via Debian Alioth service. It supports practically all VCSs. Its documentation can be found at http://wiki.debian.org/Alioth .
![]() |
Caution |
---|---|
The |
Here is an oversimplified comparison of native VCS commands to provide the big picture. The typical command sequence may require options and arguments.
Table 10.16. Comparison of native VCS commands.
CVS | Subversion | Git | function |
---|---|---|---|
cvs init
|
svn create
|
git init
|
create the (local) repository |
cvs login
|
- | - | login to the remote repository |
cvs co
|
svn co
|
git clone
|
check out the remote repository as the working tree |
cvs up
|
svn up
|
git pull
|
update the working tree by merging the remote repository |
cvs add
|
svn add
|
git add .
|
add file(s) in the working tree to the VCS |
cvs rm
|
svn rm
|
git rm
|
remove file(s) in working tree from the VCS |
cvs ci
|
svn ci
|
- | commit changes to the remote repository |
- | - |
git commit -a
|
commit changes to the local repository |
- | - |
git push
|
update the remote repository by the local repository |
cvs status
|
svn status
|
git status
|
display the working tree status from the VCS |
cvs diff
|
svn diff
|
git diff
|
diff <reference_repository> <working_tree> |
- | - |
git repack -a -d; git prune
|
repack the local repository into single pack. |
![]() |
Caution |
---|---|
Invoking a |
![]() |
Tip |
---|---|
Git can work directly with different VCS repositories such as ones provided by CVS and Subversion, and provides the local repository for local changes with |
![]() |
Tip |
---|---|
Git has commands which have no equivalents in CVS and Subversion. "Fetch", "Rebase", "Cherrypick", … |
Check
cvs
(1),
/usr/share/doc/cvs/html-cvsclient
",
/usr/share/doc/cvs/html-info
",
/usr/share/doc/cvsbook
", and
info cvs
",
for detailed information.
The following setup will allow commits to the CVS repository only by a member
of the "src
" group, and administration of CVS only by a member of the "staff
"
group, thus reducing the chance of shooting oneself.
# cd /var/lib; umask 002; mkdir cvs # export CVSROOT=/var/lib/cvs # cd $CVSROOT # chown root:src . # chmod 2775 . # cvs -d $CVSROOT init # cd CVSROOT # chown -R root:staff . # chmod 2775 . # touch val-tags # chmod 664 history val-tags # chown root:src history val-tags
You may restrict creation of new project by changing the owner of "$CVSROOT
" directory to "root:staff
" and its permission to "3775
".
The following will set up shell environments for the local access to the CVS repository:
$ export CVSROOT=/var/lib/cvs
The following will set up shell environments for the read-only remote access to the CVS repository without SSH (use RSH protocol capability in cvs
(1)):
$ export CVSROOT=:pserver:account@cvs.foobar.com:/var/lib/cvs $ cvs login
This is prone to eavesdropping attack.
The following will set up shell environments for the read-only remote access to the CVS repository:
$ export CVSROOT=:pserver:anonymous@cvs.sf.net:/cvsroot/qref $ cvs login $ cvs -z3 co qref
The following will set up shell environments for the read-only remote access to the CVS repository with SSH:
$ export CVSROOT=:ext:account@cvs.foobar.com:/var/lib/cvs
or for SourceForge:
$ export CVSROOT=:ext:account@cvs.sf.net:/cvsroot/qref
You can also use public key authentication for SSH which eliminates the password prompt.
For,
Table 10.17. Assumption for the CVS archive.
ITEM | VALUE | MEANING |
---|---|---|
source tree |
~/project-x
|
All source codes |
Project name |
project-x
|
Name for this project |
Vendor Tag |
Main-branch
|
Tag for the entire branch |
Release Tag |
Release-initial
|
Tag for a specific release |
Then,
$ cd ~/project-x
$ cvs import -m "Start project-x" project-x Main-branch Release-initial $ cd ..; rm -R ~/project-x
To work with project-x
using the local CVS repository:
$ mkdir -p /path/to; cd /path/to $ cvs co project-x
$ cd project-x
$ cvs diff -u
diff -u repository/ local/
"
$ cvs up -C modified_file
$ cvs ci -m "Describe change"
$ vi newfile_added $ cvs add newfile_added $ cvs ci -m "Added newfile_added" $ cvs up
cvs up -d -P
" instead.
C filename
" which indicates conflicting changes.
<<<<<<<
" and ">>>>>>>
" in the files for conflicting changes.
$ cvs tag Release-1
$ cvs tag -d Release-1
$ cvs ci -m "more comments" $ cvs tag Release-1
* re-add release tag
$ cd /path/to $ cvs co -r Release-initial -d old project-x
/path/to/old
" directory
$ cd old $ cvs tag -b Release-initial-bugfixes
Release-initial-bugfixes
"
$ cvs update -d -P
$ cvs up -d -P
$ cvs ci -m "check into this branch" $ cvs update -kk -A -d -P
$ cvs update -kk -d -P -j Release-initial-bugfixes
$ cvs ci -m "merge Release-initial-bugfixes" $ cd $ tar -cvzf old-project-x.tar.gz old
-j
" if you want ".tar.bz2
".
$ cvs release -d old
Table 10.18. Notable options for CVS commands (use as first argument(s) to cvs
(1)).
option | meaning |
---|---|
-n
|
dry run, no effect |
-t
|
display messages showing steps of cvs activity |
To get the latest version from CVS, use "tomorrow
":
$ cvs ex -D tomorrow module_name
Add alias to a project (local server):
$ export CVSROOT=/var/lib/cvs $ cvs co CVSROOT/modules $ cd CVSROOT $ echo "px -a project-x" >>modules $ cvs ci -m "Now px is an alias for project-x" $ cvs release -d . $ cvs co -d project px
$ cd project
In order to perform above procedure, you should have the appropriate file permission.
CVS will not overwrite the current repository file but replaces it with another one. Thus, write permission to the repository directory is critical. For every new repository creation, run the following to ensure this condition if needed.
# cd /var/lib/cvs # chown -R root:src repository # chmod -R ug+rwX repository # chmod 2775 repository
Subversion is a next-generation version control system, intended to replace CVS, so it has most of CVS's features. Generally, Subversion's interface to a particular feature is similar to CVS's, except where there's a compelling reason to do otherwise.
You need to install subversion
, libapache2-svn
and subversion-tools
packages to set up a server.
Currently, the subversion
package does not set up a repository, so one must be set up manually. One possible location for a repository is in "/var/local/repos
".
Create the directory:
# mkdir -p /var/local/repos
Create the repository database:
# svnadmin create /var/local/repos
Make the repository writable by the WWW server:
# chown -R www-data:www-data /var/local/repos
To allow access to the repository via user authentication, add (or uncomment) the following in "/etc/apache2/mods-available/dav_svn.conf
":
<Location /repos> DAV svn SVNPath /var/local/repos AuthType Basic AuthName "Subversion repository" AuthUserFile /etc/subversion/passwd <LimitExcept GET PROPFIND OPTIONS REPORT> Require valid-user </LimitExcept> </Location>
Then, create a user authentication file with the command:
# htpasswd2 -c /etc/subversion/passwd some-username
Restart Apache2, and your new Subversion repository will be accessible with the URL "http://hostname/repos
".
The following sections teach you how to use different commands in Subversion.
To create a new Subversion archive, type the following:
$ cd ~/your-project # go to your source directory $ svn import http://localhost/repos your-project project-name -m "initial project import"
This creates a directory named project-name in your Subversion repository which contains your project files. Look at "http://localhost/repos/
" to see if it's there.
Working with project-y using Subversion:
$ mkdir -p /path/to ;cd /path/to $ svn co http://localhost/repos/project-y
$ cd project-y
$ svn diff
-similar to "diff -u repository/ local/
"
$ svn revert modified_file
$ svn ci -m "Describe changes"
$ vi newfile_added $ svn add newfile_added $ svn add new_dir
$ svn add -N new_dir2
$ svn ci -m "Added newfile_added, new_dir, new_dir2" $ svn up
$ svn log
$ svn copy http://localhost/repos/project-y \ http://localhost/repos/project-y-branch \ -m "creating my branch of project-y"
$ svn copy http://localhost/repos/project-y \ http://localhost/repos/projct-y-release1.0 \ -m "project-y 1.0 release"
$ svn merge http://localhost/repos/project-y \ http://localhost/repos/project-y-branch
$ svn co -r 4 http://localhost/repos/project-y
Git can do everything for both local and remote source code management. This means that you can record the source code changes without needing network connectivity to the remote repository.
You may wish to set several global configuration in "~/.gitconfig
" such as your name and email address used by Git:
$ git config --global user.name "Name Surname" $ git config --global user.email yourname@example.com
If you are too used to CVS or Subversion commands, you may wish to set several command aliases;
$ git config --global alias.ci "commit -a" $ git config --global alias.co checkout
You can check your global configuration by:
$ git config --global --list
There are good references for Git.
/usr/share/doc/git-doc/git.html
)
/usr/share/doc/git-doc/user-manual.html
)
/usr/share/doc/git-doc/gittutorial.html
)
/usr/share/doc/git-doc/gittutorial-2.html
)
/usr/share/doc/git-doc/everyday.html
)
git for CVS users (/usr/share/doc/git-doc/gitcvs-migration.html
)
Other git resources available on the web:
/usr/share/doc/gitmagic/html/index.html
)
git-gui
(1) and gitk
(1) commands make using Git very easy.
![]() |
Warning |
---|---|
Do not use the tag string with spaces in it even if some tools such as |
Even if your upstream uses different VCS, it is good idea to use git
(1) for local activity since you can manage your local copy of source tree without the network connection to the upstream. Here are commands used with git
(1).
Table 10.19. List of git packages and commands.
command | package | popcon | size | description |
---|---|---|---|---|
N/A |
git-doc
|
I:2 | 5804 | This provides the oficial documentation for Git. |
N/A |
gitmagic
|
I:0.2 | 504 | "Git Magic" provides easier to understand guide for Git. |
git (7)
|
git-core
|
V:6, I:10 | 14344 | The main command for Git. |
gitk (1)
|
gitk
|
V:0.7, I:3 | 752 | The GUI Git repository browser with history. |
git-gui (1)
|
git-gui
|
V:0.2, I:2 | 1428 | The GUI for Git. (No history) |
git-svnimport (1)
|
git-svn
|
V:0.4, I:2 | 492 | This import the data out of Subversion into Git. |
git-svn (1)
|
git-svn
|
V:0.4, I:2 | 492 | This provides bidirectional operation between the Subversion and Git. |
git-cvsimport (1)
|
git-cvs
|
V:0.17, I:1.3 | 620 | This import the data out of CVS into Git. |
git-cvsexportcommit (1)
|
git-cvs
|
V:0.17, I:1.3 | 620 | This exports a commit to a CVS checkout from Git. |
git-cvsserver (1)
|
git-cvs
|
V:0.17, I:1.3 | 620 | A CVS server emulator for Git. |
git-send-email (1)
|
git-email
|
V:0.12, I:1.2 | 364 | This sends a collection of patches as email from the Git. |
stg (1)
|
stgit
|
V:0.08, I:0.6 | 844 | This is quilt on top of git. (Python) |
git-buildpackage (1)
|
git-buildpackage
|
V:0.16, I:0.8 | 440 | This automates the Debian packaging with the Git. |
guilt (7)
|
guilt
|
V:0.02, I:0.09 | 336 | This is quilt on top of git. (SH/AWK/SED/…) |
You can manually record chronological history of configuration using Git tools. Here is a simple example for your practice to record "/etc/apt/
" contents.:
$ cd /etc/apt/ $ sudo git init $ sudo chmod 700 .git $ sudo git add . $ sudo git commit -a
$ cd /etc/apt/ $ sudo git commit -a
$ cd /etc/apt/ $ sudo gitk --all
![]() |
Note |
---|---|
|
![]() |
Note |
---|---|
The " |
![]() |
Tip |
---|---|
For more complete setup for recording configuration history, please look for the |
Tools and tips for converting data formats on the Debian system are described.
Standard based tools are in very good shape but support for proprietary data formats are limited.
Following packages for the text data conversion caught my eyes:
Table 11.1. List of text data conversion tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
libc6
|
V:95, I:99 | 11496 | charset |
The text encoding converter between locales by iconv (1). (fundamental)
|
recode
|
V:1.7, I:7 | 780 | charset+eol | The text encoding converter between locales. (versatile, more aliases and features) |
konwert
|
V:0.4, I:4 | 192 | charset | The text encoding converter between locales. (fancy) |
nkf
|
V:0.3, I:2 | 300 | charset | The character set translator for Japanese. |
tcs
|
V:0.03, I:0.17 | 544 | charset | The character set translator. |
unaccent
|
V:0.03, I:0.10 | 60 | charset | Replace accented letters by their unaccented equivalent. |
tofrodos
|
V:1.3, I:8 | 80 | eol |
The text format converter between DOS and Unix: fromdos (1) and todos (1)
|
macutils
|
V:0.10, I:0.7 | 356 | eol |
The text format converter between Macintosh and Unix: frommac (1) and tomac (1)
|
iconv
(1) is provided as a part of the libc6
package and it is always available on all system to convert the encoding of characters:
$ iconv -f encoding1 -t encoding2 input.txt >output.txt
Encoding values are case insensitive and ignore "-
" and "_
" for matching. Supported encodings can be checked by the "iconv -l
" command.
Table 11.2. List of encoding values and their usage.
encoding value | usage |
---|---|
ASCII. | American Standard Code for Information Interchange. 7 bit code w/o accented characters. |
UTF-8 | Standard multilingual compatibility for all modern OSs. |
ISO-8859-1 | Old standard for western European languages, ASCII + accented characters. |
ISO-8859-2 | Old standard for eastern European languages, ASCII + accented characters. |
ISO-8859-15 | Old standard for western European languages, ISO-8859-1 with euro sign. |
CP850 | Code page 850, Microsoft DOS characters with graphics for western European languages. ISO-8859-1 variant. |
CP932 | Code page 932, Microsoft Windows style Shift-JIS variant, for Japanese. |
CP936 | Code page 936, Microsoft Windows style GB2312, GBK, or GB18030 variant, for Simplified Chinese. |
CP949 | Code page 949, Microsoft Windows style EUC-KR or Unified Hangul Code variant, for Korean. |
CP950 | Code page 950, Microsoft Windows style Big5 variant, for Traditional Chinese. |
CP1251 | Code page 1251, Microsoft Windows style encoding for the Cyrillic alphabet. |
CP1252 | Code page 1252, Microsoft Windows style ISO-8859-15 variant for western European languages. |
KOI8-R | Old Russian UNIX standard for the Cyrillic alphabet. |
ISO-2022-JP | Standard encoding for Japanese e-mail which uses only 7 bit codes. |
eucJP | Old Japanese UNIX standard 8 bit code and completely different from Shift-JIS. |
Shift-JIS | JIS X 0208 Appendix 1 standard, for Japanese. See CP932 above. |
![]() |
Note |
---|---|
Some encodings are only supported for the data conversion and are not used as locale values (Section 8.3.1, “Basics of encoding”). |
For character sets which fit in single byte such as ASCII and ISO-8859 character sets, the character encoding means almost the same thing as the character set.
For character sets with many characters such as JIS X 0213 for Japanese or Universal Character Set (UCS, Unicode, ISO-10646-1) for practically all languages, there are many encoding schemes to fit them into the sequence of the byte data:
The code page is used as the synonym to the character encoding tables for some vendor specific ones.
![]() |
Note |
---|---|
Please note most encoding systems share the same code with ASCII for the 7 bit characters. But there are some exceptions. If you are converting old Japanese C programs and URLs data from the casually-called shift-JIS encoding format to UTF-8 format, use " |
![]() |
Tip |
---|---|
|
Here is an example script to convert encoding of file names from ones created under older OS to modern UTF-8 ones in a single directory.
#!/bin/sh ENCDN=iso-8859-1 for x in *; do mv "$x" $(echo "$x" | iconv -f $ENCDN -t utf-8) done
The "$ENCDN
" variable should be set by the encoding value in Table 11.2, “List of encoding values and their usage.”.
For more complicated case, please mount disk drive containing such file names with proper encoding as the mount
(8) option (see Section 8.3.6, “Filename encoding”) and copy entire disk to another disk drive mounted as UTF-8 with "cp -a
" command.
The text file format, specifically the end-of-line (EOL) code, is dependent on the platform:
Table 11.3. List of EOL conversion tools.
platform | EOL code | EOL control sequence | EOL ASCII value |
---|---|---|---|
Debian (unix) | LF |
^J
|
10 |
MSDOS and Windows | CR-LF |
^M^J
|
13, 10 |
Apple's Macintosh | CR |
^M
|
13 |
The EOL format conversion programs, fromdos
(1), todos
(1), frommac
(1), and tomac
(1), are quite handy. recode
(1) is also useful.
![]() |
Note |
---|---|
Some data on the Debian system, such as the wiki page data for the |
![]() |
Note |
---|---|
Most editors (eg. |
![]() |
Tip |
---|---|
The use of " |
There are few popular specialized programs to convert the tab codes:
Table 11.4. List of TAB conversion commands from bsdmainutils
and coreutils
packages.
function |
bsdmainutils
|
coreutils
|
---|---|---|
expand tab to spaces |
"col -x "
|
expand
|
unexpand tab from spaces |
"col -h "
|
unexpand
|
indent
(1) from the indent
package completely reformats whitespaces in the C program.
Editor programs such as vim
and emacs
can be used for TAB conversion, too. For example with vim
, you can expand TAB with ":set expandtab
" and ":%retab
" command sequence. You can revert this with ":set noexpandtab
" and ":%retab!
" command sequence.
Intelligent modern editors such as the vim
program are quite smart and copes well with any encoding systems and any file formats. You should use these editors under the UTF-8 locale in the UTF-8 capable console for the best compatibility.
An old western European Unix text file, "u-file.txt
", stored in the latin1 (iso-8859-1) encoding can be edited simply with vim
as:
$ vim u-file.txt
This is possible since the auto detection mechanism of the file encoding in vim
assumes the UTF-8 encoding first and, if it fails, assumes it to be latin1.
An old Polish Unix text file, "pu-file.txt
", stored in the latin2 (iso-8859-2) encoding can be edited with vim
as:
$ vim '+e ++enc=latin2 pu-file.txt'
An old Japanese unix text file, "ju-file.txt
", stored in the eucJP encoding can be edited with vim
as:
$ vim '+e ++enc=eucJP ju-file.txt'
An old Japanese MS-Windows text file, "jw-file.txt
", stored in the so called shift-JIS encoding (more precisely: CP932) can be edited with vim
as:
$ vim '+e ++enc=CP932 ++ff=dos jw-file.txt'
When a file is opened with "
" options, "enc
" and "ff
:w
" in the Vim command line stores it in the original format and overwrite the original file. You can also specify the saving format and the file name in the Vim command line, e.g., ":w ++enc=utf8 new.txt
".
Please refer to the mbyte.txt "multi-byte text support" in vim
on-line help.
The emacs
family of programs can perform the equivalent functions.
Following will read a web page into a text file. This is very useful when copying configurations off the Web or applying basic Unix text tools such as grep
(1) on the web page.
$ w3m -dump http://www.remote-site.com/help-info.html >textfile
Similarly, you can extract plain text data from other formats using followings:
Table 11.5. List of tools to extract plain text data.
package | popcon | size | keyword | function |
---|---|---|---|---|
w3m
|
V:23, I:85 | 1968 | html→text |
An HTML to text converter with the "w3m -dump " command.
|
html2text
|
V:14, I:40 | 308 | html→text | An advanced HTML to text converter. (ISO 8859-1) |
lynx
|
V:2, I:25 | 48 | html→text |
An HTML to text converter with the "lynx -dump " command.
|
elinks
|
V:2, I:6 | 1452 | html→text |
An HTML to text converter with the "elinks -dump " command.
|
links
|
V:3, I:9 | 1372 | html→text |
An HTML to text converter with the "links -dump " command.
|
links2
|
V:1.0, I:4 | 3280 | html→text |
An HTML to text converter with the "links2 -dump " command.
|
antiword
|
V:1.0, I:2 | 796 | MSWord→text,ps | This converts MSWord files to plain text or ps. |
catdoc
|
V:0.8, I:2 | 2664 | MSWord→text,TeX | This converts MSWord files to plain text or TeX. |
pstotext
|
V:0.9, I:1.6 | 160 | ps/pdf→text | Extract text from PostScript and PDF files. |
unhtml
|
V:0.02, I:0.19 | 76 | html→text | Remove the markup tags from an HTML file. |
odt2txt
|
V:0.6, I:1.1 | 104 | odt→text | The converter from OpenDocument Text to text. |
wpd2sxw
|
V:0.02, I:0.16 | 156 | WordPerfect→sxw | WordPerfect to OpenOffice.org/StarOffice writer document converter. |
Table 11.6. List of tools to highlight plain text data.
package | popcon | size | keyword | function |
---|---|---|---|---|
vim-runtime
|
V:3, I:35 | 24688 | highlight |
Vim can convert source code to HTML with ":source $VIMRUNTIME/syntax/html.vim " (vim MACRO)
|
cxref
|
V:0.08, I:0.6 | 1104 | c→html | The converter for the C program to latex and HTML. (C language) |
src2tex
|
V:0.03, I:0.3 | 1968 | highlight | This convert many source codes to TeX. (C language) |
source-highlight
|
V:0.13, I:0.8 | 1980 | highlight | This convert many source codes to HTML, XHTML, LaTeX, Texinfo, ANSI color escape sequences and DocBook files with highlight. (C++) |
highlight
|
V:0.07, I:0.4 | 680 | highlight | This convert many source codes to HTML, XHTML, RTF, LaTeX, TeX or XSL-FO files with highlight. (C++) |
grc
|
V:0.03, I:0.12 | 164 | text→color | The generic colouriser for everything. (Python) |
txt2html
|
V:0.08, I:0.4 | 296 | text→html | Text to HTML converter. (Perl) |
markdown
|
V:0.09, I:0.4 | 96 | text→html | Markdown text document formatter to (X)HTML. (Perl) |
asciidoc
|
V:0.13, I:0.8 | 4316 | text→any | AsciiDoc text document formatter to XML/HTML. (Python) |
python-docutils
|
V:0.4, I:2 | 5052 | text→any | ReStructured Text document formatter to XML. (Python) |
txt2tags
|
V:0.06, I:0.3 | 1556 | text→any | The document conversion from text to HTML, SGML, LaTeX, man page, MoinMoin, Magic Point and PageMaker. (Python) |
udo
|
V:0.01, I:0.08 | 556 | text→any | universal document - text processing utility. (C language) |
stx2any
|
V:0.00, I:0.05 | 484 | text→any | The document converter from structured plain text to other formats. (m4) |
rest2web
|
V:0.02, I:0.10 | 576 | text→html | The document converter from ReStructured Text to html. (Python) |
aft
|
V:0.01, I:0.08 | 336 | text→any | The "free form" document preparation system. (Perl) |
yodl
|
V:0.01, I:0.07 | 536 | text→any | A pre-document language and tools to process it. (C language) |
sdf
|
V:0.01, I:0.09 | 1940 | text→any | The simple document parser. (Perl) |
sisu
|
V:0.01, I:0.07 | 5484 | text→any | The document structuring, publishing and search framework. (Ruby) |
The Extensible Markup Language (XML) is a markup language for documents containing structured information.
XML.COM has good introductory information:
XML text looks somewhat like HTML. It enables us to manage multiple formats of output for a document. One easy XML system is the docbook-xsl
package, which is used here.
Each XML file starts with standard XML declaration:
<?xml version="1.0" encoding="UTF-8"?>
The basic syntax for one XML element is marked up as:
<name attribute="value">content</name>
XML element with empty content is marked up in the short form as:
<name attribute="value"/>
The "attribute="value"
" in the above examples are optional.
The comment section in XML is marked up as:
<!-- comment -->
Other than adding markups, XML requires minor conversion to the content using predefined entities for the following character:
Table 11.7. List of predefined entities for XML.
predefined entity | character to be converted from |
---|---|
"
|
" : quote
|
'
|
' : apostrophe
|
<
|
< : less-than
|
>
|
> : greater-than
|
&
|
& : ampersand
|
![]() |
Caution |
---|---|
" |
![]() |
Note |
---|---|
When SGML style user defined entities, e.g. " |
![]() |
Note |
---|---|
As long as the XML markup are done consistently with certain set of the tag name (either some data as content or attribute value), conversion to another XML is trivial task using Extensible Stylesheet Language Transformations (XSLT). |
There are many tools available to process XML files such as the Extensible Stylesheet Language (XSL).
Basically, once you create well formed XML file, you can convert it to any format using Extensible Stylesheet Language Transformations (XSLT).
The Extensible Stylesheet Language for Formatting Object (XSL-FO) is supposed to be solution for formatting. The fop
package is in the Debian contrib
(not main
) archive still. So the LaTeX code is usually generated from XML using XSLT and the LaTeX system is used to create printable file such as DVI, PostScript, and PDF.
Table 11.8. List of XML tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
docbook-xml
|
V:23, I:55 | 2488 | xml | This package contains the XML document type definition (DTD) for DocBook. |
xsltproc
|
V:5, I:52 | 180 | xslt | XSLT command line processor. (XML→ XML, HTML, plain text, etc.) |
docbook-xsl
|
V:0.7, I:6 | 12968 | xml/xslt | This contains XSL stylesheets for processing DocBook XML to various output formats with XSLT. |
xmlto
|
V:0.4, I:2 | 272 | xml/xslt | XML-to-any converter with XSLT. |
dblatex
|
V:0.18, I:1.2 | 6420 | xml/xslt | This converts Docbook files to DVI, PostScript, PDF documents with XSLT. |
fop
|
V:0.15, I:1.0 | 2296 | xml/xsl-fo | This converts Docbook XML files to PDF. |
Since XML is subset of Standard Generalized Markup Language (SGML), it can be processed by the extensive tools available for SGML, such as Document Style Semantics and Specification Language (DSSSL).
Table 11.9. List of DSSL tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
openjade
|
V:0.6, I:3 | 1212 | dsssl | Implementation of the DSSSL language based on James Clark's Jade software. |
jade
|
V:0.5, I:3 | 1056 | dsssl | James lark's DSSSL language. |
docbook-dsssl
|
V:0.7, I:5 | 3100 | xml/dsssl | This contains DSSSL stylesheets for processing DocBook XML to various output formats with DSSSL. |
docbook-utils
|
V:0.3, I:2 | 440 | xml/dsssl |
The utilities for Docbook files including conversion to other formats (HTML, RTF, PS, man, PDF) with docbook2* commands with DSSSL.
|
sgml2x
|
V:0.01, I:0.10 | 216 | SGML/dsssl | The converter from SGML and XML using DSSSL stylesheets. |
You can extract HTML or XML data from other formats using followings:
Table 11.10. List of XML data extraction tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
wv
|
V:1.5, I:3 | 2136 | MSWord→any | The document converter from Microsoft Word to HTML, LaTeX, etc.. |
texi2html
|
V:0.4, I:3 | 1752 | texi→html | The converter from Texinfo to HTML. |
man2html
|
V:0.3, I:1.6 | 372 | manpage→html | The converter from manpage to HTML. (CGI support) |
tex4ht
|
V:0.2, I:2 | 932 | tex↔html | The converter between (La)TeX and HTML. |
xlhtml
|
V:0.6, I:1.6 | 184 | MSExcel→html | The converter from MSExcel .xls to HTML. |
ppthtml
|
V:0.6, I:1.6 | 120 | MSPowerPoint→html | The converter from MSPowerPoint to HTML. |
unrtf
|
V:0.4, I:1.0 | 276 | rtf→html | The document converter from RTF to HTML, etc.. |
info2www
|
V:0.5, I:1.4 | 156 | info→html | The converter from GNU info to HTML. (CGI support) |
ooo2dbk
|
V:0.03, I:0.2 | 941 | sxw→xml | The converter from OpenOffice.org SXW documents to DocBook XML. |
wp2x
|
V:0.02, I:0.10 | 240 | WordPerfect→any | WordPerfect 5.0 and 5.1 files to TeX, LaTeX, troff, GML and HTML. |
doclifter
|
V:0.00, I:0.05 | 420 | troff→xml | The converter from troff to DocBook XML. |
For non-XML HTML files, you can convert them to XHTML which is an instance of well formed XML and can be processed by XML tools.
Table 11.11. List of XML pretty print tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
libxml2-utils
|
V:5, I:53 | 120 | xml↔html↔xhtml |
The command line XML tool with xmllint (1). (syntax check, reformat, lint, …)
|
tidy
|
V:1.8, I:15 | 108 | xml↔html↔xhtml | HTML syntax checker and reformatter. |
Once proper XML is generated, you can use XSLT technology to extract data based on the mark-up context etc.
Printable data is expressed in the PostScript format on the Debian system. Common Unix Printing System (CUPS) uses Ghostscript as its rasterizer backend program for non-PostScript printers.
The core of printable data manipulation is the Ghostscript PostScript (PS) interpreter which generates raster image.
The latest upstream Ghostscript from Artifex was re-licensed from AFPL to GPL and merged all the latest ESP version changes such as CUPS related ones at 8.60 release as unified release.
Table 11.12. List of Ghostscript PostScript interpreters.
package | popcon | size | description |
---|---|---|---|
ghostscript
|
V:17, I:47 | 3316 | The GPL Ghostscript PostScript/PDF interpreter |
ghostscript-x
|
V:13, I:30 | 256 | The GPL Ghostscript PostScript/PDF interpreter - X Display support |
gs-cjk-resource
|
I:0.5 | 4652 | Resource files for gs-cjk, Ghostscript CJK-TrueType extension |
cmap-adobe-cns1
|
I:0.4 | 1588 | CMaps for Adobe-CNS1 (for traditional Chinese support) |
cmap-adobe-gb1
|
I:0.4 | 1580 | CMaps for Adobe-GB1 (for simplified Chinese support) |
cmap-adobe-japan1
|
I:0.9 | 2476 | CMaps for Adobe-Japan1 (for Japanese standard support) |
cmap-adobe-japan2
|
I:0.4 | 440 | CMaps for Adobe-Japan2 (for Japanese extra support) |
cmap-adobe-korea1
|
I:0.2 | 912 | CMaps for Adobe-Korea1 (for Korean support) |
libpoppler4
|
I:12 | 2206 | PDF rendering library based on xpdf PDF viewer |
libpoppler-glib4
|
V:4, I:11 | 388 | PDF rendering library (GLib-based shared library) |
poppler-data
|
I:0.3 | 12276 | CMaps for PDF rendering library (for CJK support: Adobe-*) |
![]() |
Tip |
---|---|
" |
You can merge two PostScript (PS) or Portable Document Format (PDF) files using gs
(1) of Ghostscript.
$ gs -q -dNOPAUSE -dBATCH -sDEVICE=pswrite -sOutputFile=bla.ps -f foo1.ps foo2.ps $ gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=bla.pdf -f foo1.pdf foo2.pdf
![]() |
Note |
---|---|
The PDF, which is widely used cross-platform printable data format, is essentially the compressed PS format with few additional features and extensions. |
![]() |
Tip |
---|---|
From command line, |
The following packages for the printable data utilities caught my eyes:
Table 11.13. List of printable data utilities.
package | popcon | size | keyword | function |
---|---|---|---|---|
poppler-utils
|
V:6, I:51 | 428 | pdf→ps,text,… | PDF utilities. (pdftops, pdfinfo, pdfimages, pdftotext, and pdffonts) |
psutils
|
V:3, I:26 | 408 | ps→ps | PostScript document conversion tools |
poster
|
V:2, I:16 | 80 | ps→ps | Create large posters out of PostScript pages. |
xpdf-utils
|
V:1.9, I:8 | 4680 | pdf→ps,text,… | PDF utilities. (pdftops, pdfinfo, pdfimages, pdftotext, and pdffonts) |
enscript
|
V:2, I:21 | 2432 | text→ps, html, rtf | Converts ASCII text to Postscript, HTML, RTF or Pretty-Print. |
a2ps
|
V:1.7, I:9 | 4288 | text→ps | 'Anything to PostScript' converter and pretty-printer. |
pdftk
|
V:0.9, I:4 | 3292 | pdf→pdf |
PDF document conversion tool: (pdftk )
|
mpage
|
V:0.2, I:1.9 | 224 | text,ps→ps | Print multiple pages per sheet. |
html2ps
|
V:0.3, I:2 | 260 | html→ps | The converter from HTML to PostScript. |
pdfjam
|
V:0.3, I:1.8 | 112 | pdf→pdf |
PDF document conversion tools: pdf90 , pdfjoin , and pdfnup
|
gnuhtml2latex
|
V:0.13, I:0.9 | 24 | html→latex | The converter from html to latex. |
latex2rtf
|
V:0.19, I:1.0 | 544 | latex→rtf | This converts documents from LaTeX to RTF which can be read by MS Word. |
ps2eps
|
V:1.4, I:10 | 116 | ps→eps | The converter from PostScript to EPS (Encapsulated PostScript). |
e2ps
|
V:0.02, I:0.17 | 188 | text→ps | Text to PostScript converter with Japanese encoding support. |
impose+
|
V:0.03, I:0.19 | 180 | ps→ps | Postscript utilities. |
trueprint
|
V:0.02, I:0.17 | 188 | text→ps | This pretty print many source codes (C, C++, Java, Pascal, Perl, Pike, Sh, and Verilog) to PostScript. (C language) |
pdf2svg
|
V:0.08, I:0.4 | 60 | ps→svg | Converter from PDF to Scalable vector graphics format. |
pdftoipe
|
V:0.04, I:0.18 | 648 | ps→ipe | Converter from PDF to IPE's XML format. |
Both lp
(1) and lpr
(1) commands offered by Common Unix Printing System (CUPS) provides options for customized printing the printable data.
For printing 3 copies of a file collated:
$ lp -n 3 -o Collate=True filename
, or
$ lpr -#3 -o Collate=True filename
You can further customize printer operation by using printer option such as "-o number-up=2
", "-o page-set=even
", "-o page-set=odd
", "-o scaling=200
", "-o natural-scaling=200
", etc., documented at Command-Line Printing and Options.
The Unix troff program originally developed by AT&T can be used for simple type setting. It is usually used to create manpages.
TeX created by Donald Knuth is very powerful type setting tool and is the de facto standard. LaTeX originally written by Leslie Lamport enables a high-level access to the power of TeX.
Table 11.14. List of type setting tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
texlive-base
|
V:6, I:20 | 17368 | (La)TeX | TeX system for typesetting, previewing and printing. |
groff
|
V:1.0, I:7 | 5540 | troff | GNU troff text-formatting system. |
Traditionally, roff
is the main Unix text processing system. See roff
(7), groff
(7), groff
(1), grotty
(1), troff
(1), groff_mdoc
(7), groff_man
(7), groff_ms
(7), groff_me
(7), groff_mm
(7), and "info groff
".
A good tutorial on "-me
" macros is availabe:
groff
package,
/usr/share/doc/groff/meintro.me.gz
", and
$ zcat /usr/share/doc/groff/meintro.me.gz | \ groff -Tascii -me - | less -R
The following will make a completely plain text file:
$ zcat /usr/share/doc/groff/meintro.me.gz | \ GROFF_NO_SGR=1 groff -Tascii -me - | col -b -x > meintro.txt
For printing, use PostScript output.
$ groff -Tps meintro.txt | lpr $ groff -Tps meintro.txt | mpage -2 | lpr
Preparation:
# aptitude install texlive
References for LaTeX:
tex
(1)
latex
(1)
This is the most powerful typesetting environment. Many SGML processors use this as their back end text processor. Lyx provided by the lyx
package and GNU TeXmacs provided by the texmacs
package offer nice WYSIWYG editing environment for LaTeX while many use Emacs and Vim as the choice for the source editor.
There are many online resources available:
/usr/share/doc/texlive-doc-base/english/texlive-en/live.html
") (texlive-doc-base
package)
When documents become bigger, sometimes TeX may cause errors. You must increase pool size in "/etc/texmf/texmf.cnf
" (or more appropriately edit "/etc/texmf/texmf.d/95NonPath
" and run update-texmf
(8)) to fix this.
![]() |
Note |
---|---|
The TeX source of "The TeXbook" is available at http://tug.ctan.org/tex-archive/systems/knuth/dist/tex/texbook.tex. |
This file contains most of the required macros. I heard that you can process this document with tex
(1) after commenting lines 7 to 10 and adding "\input manmac \proofmodefalse
". It's strongly recommended to buy this book (and all other books from Donald E. Knuth) instead of using the online version but the source is a great example of TeX input!
The following will print a manual page in PostScript and print it.
$ man -Tps some_manpage | lpr $ man -Tps some_manpage | mpage -2 | lpr
Although writing a manual page (manpage) in the plain troff format is possible, there are few helper packages to create it.
Table 11.15. List of packages to help creating the manpage.
package | popcon | size | keyword | function |
---|---|---|---|---|
docbook-to-man
|
V:0.5, I:3 | 248 | SGML→manpage | The converter from DocBook SGML into roff man macros. |
help2man
|
V:0.17, I:1.1 | 236 | text→manpage | Automatic manpage generator from --help. |
info2man
|
V:0.03, I:0.17 | 204 | info→manpage | The converter from GNU info to POD or man pages. |
txt2man
|
V:0.03, I:0.19 | 88 | text→manpage | Converts flat ASCII text to man page format. |
The following packages for the mail data conversion caught my eyes:
Table 11.16. List of packages to help mail data conversion.
package | popcon | size | keyword | function |
---|---|---|---|---|
sharutils
|
V:4, I:60 | 976 |
shar (1), unshar (1), uuencode (1), uudecode (1)
|
|
mpack
|
V:3, I:51 | 84 |
The encoder and decoder MIME messages: mpack (1) and munpack (1).
|
|
tnef
|
V:0.5, I:1.5 | 160 | unpacking MIME attachments of type "application/ms-tnef" which is a Microsoft only format. | |
uudeview
|
V:0.2, I:1.4 | 128 | The encoder and decoder for the following formats: uuencode, xxencode, BASE64, quoted printable, and BinHex | |
mimedecode
|
V:0.12, I:0.8 | 76 | This decodes transfer encoded text type MIME messages. | |
readpst
|
V:0.05, I:0.3 | 228 | windows/mail | This converts Outlook PST files to mbox format. |
![]() |
Tip |
---|---|
The Internet Message Access Protocol version 4 (IMAP4) server (see Section 6.2.5.5, “POP3/IMAP4 server”) may be used to move mails out from proprietary mail systems if the mail client software can be configured to use IMAP4 server too. |
Mail (SMTP) data should be limited to 7 bit. So binary data and 8 bit text data are encoded into 7 bit format with the Multipurpose Internet Mail Extensions (MIME) and the selection of the charset (see Section 8.3.1, “Basics of encoding”).
The standard mail storage format is mbox formatted according to RFC2822 (updated RFC822). See mbox
(5) (provided by the mutt
package).
For European languages, "Content-Transfer-Encoding: quoted-printable
" with the ISO-8859-1 charset is usually used since there are no much 8 bit characters. If the text is in UTF-8, "Content-Transfer-Encoding: quoted-printable
" is also used since it is mostly 7 bit data.
For Japanese, traditionally "Content-Type: text/plain; charset=ISO-2022-JP
" should be used to keep text in 7 bits. But mails from older Microsoft systems may use in Shift-JIS without proper declaration. For Japanese, if the text is in UTF-8, it contains many 8 bit data and is encoded into 7 bit data by Base64. The situation of other Asian languages is similar.
![]() |
Note |
---|---|
If your non-Unix mail data is accessible by a non-Debian client software which can talk to the IMAP4 server, you may be able to move them out by running your own IMAP4 server (see Section 6.2.5.5, “POP3/IMAP4 server”). |
![]() |
Note |
---|---|
If you use other mail storage formats, moving them to mbox format is the good first step. The versatile client program such as |
You can split mailbox contents to each message using procmail
(1) and formail
(1).
Each mail message can be unpacked using munpack
(1) from the mpack
package (or other specialized tools) to obtain the MIME encoded contents.
The following packages for the graphic data conversion, editing, and organization tools caught my eyes:
Table 11.17. List of graphic data tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
gimp
|
V:14, I:50 | 13468 | image(bitmap) | The GNU Image Manipulation Program. |
imagemagick
|
V:15, I:32 | 304 | image(bitmap) | Image manipulation programs. |
graphicsmagick
|
V:1.5, I:3 | 3696 | image(bitmap) |
Image manipulation programs. (folk of imagemagick )
|
xsane
|
V:7, I:42 | 744 | image(bitmap) | GTK+-based X11 frontend for SANE (Scanner Access Now Easy). |
netpbm
|
V:4, I:23 | 4408 | image(bitmap) | Graphics conversion tools. |
icoutils
|
V:0.07, I:0.5 | 200 | png↔ico(bitmap) | Converts MS Windows icons and cursors to and from PNG formats (favicon.ico) |
xpm2wico
|
V:0.03, I:0.13 | 80 | xpm→ico(bitmap) | Converts XPM to MS Windows icon formats |
scribus
|
V:0.7, I:3 | 26864 | ps/pdf/SVG/… | The Scribus DTP editor. |
openoffice.org-draw
|
V:22, I:47 | 8808 | image(vector) | OpenOffice.org office suite - drawing |
inkscape
|
V:12, I:28 | 61584 | image(vector) | The SVG (Scalable Vector Graphics) editor. |
dia-gnome
|
V:1.6, I:4 | 596 | image(vector) | Diagram editor (GNOME) |
dia
|
V:2, I:5 | 596 | image(vector) | Diagram editor (Gtk) |
xfig
|
V:2, I:5 | 1768 | image(vector) | Facility for Interactive Generation of figures under X11 |
pstoedit
|
V:1.1, I:9 | 880 | ps/pdf→image(vector) | PostScript and PDF files to editable vector graphics converter. (SVG) |
libwmf-bin
|
V:1.0, I:8 | 88 | Windows/image(vector) | Windows metafile (vector graphic data) conversion tools. |
fig2sxd
|
V:0.06, I:0.3 | 200 | fig→sxd(vector) | Convert XFig files to OpenOffice.org Draw format |
unpaper
|
V:0.2, I:1.4 | 736 | image→image | Post-processing tool for scanned pages for OCR. |
tesseract-ocr
|
V:0.4, I:2 | 2072 | image→text | Free OCR software based on the HP's commercial OCR engine. |
tesseract-ocr-eng
|
V:0.08, I:0.8 | 1760 | image→text | OCR engine data: tesseract-ocr language files for English text. |
clara
|
V:0.06, I:0.4 | NOT_FOUND | image→text | Free OCR software. |
gocr
|
V:1.1, I:6 | 484 | image→text | Free OCR software. |
ocrad
|
V:0.9, I:6 | 364 | image→text | Free OCR software. |
gtkam
|
V:0.3, I:2 | 1348 | image(Exif) | Manipulates digital camera photo files (GNOME) - GUI |
gphoto2
|
V:0.5, I:3 | 1008 | image(Exif) | Manipulates digital camera photo files (GNOME) - command line |
kamera
|
V:1.2, I:20 | 292 | image(Exif) | Manipulates digital camera photo files (KDE) |
jhead
|
V:0.6, I:3 | 128 | image(Exif) | Manipulates the non-image part of Exif compliant JPEG (digital camera photo) files |
exif
|
V:0.3, I:1.7 | 276 | image(Exif) | Command-line utility to show EXIF information in JPEG files |
exiftags
|
V:0.18, I:1.0 | 248 | image(Exif) | Utility to read Exif tags from a digital camera JPEG file |
exiftran
|
V:0.2, I:1.3 | 92 | image(Exif) | Transforms digital camera jpeg images |
exifprobe
|
V:0.06, I:0.3 | 484 | image(Exif) | Reads metadata from digital pictures |
dcraw
|
V:1.2, I:6 | 408 | image(Raw)→ppm | Decodes raw digital camera images |
findimagedupes
|
V:0.07, I:0.4 | 136 | image→fingerprint | Finds visually similar or duplicate images |
ale
|
V:0.03, I:0.2 | 812 | image→image | Merges images to increase fidelity or create mosaics |
imageindex
|
V:0.05, I:0.3 | 192 | image(Exif)→html | Generates static HTML galleries from images |
f-spot
|
V:0.6, I:1.8 | 10504 | image(Exif) | Personal photo management application (GNOME) |
bins
|
V:0.02, I:0.2 | 2008 | image(Exif)→html | Generates static HTML photo albums using XML and EXIF tags |
galrey
|
V:0.02, I:0.16 | 116 | image(Exif)→html | Generates browsable HTML photo albums with thumbnails |
outguess
|
V:0.03, I:0.16 | 252 | jpeg,png | Universal Steganographic tool |
qcad
|
V:1.3, I:2 | 3824 | DXF | CAD data editor (KDE) |
blender
|
V:0.7, I:3 | 28588 | blend, TIFF, VRML, … | 3D content editor for animation etc. |
open-font-design-toolkit
|
I:0.02 | 36 | ttf, ps, … | Metapackage for open font design |
fontforge
|
V:0.2, I:2 | 6112 | ttf, ps, … | Font editor for PS, TrueType and OpenType fonts |
xgridfit
|
V:0.00, I:0.05 | 724 | ttf | a program for gridfitting, or "hinting," TrueType fonts |
gbdfed
|
V:0.02, I:0.16 | 536 | bdf | Editor for BDF fonts |
![]() |
Tip |
---|---|
Search more image tools using regex " |
Although GUI programs such as gimp
(1) are very powerful, command line tools such as imagemagick
(1) are quite useful for automating image manipulation with the script.
The de facto image file format of the digital camera is the Exchangeable Image File Format (EXIF) which is the JPEG image file format with additional metadata tags. It can hold information such as date, time, and camera settings.
The Lempel-Ziv-Welch (LZW) lossless data compression patent has been expired. Graphics Interchange Format (GIF) utilities which use the LZW compression method are now freely available on the Debian system.
![]() |
Tip |
---|---|
Any digital camera or scanner with removable recording media will work with Linux through USB Mass Storage readers since it follows the Design rule for Camera File system. |
There are many other programs for converting data. Following packages caught my eyes using regex "~Guse::converting
" in aptitude
(8) (see Section 2.2.5, “Search method options with aptitude”):
Table 11.18. List of miscellaneous data conversion tools.
package | popcon | size | keyword | function |
---|---|---|---|---|
alien
|
V:1.6, I:12 | 276 | rpm/tgz→deb | The converter for the foreign package into the Debian package. |
freepwing
|
V:0.00, I:0.03 | 568 | EB→EPWING | The converter from "Electric Book" (popular in Japan) to a single JIS X 4081 format (a subset of the EPWING V1). |
You can also extract data from RPM format with:
$ rpm2cpio file.src.rpm | cpio --extract
I provide some pointers for people to learn programming on the Debian system enough to trace the packaged source code. Here are notable packages and corresponding documentation packages for programing.
Table 12.1. List of packages to help programing.
package | popcon | size | documentation |
---|---|---|---|
autoconf
|
V:4, I:26 | 1868 |
"info autoconf " provided by autoconf-doc
|
automake
|
V:3, I:18 | 1716 |
"info automake " provided by automake1.10-doc
|
bash
|
V:91, I:99 | 1336 |
"info bash " provided by bash-doc
|
bison
|
V:2, I:17 | 1820 |
"info bison " provided by bison-doc
|
cpp
|
V:45, I:85 | 76 |
"info cpp " provided by cpp-doc
|
ddd
|
V:0.4, I:3 | 4104 |
"info ddd " provided by ddd-doc
|
exuberant-ctags
|
V:1.3, I:6 | 288 |
exuberant-ctags (1)
|
flex
|
V:2, I:17 | 1004 |
"info flex " provided by flex-doc
|
gawk
|
V:24, I:28 | 2116 |
"info gawk " provided by gawk-doc
|
gcc
|
V:18, I:69 | 64 |
"info gcc " provided by gcc-doc
|
gdb
|
V:6, I:34 | 7128 |
"info gdb " provided by gdb-doc
|
gettext
|
V:9, I:51 | 7856 |
"info gettext " provided by gettext-doc
|
gfortran
|
V:1.3, I:6 | 40 |
"info gfortran " provided by gfortran-doc
|
glade
|
V:0.4, I:2 | 1397 | Help provided via menu |
glade-gnome
|
V:0.17, I:1.5 | 434 | Help provided via menu |
libc6
|
V:95, I:99 | 11496 |
"info libc " provided by glibc-doc and glibc-doc-reference
|
make
|
V:22, I:76 | 1220 |
"info make " provided by make-doc
|
mawk
|
V:69, I:99 | 248 |
mawk (1)
|
perl
|
V:89, I:99 | 18824 |
perl (1) and html pages provided by perl-doc and perl-doc-html
|
python
|
V:62, I:96 | 620 |
python (1) and html pages provided by python-doc
|
tcl8.4
|
V:8, I:45 | 3336 |
tcl (3) and detail manual pages provided by tcl8.4-doc
|
tk8.4
|
V:6, I:35 | 2800 |
tk (3) and detail manual pages provided by tk8.4-doc
|
ruby
|
V:10, I:25 | 100 |
ruby (1) and interactive reference provided by ri
|
vim
|
V:14, I:30 | 1740 |
Help(F1) menu provided by vim-doc
|
susv2
|
I:0.03 | 48 | Fetch "The Single Unix Specifications v2" |
susv3
|
I:0.09 | 48 | Fetch "The Single Unix Specifications v3" |
Online references are available by typing "man name
" after installing manpages
and manpages-dev
packages. Online references for the GNU tools are available by typing "info program_name
" after installing the pertinent documentation packages. You may need to include the contrib
and non-free
archives in addition to the main
archive since some GFDL documentations are not considered to be DSFG compliant.
![]() |
Warning |
---|---|
Do not use " |
![]() |
Caution |
---|---|
You should install software programs directly compiled from source into " |
![]() |
Tip |
---|---|
Code examples of creating "Song 99 Bottles of Beer" should give you good idea of practically all the programming languages. |
The shell script is a text file with the execution bit set and contains the commands in the following format.
#!/bin/sh ... command lines ...
The first line specifies the shell interpreter which read and execute this file contents.
Reading shell scripts is the best way to understand how a Unix-like system works. Here, I give some pointers and reminders for shell programming. See "Shell Mistakes" (http://www.greenend.org.uk/rjk/2001/04/shell.html) to learn from mistakes.
Unlike shell interactive mode (see Section 1.5, “The simple shell command” and Section 1.6, “Unix-like text processing”), parameters, conditionals, and loops are used frequently.
Many system scripts may be interpreted by any one of POSIX shells (see Table 1.14, “List of shell programs.”). The default shell for the system is "/bin/sh
" which is a symlink pointing to the actual program:
bash
(1) for lenny
or older.
dash
(1) for squeeze
or newer.
Avoid writing a shell script with "bashisms" or "zshisms" to make it portable among all POSIX shells. You can check it using checkbashisms
(1).
Table 12.2. List of typical bashisms.
Good: POSIX | Avoid: bashism |
---|---|
if [ "$foo" = "$bar" ] ; then …
|
if [ "$foo" == "$bar" ] ; then …
|
diff -u file.c.orig file.c
|
diff -u file.c{.orig, }
|
mkdir /foobar /foobaz
|
mkdir /foo{bar,baz }
|
octal format: "\377"
|
hexadecimal format: "\xff"
|
The "echo
" command must be used with care since its implementation differs among shell builtin commands and external command:
-n
". (Notably avoid "-e
" and "-E
")
![]() |
Note |
---|---|
Although " |
![]() |
Tip |
---|---|
Use the " |
Special shell parameters are frequently used in the shell script:
Table 12.3. List of shell parameters.
shell parameter | value |
---|---|
$0
|
name of the shell or shell script |
$1
|
first(1) shell argument |
$9
|
ninth(9) shell argument |
$#
|
number of positional parameters |
"$*"
|
"$1 $2 $3 $4 … "
|
"$@"
|
"$1" "$2" "$3" "$4" …
|
$?
|
exit status of the most recent command |
$$
|
PID of this shell script |
$!
|
PID of most recently started background job |
Basic parameter expansions to remember:
Table 12.4. List of shell parameter expansions.
parameter expression form |
value if var is set
|
value if var is not set
|
---|---|---|
${var:-string}
|
"$var "
|
"string "
|
${var:+string}
|
"string "
|
"null "
|
${var:=string}
|
"$var "
|
"string " (and run "var=string ")
|
${var:?string}
|
"$var "
|
echo "string " to stderr (and exit with error)
|
Here, the colon ":
" in all of these operators is actually optional.
:
" = operator test for exist and not null.
:
" = operator test for exist only.
Table 12.5. List of key shell parameter substitutions.
parameter substitution form | Result |
---|---|
${var%suffix}
|
Remove smallest suffix pattern |
${var%%suffix}
|
Remove largest suffix pattern |
${var#prefix}
|
Remove smallest prefix pattern |
${var##prefix}
|
Remove largest prefix pattern |
Each command returns an exit status which can be used for conditional expressions:
[
" is the equivalent of
the test
command, which evaluates its arguments up to "]
" as a
conditional expression.
Basic conditional idioms to remember are:
<command> && <if_success_run_this_command_too> || true
",
<command> || <if_not_success_run_this_command_too> || true
", and
if [ <conditional_expression> ]; then <if_success_run_this_command> else <if_not_success_run_this_command> fi
Here trailing "|| true
" was needed to ensure this shell script will not exit at this line accidentally when shell is invoked with "-e
" flag.
Table 12.6. List of file comparison operators in the conditional expression.
equation | value |
---|---|
-e <file>
|
True if <file> exists. |
-d <file>
|
True if <file> exists and is a directory. |
-f <file>
|
True if <file> exists and is a regular file. |
-w <file>
|
True if <file> exists and is writable. |
-x <file>
|
True if <file> exists and is executable. |
<file1> -nt <file2>
|
True if <file1> is newer than <file2>. (modification). |
<file1> -ot <file2>
|
True if <file1> is older than <file2>. (modification). |
<file1> -ef <file2>
|
True if they are the same device and inode number. |
Table 12.7. List of string comparison operators in the conditional expression.
equation | value |
---|---|
-z <str>
|
True if the length of <str> is zero. |
-n <str>
|
True if the length of <str> is non-zero. |
<str1> = <str2>
|
True if <str1> and <str2> are equal. |
<str1> != <str2>
|
True if <str1> and <str2> are not equal. |
<str1> < <str2>
|
True if <str1> sorts before <str2>. (locale dependent) |
<str1> > <str2>
|
True if <str1> sorts after <str2>. (locale dependent) |
Arithmetic integer comparison operators in the conditional expression are "-eq
", "-ne
", "-lt
", "-le
", "-gt
", and "-ge
".
There are several loop idioms to use in POSIX shell:
for name in word ; do list ; done
": loops over list of words.
while list; do list; done
": repeats while true.
until list; do list; done
": repeats while not true.
break
": enables to exit from the loop.
continue
" enables to resume the next iteration of the loop.
![]() |
Tip |
---|---|
The C-language like numeric iteration can be realized by using |
The shell processes a script as following sequence:
SPACE TAB NEWLINE ; ( ) < > | &
"…"
or '…'
(loop)
"…"
or '…'
(loop)
~<user>
" → <user>
's home directory, if not within "…"
or '…'
$PARAMETER
", if not within '…'
$( command )
", if not within '…'
$IFS
" if not within "…"
or '…'
* ? [ ]
in pathname if not within "…"
or '…'
look up command from:
$PATH
"
Single quotes within double quotes have no effect.
Executing "set -x
" in the shell or invoking the shell with "-x
" option make the shell to print all of commands executed. This is quite handy for debugging.
In order to make your shell program as portable as possible across Debian system, it is good idea to limit utility programs used within Essential programs listed by "aptitude search ~E
" as much as possible.
coreutils
, bsdutils
, and debianutils
packages contain many useful small utilities.
The user interface of a simple shell program can be improved from dull interaction by echo
and read
commands to more interactive one by using one of the so-called dialog program etc.
Table 12.8. List of user interface programs.
package | popcon | size | function |
---|---|---|---|
x11-utils
|
V:21, I:46 | 644 |
xmessage (1) displays a message or query in a window. (X)
|
whiptail
|
V:49, I:99 | 64 | displays user-friendly dialog boxes from shell scripts. (newt) |
dialog
|
V:5, I:24 | 1508 | displays user-friendly dialog boxes from shell scripts. (ncurses) |
zenity
|
V:6, I:46 | 4032 | display graphical dialog boxes from shell scripts. (gtk2.0) |
gtkdialog
|
V:0.07, I:0.3 | 488 | GUI-creation command-line utility based on GTK+ library. (gtk2.0+glade2) |
ssft
|
V:0.01, I:0.12 | 152 | Shell Scripts Frontend Tool. (wrapper for zenity, kdialog, and dialog with gettext) |
gettext
|
V:9, I:51 | 7856 |
"/usr/bin/gettext.sh " for translate message
|
Here is a simple script which creates ISO image with RS02 data supplemented by dvdisaster
(1):
#!/bin/sh -e # gmkrs02 : Copyright (C) 2007 Osamu Aoki <osamu@debian.org>, Public Domain #set -x error_exit() { echo "$1" >&2 exit 1 } # Initialize variables DATA_ISO="$HOME/Desktop/iso-$$.img" LABEL=$(date +%Y%m%d-%H%M%S-%Z) if [ $# != 0 ] && [ -d "$1" ]; then DATA_SRC="$1" else # Select directory for creating ISO image from folder on desktop DATA_SRC=$(zenity --file-selection --directory \ --title="Select the directory tree root to create ISO image") \ || error_exit "Exit on directory selection" fi # Check size of archive xterm -T "Check size $DATA_SRC" -e du -s $DATA_SRC/* SIZE=$(($(du -s $DATA_SRC | awk '{print $1}')/1024)) if [ $SIZE -le 520 ] ; then zenity --info --title="Dvdisaster RS02" --width 640 --height 400 \ --text="The data size is good for CD backup:\\n $SIZE MB" elif [ $SIZE -le 3500 ]; then zenity --info --title="Dvdisaster RS02" --width 640 --height 400 \ --text="The data size is good for DVD backup :\\n $SIZE MB" else zenity --info --title="Dvdisaster RS02" --width 640 --height 400 \ --text="The data size is too big to backup : $SIZE MB" error_exit "The data size is too big to backup :\\n $SIZE MB" fi # only xterm is sure to have working -e option # Create raw ISO image rm -f "$DATA_ISO" || true xterm -T "genisoimage $DATA_ISO" \ -e genisoimage -r -J -V "$LABEL" -o "$DATA_ISO" "$DATA_SRC" # Create RS02 supplemental redundancy xterm -T "dvdisaster $DATA_ISO" -e dvdisaster -i "$DATA_ISO" -mRS02 -c zenity --info --title="Dvdisaster RS02" --width 640 --height 400 \ --text="ISO/RS02 data ($SIZE MB) \\n created at: $DATA_ISO" # EOF
You may wish to create launcher on the desktop with command set something like "/usr/local/bin/gmkrs02 %d
".
Make is a utility to maintain groups of programs. Upon execution of make
(1), make
read the rule file, "Makefile
", and updates a target if it depends on prerequisite files that have been modified since the target was last modified, or if the target does not exist. The execution of these updates may occur concurrently.
The rule file syntax is :
target: [ prerequisites ... ] [TAB] command1 [TAB] -command2 # ignore errors [TAB] @command3 # suppress echoing
Here " [TAB]
" is a TAB code. Each line is interpreted by the shell after make variable substitution. Use "\
" at the end of a line to continue the script. Use "$$
" to enter "$
" for environment values for a shell script.
Implicit rules for the target and prerequisites can be written, for example, as:
%.o: %.c header.h
Here, the target contains the character "%
" (exactly one of them). The "%
" can match any nonempty substring in the actual target filenames. The prerequisites likewise use "%
" to show how their names relate to the actual target name.
Table 12.9. List of make automatic variables.
automatic variable | value |
---|---|
$@
|
target |
$<
|
first prerequisite |
$?
|
all newer prerequisites |
$^
|
all prerequisites |
$*
|
"% " matched stem in the target pattern
|
Table 12.10. List of make variable expansions.
variable expansion | description |
---|---|
foo1 := bar
|
One-time expansion |
foo2 = bar
|
Recursive expansion |
foo3 += bar
|
Append |
Run "make -p -f/dev/null
" to see automatic internal rules.
You can set up proper environment to compile programs written in the C programming language by:
# aptitude install glibc-doc manpages-dev libc6-dev gcc build-essential
The libc6-dev
package, i.e., GNU C Library, provides C standard library which is collection of header files and library routines used by the C programming language.
References for C:
info libc
" (C library function reference)
gcc
(1) and "info gcc
"
each_C_library_function_name
(3)
A simple example "example.c
" is compiled with a library "libm
" into an executable "run_example
":
$ cat > example.c << EOF #include <stdio.h> #include <math.h> #include <string.h> int main(int argc, char **argv, char **envp){ double x; char y[11]; x=sqrt(argc+7.5); strncpy(y, argv[0], 10); /* prevent buffer overflow */ y[10] = '\0'; /* fill to make sure string ends with '\0' */ printf("%5i, %5.3f, %10s, %10s\n", argc, x, y, argv[1]); return 0; } EOF $ gcc -Wall -g -o run_example example.c -lm $ ./run_example 1, 2.915, ./run_exam, (null) $ ./run_example 1234567890qwerty 2, 3.082, ./run_exam, 1234567890qwerty
Here, "-lm
" is needed to link library "/usr/lib/libm.so
" from the libc6
package for sqrt
(3). The actual library is in "/lib/
" with filename "libm.so.6
", which is a symlink to "libm-2.7.so
".
Look at the last parameter in the output text. There are more than 10 characters even though "%10s
" is specified.
The use of pointer memory operation functions without boundary checks, such as sprintf
(3) and strcpy
(3), is deprecated to prevent buffer overflow exploits that leverage the above overrun effects. Instead, use snprintf
(3) and strncpy
(3).
In order to be a good Debian user, you must be able to produce meaningful bug report using debugger. The fist step is to install gdb
:
# aptitude install gdb gdb-doc build-essential devscripts
Good tutorial of gdb
is provided by "info gdb
" or found elsewhere on the web.
Here is a simple example of using gdb
(1) on a "program
" compiled with the "-g
" option to produce debugging information.
$ gdb program (gdb) b 1 # set break point at line 1 (gdb) run args # run program with args (gdb) next # next line ... (gdb) step # step forward ... (gdb) p parm # print parm ... (gdb) p parm=12 # set value to 12 ... (gdb) quit
![]() |
Tip |
---|---|
Many |
Since all installed binaries should be stripped on the Debian system by default, most debugging symbols are removed in the normal package. In order to debug Debian packages with gdb
(1), corresponding *-dbg
packages need to be installed (e.g. libc6-dbg in the case of libc6).
If a package to be debugged does not provide its *-dbg
package, you need to install it after rebuilding it:
$ mkdir /path/new ; cd /path/new $ sudo aptitude update $ sudo aptitude dist-upgrade $ sudo aptitude install fakeroot devscripts build-essential $ sudo apt-get build-dep source_package_name $ apt-get source package_name $ cd package_name*
$ dch -i
+debug1
" when recompiling existing package version, or one appended with "~pre1
" when compiling unreleased package version.
$ export DEB_BUILD_OPTIONS=nostrip,noopt $ debuild $ cd .. $ sudo debi package_name*.changes
You need to check build scripts of the package and ensure to use "CFLAGS=-g -Wall
" for compiling binaries.
When you encounter program crash, reporting bug report with cut-and-pasted backtrace information is a good idea.
The backtrace can be obtained by the following steps:
gdb
(1),
gdb
prompt), and
bt
" at the gdb
prompt.
In case of program freeze, you can crash the program by pressing Ctrl-C
in the terminal running gdb
to obtain gdb
prompt.
![]() |
Tip |
---|---|
Often, you will see a backtrace where one or more of the top lines is in " |
$ MALLOC_CHECK_=2 gdb hello
Table 12.11. List of advanced gdb commands
objective | commands |
---|---|
To get a backtrace for all threads for multi-threaded program. |
(gdb) thread apply all bt
|
To get parameters came on the stack of function calls. |
(gdb) bt full
|
To get a backtrace and parameters as the combination of the preceding options. |
(gdb) thread apply all bt full
|
To get them for top 10 calls to cut off irrelevant output. |
(gdb) thread apply all bt full 10
|
To write log of gdb output to a file (the default is gdb.txt). |
(gdb) set logging on
|
Use ldd
(1) to find out a program's dependency on libraries:
$ ldd /bin/ls librt.so.1 => /lib/librt.so.1 (0x4001e000) libc.so.6 => /lib/libc.so.6 (0x40030000) libpthread.so.0 => /lib/libpthread.so.0 (0x40153000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
For ls
(1) to work in a `chroot`ed environment, the above libraries must be available in your `chroot`ed environment.
There are several memory leak detection tools available in Debian.
Table 12.12. List of memory leak detection tools
package | popcon | size | description |
---|---|---|---|
libc6-dev
|
V:39, I:68 | 11152 |
mtrace (1): malloc debugging functionality in glibc
|
valgrind
|
V:1.2, I:6 | 129599 | memory debugger and profiler |
kmtrace
|
V:0.19, I:1.7 | 324 |
KDE memory leak tracer using glibc's mtrace (1)
|
alleyoop
|
V:0.06, I:0.4 | 1516 | GNOME front-end to the Valgrind memory checker |
electric-fence
|
V:0.06, I:0.9 | 108 |
malloc (3) debugger
|
ccmalloc
|
V:0.05, I:0.4 | 232 | memory profiler/debugger |
leaktracer
|
V:0.01, I:0.13 | 116 | memory-leak tracer for C++ programs |
libdmalloc5
|
V:0.02, I:0.13 | 356 | debug memory allocation library |
mpatrolc2
|
V:0.00, I:0.03 | 3592 | library for debugging memory allocations |
You can disassemble binary code with objdump
(1). For example:
$ objdump -m i386 -b binary -D /usr/lib/grub/x86_64-pc/stage1
![]() |
Note |
---|---|
|
Flex is a a Lex-compatible fast lexical analyzer generator.
Tutorial for flex
(1) can be found in "info flex
".
You need to provide your own "main()
" and "yywrap()
", or your "program.l
" should look like this to compile without a library ("yywrap
" is a macro; "%option main
" turns on "%option noyywrap
" implicitly):
%option main %% .|\n ECHO ; %%
Alternatively, you may compile with the "-lfl
" linker option at the end of your cc
(1) command line (like AT&T-Lex with "-ll
"). No "%option
" is needed in this case.
Several packages provide a Yacc-compatible lookahead LR parser or LALR parser generator in Debian:
Table 12.13. List of Yacc-compatible LALR parser generators
package | popcon | size | description |
---|---|---|---|
bison
|
V:2, I:17 | 1820 | GNU LALR parser generator |
byacc
|
V:0.15, I:1.2 | 160 | The Berkeley LALR parser generator |
btyacc
|
V:0.00, I:0.06 | 248 |
Backtracking parser generator based on byacc
|
Tutorial for bison
(1) can be found in "info bison
".
You need to provide your own "main()
" and "yyerror()
". "main()
" calls "yyparse()
" which calls "yylex()
", usually created with Flex.
%% %%
Autoconf is a tool for producing shell scripts that automatically configure software source code packages to adapt to many kinds of Unix-like systems using the entire GNU build system.
autoconf
(1) produces the configuration script "configure
". "configure
" automatically creates a customized "Makefile
" using the "Makefile.in
" template.
![]() |
Warning |
---|---|
Do not overwrite system files with your compiled programs when installing them. |
Debian does not touch files in "/usr/local/
" or "/opt
". So if you compile a program from source, install it into "/usr/local/
" so it will not interfere with Debian.
$ cd src $ ./configure --prefix=/usr/local $ make $ make install # this puts the files in the system
If you still have the source and if it uses autoconf
(1)/automake
(1) and if you can remember how you configured it:
$ ./configure ''all-of-the-options-you-gave-it'' # make uninstall
Alternatively, if you are absolutely sure that the install process puts files only under "/usr/local/
" and there is nothing important there, you can erase all its contents by:
# find /usr/local -type f -print0 | xargs -0 rm -f
If you are not sure where files are installed, you should consider using checkinstall
(8) from the checkinstall
package, which provides a clean path for the uninstall. It now supports to create a Debian package with "-D
" option.
Although any AWK scripts can be automatically rewritten in Perl using a2p
(1), one-liner AWK scripts are best converted to one-liner perl scripts manually. For example
awk '($2=="1957") { print $3 }' |
is equivalent to any one of the following lines:
perl -ne '@f=split; if ($f[1] eq "1957") { print "$f[2]\n"}' |
perl -ne 'if ((@f=split)[1] eq "1957") { print "$f[2]\n"}' |
perl -ne '@f=split; print $f[2] if ( $f[1]==1957 )' |
perl -lane 'print $F[2] if $F[1] eq "1957"' |
perl -lane 'print$F[2]if$F[1]eq+1957' |
The last one is a riddle. It took advantage of the Perl features that
perlrun
(1) for the command-line options. For more crazy Perl scripts, Perl Golf may be interesting.
Basic interactive dynamic web pages can be made as follows:
Filling and clicking on the form entries will send an URL with encoded parameters from the browser to the web server. For example:
http://www.foo.dom/cgi-bin/program.pl?VAR1=VAL1&VAR2=VAL2&VAR3=VAL3
"
http://www.foo.dom/cgi-bin/program.py?VAR1=VAL1&VAR2=VAL2&VAR3=VAL3
"
http://www.foo.dom/program.php?VAR1=VAL1&VAR2=VAL2&VAR3=VAL3
"
%nn
" in URL is replaced with a character with hexadecimal nn
value.
QUERY_STRING="VAR1=VAL1 VAR2=VAL2 VAR3=VAL3"
"
program.*
") on the web server executes itself with the environment variable "$QUERY_STRING
".
stdout
of CGI program will be sent to the web browser and is presented as an interactive dynamic web page.
For security reasons it is better not to hand craft new hacks for parsing CGI parameters. There are established modules for them in Perl and Python. PHP comes with these functionalities. When client data storage is needed, cookies are used. When client side data processing is needed, javascript is frequently used.
For more, see The Common Gateway Interface, The Apache Software Foundation, and JavaScript.
Searching "CGI tutorial" on Google by typing encoded URL http://www.google.com/search?hl=en&ie=UTF-8&q=CGI+tutorial directly to the browser address is a good way to see the CGI script in action on the Google server.
There are lint like tools for static code analysis:
Table 12.14. List of tools for static code analysis
package | popcon | size | description |
---|---|---|---|
splint
|
V:0.06, I:0.5 | 1836 | A tool for statically checking C programs for bugs |
rats
|
V:0.06, I:0.2 | 768 | Rough Auditing Tool for Security (C, C++, PHP, Perl, and Python code) |
flawfinder
|
V:0.03, I:0.2 | 192 | A tool to examine C/C++ source code and looks for security weaknesses |
perl
|
V:89, I:99 | 18824 |
This package has internal code static checker: B::Lint (3perl)
|
pylint
|
V:0.10, I:0.5 | 688 | A python code static checker |
jlint
|
V:0.01, I:0.10 | 184 | A Java program checker |
weblint-perl
|
V:0.14, I:0.8 | 64 | A syntax and minimal style checker for HTML |
linklint
|
V:0.06, I:0.3 | 432 | A fast link checker and web site maintenance tool |
libxml2-utils
|
V:5, I:53 | 120 |
This package provides xmllint (1) to validate XML files
|
There are programs to convert source codes:
Table 12.15. List of source code translation tools.
package | popcon | size | keyword | description |
---|---|---|---|---|
perl
|
V:89, I:99 | 18824 | AWK→PERL |
a2p (1) converts source codes from AWK to PERL.
|
f2c
|
V:0.17, I:1.1 | 440 | FORTRAN→C |
f2c (1) converts source codes from A FORTRAN 77 to C/C++.
|
protoize
|
V:0.01, I:0.07 | 172 | ANSI C | Create/remove ANSI prototypes from C code. |
intel2gas
|
V:0.01, I:0.08 | 344 | intel→gas | The converter from NASM (intel format) to the GNU Assembler (GAS). |
If you want to make a Debian package, read:
debuild
(1), pbuilder
(1) and pdebuild
(1),
maint-guide
package),
developers-reference
package), and
debian-policy
package).
There are packages such as dh-make
, dh-make-perl
, etc., which help packaging.
Here are backgrounds of this document.
The Linux system is a very powerful computing platform for a networked computer. However, learning how to use all its capabilities is not easy. Setting up the LPR printer with non-PostScript printer was a good example of stumble points. (There are no issues anymore since newer installations use new CUPS system.)
There is a complete, detailed map called the "SOURCE CODE". This is very accurate but very hard to understand. There are also references called HOWTO and mini-HOWTO. They are easier to understand but tend to give too much detail and lose the big picture. I sometimes have a problem finding the right section in a long HOWTO when I need a few commands to invoke.
I hope this "Debian Reference (version 2)" will provide good starting direction for people in the Debian maze.
Debian Reference was initiated by Osamu Aoki <osamu at debian dot org> as a personal system administration memo. Many contents came from the knowledge I gained from the debian-user mailing list and other Debian resources.
Following a suggestion from Josip Rodin, who was very active with the Debian Documentation Project (DDP), "Debian Reference (version 1, 2001-2007)" was created as a part of DDP documents.
After 6 years, Osamu realized that the original "Debian Reference (version 1)" was outdated and started to rewrite many contents. New "Debian Reference (version 2)" is released in 2008.
The tutorial contents can trace its origin and its inspiration in:
"Linux User's Guide" by Larry Greenfield. (December 1996)
"Debian Tutorial" by Havoc Pennington. (11 December, 1998)
"Debian GNU/Linux: Guide to Installation and Usage" by John Goerzen and Ossama Othman. (1999)
The package and archive description can trace some of their origin and their inspiration in:
The other contents can trace some of their origin and their inspiration in:
"Debian Reference (version 1)" by Osamu Aoki. (2001–2007)
The previous "Debian Reference (version 1)" was created with,
Many manual pages and info pages on the Debian system were used as the primary references to write this document. To the extent Osamu Aoki considered within the fair use, many parts of them, especially command definitions, were used as phrase pieces after careful editorial efforts to fit them into the style and the objective of this document.
The gdb debugger description was expanded using Debian wiki contents on backtrace with consent by Ari Pollak, Loïc Minier, and Dafydd Harries.
Contents of "Debian Reference (version 2)" are mostly my own work except as mentioned above. These has been updated by the contributors too.
The author, Osamu Aoki, thanks all those who helped make this document possible.
The source of the English original document is written in AsciiDoc text files. It is less typing than straight XML and supports table in very intuitive format. Via build script, it is converted to DocBook XML format and automatically generated data are inserted to form a final Docbook XML source. This final Docbook XML source can be converted to HTML, plain text, PostScript, and PDF. Currently, only HTML and plain text conversions are enabled.