Debian Reference

Upon starting the system, you are presented with the character based login screen if you did not install X Window System with the display manager such as gdm. Suppose your hostname is foo, the login prompt looks like:

foo login:

If you did install a GUI environment such as GNOME or KDE, then you can get to a login prompt by Ctrl-Alt-F1, and you can return to the GUI environment via Alt-F7 (see Section 1.1.6, “Virtual consoles” below for more).

At the login prompt, you type your username, e.g. penguin, and press the Enter-key, then type your password and press the Enter-key again.

The system starts with the greeting message stored in "/etc/motd" (Message Of The Day) and with the command prompt as:

Debian GNU/Linux lenny/sid foo tty1
foo login: penguin
Password:
Last login: Sun Apr 22 09:29:34 2007 on tty1
Linux snoopy 2.6.20-1-amd64 #1 SMP Sun Apr 15 20:25:49 UTC 2007 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
foo:~$

Here, the main part of the greeting message can be customized by editing the "/etc/motd.tail" file. The first line is generated from the system information using "uname -snrvm".

Now you are in the shell. The shell interprets your commands.

If you installed X Window System with a display manager such as GNOME's gdm by selecting "Desktop environment" task during the installation, you will be presented with the graphical login screen upon starting your system. You type your username and your password to login to the non-privileged user account. Use tab to navigate between username and password, or use the mouse and primary click.

You can gain the shell prompt under X by starting a x-terminal-emulator program such as gnome-terminal(1), rxvt(1) or xterm(1). Under the GNOME Desktop environment, clicking "Applications" → "Accessories" → "Terminal" does the trick.

You can also see the section below Section 1.1.6, “Virtual consoles”.

Under some other Desktop systems (like fluxbox), there may be no obvious starting point for the menu. If this happens, just try (right) clicking the center of the screen and hope for a menu to pop-up.

The root account is also called superuser or privileged user. From this account, you can perform the following system administration activities:

This unlimited power of root account requires you to be considerate and responsible when using it.

Here are a few basic methods to gain the root shell prompt by using the root password:

When your desktop menu does not start GUI system administration tools automatically with the appropriate privilege, you can start them from the root shell prompt of the X terminal emulator, such as gnome-terminal(1), rxvt(1), or xterm(1). See Section 1.1.4, “The root shell prompt” and Section 7.8.4, “Running X clients as root”.

In the default Debian system, there are six switchable VT100-like character consoles available to start the command shell directly on the Linux host. Unless you are in a GUI environment, you can switch between the virtual consoles by pressing the Left-Alt-key and one of the F1 — F6 keys simultaneously. Each character console allows independent login to the account and offers the multiuser environment. This multiuser environment is a great Unix feature, and very addictive.

If you are under the X Window System, you gain access to the character console 1 by pressing Ctrl-Alt-F1 key, i.e., the left-Ctrl-key, the left-Alt-key, and the F1-key are pressed together. You can get back to the X Window System, normally running on the virtual console 7, by pressing Alt-F7.

You can alternatively change to another virtual console, e.g. to the console 1, by the command:

# chvt 1

You type Ctrl-D, i.e., the left-Ctrl-key and the d-key pressed together, at the command prompt to close the shell activity. If you are at the character console, you will return to the login prompt with this. Even though these control characters are referred as "control D" with the upper case, you do not need to press the Shift-key. The short hand expression, ^D, is also used for Ctrl-D. Alternately, you can type "exit".

If you are at x-terminal-emulator(1), you can close x-terminal-emulator window with this.

Just like any other modern OS where the file operation involves caching data in memory for improved performance, the Debian system needs the proper shutdown procedure before power can safely be turned off. This is to maintain the integrity of files, by forcing all changes in memory to be written to disk. If the software power control is available, the shutdown procedure automatically turns off power of the system. (Otherwise, you may have to press power button for few seconds after the shutdown procedure.)

Under the normal multiuser mode, use following from the root command prompt to shutdown the system:

# shutdown -h now

Under the single-user mode, use following from the root command prompt to shutdown the system:

# poweroff -i -f

Alternatively, you may type Ctrl-Alt-Delete (The left-Ctrl-key, the left-Alt-Key, and the Delete are pressed together) to shutdown if "/etc/inittab" contains "ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -h now" in it. See inittab(5) for details.

When the screen goes berserk after doing some funny things such as "cat <some-binary-file>", type "reset" at the command prompt. You may not be able to see the command echoed as you type. You may also issue "clear" to clean up the screen.

Although even the minimal installation of the Debian system without any desktop environment tasks provides the basic Unix functionality, it is a good idea to install few additional commandline and curses based character terminal packages such as mc and vim with aptitude(8) for beginners to get started. From the shell prompt as root:

# aptitude update
...
# aptitude install mc vim sudo
...

If you already had these packages installed, nothing will be installed.

It may be a good idea to read some informative documentations.

You can install some of these packages by issuing the following command from the root shell prompt:

# aptitude install package_name

If you do not want to use your main user account for the following training activities, you can create a training user account, e.g. fish. Type at root shell prompt:

# adduser fish

This will create a new account named as fish. After your practice, you can remove this user account and its home directory by:

# deluser --remove-home fish

For the typical single user workstation such as the desktop Debian system on the laptop PC, it is common to deploy simple configuration of sudo(8) as follows to let the non-privileged user, e.g. penguin, to gain administrative privilege just with his user password (not with the root password).

# echo "penguin  ALL=(ALL) ALL" >> /etc/sudoers

This trick should only be used for the single user workstation which you administer and where you are the only user.

Now you are ready to play with the Debian system without risks as long as you use the non-privileged user account.

This is because the Debian system is, even after the default installation, configured with proper file permissions which prevent non-privileged users from damaging the system. Of course, there may still be some holes which can be exploited but those who worry about these issues should not be reading this section but should be reading Securing Debian Manual.

We will learn the Debian system as a Unix-like system with:

Here are Unix file basics:

The detailed best practices for the file hierarchy are described in the Filesystem Hierarchy Standard ("/usr/share/doc/debian-policy/fhs/fhs-2.3.txt.gz" and hier(7)). You should remember the following facts as the starter:

Following the Unix tradition, the Debian GNU/Linux system provides the filesystem under which physical data on harddisks and other storage devices reside, and the interaction with the hardware devices such as console screens and remote serial consoles are represented in an unified manner under "/dev/".

Each file, directory, named pipe (a way two programs can share data), or physical device on a Debian GNU/Linux system has a data structure called an inode which describes its associated attributes such as the user who owns it (owner), the group that it belongs to, the time last accessed, etc. If you are really interested, see "/usr/include/linux/fs.h" for the exact definition of "struct inode" in the Debian GNU/Linux system. The idea of representing just about everything in the file system was a Unix innovation, and modern Linux kernels have developed this idea ever further. Now, even information about processes running in the computer can be found in the file system.

This abstract and unified representation of physical entities and internal processes is very powerful since this allows us to use the same command for the same kind of operation on many totally different devices. It is even possible to change the way the kernel works by writing data to special files that are linked to running processes.

Filesystem permissions of Unix-like system are defined for three categories of affected users:

For the file, each corresponding permission allows:

For the directory, each corresponding permission allows:

Here, the execute permission on a directory means not only to allow reading of files in that directory but also to allow viewing their attributes, such as the size and the modification time.

ls(1) is used to display permission information (and more) for files and directories. When it is invoked with the "-l" option, it displays the following information in the order given:

chown(1) is used from the root account to change the owner of the file. chgrp(1) is used from the file's owner or root account to change the group of the file. chmod(1) is used from the file's owner or root account to change file and directory access permissions. Basic syntax to manipulate a foo file is:

# chown <newowner> foo
# chgrp <newgroup> foo
# chmod  [ugoa][+-=][rwxXst][,...] foo

For example, in order to make a directory tree to be owned by a user foo and shared by a group bar, issue following commands from the root account:

# cd /some/location/
# chown -R foo:bar .
# chmod -R ug+rwX,o=rX .

There are three more special permission bits:

Here the output of "ls -l" for these bits is capitalized if execution bits hidden by these outputs are unset.

Setting set user ID on an executable file allows a user to execute the executable file with the owner ID of the file (for example root). Similarly, setting set group ID on an executable file allows a user to execute the executable file with the group ID of the file (for example root). Because these settings can cause security risks, enabling them requires extra caution.

Setting set group ID on a directory enables the BSD-like file creation scheme where all files created in the directory belong to the group of the directory.

Setting the sticky bit on a directory prevents a file in the directory from being removed by a user who is not the owner of the file. In order to secure contents of a file in world-writable directories such as "/tmp" or in group-writable directories, one must not only reset the write permission for the file but also set the sticky bit on the directory. Otherwise, the file can be removed and a new file can be created with the same name by any user who has write access to the directory.

Here are a few interesting examples of file permissions.

$ ls -l /etc/passwd /etc/shadow /dev/ppp /usr/sbin/exim4
crw------- 1 root root   108, 0 2007-04-29 07:00 /dev/ppp
-rw-r--r-- 1 root root     1427 2007-04-16 00:19 /etc/passwd
-rw-r----- 1 root shadow    943 2007-04-16 00:19 /etc/shadow
-rwsr-xr-x 1 root root   700056 2007-04-22 05:29 /usr/sbin/exim4
$ ls -ld /tmp /var/tmp /usr/local /var/mail /usr/src
drwxrwxrwt 10 root root  4096 2007-04-29 07:59 /tmp
drwxrwsr-x 10 root staff 4096 2007-03-24 18:48 /usr/local
drwxrwsr-x  4 root src   4096 2007-04-27 00:31 /usr/src
drwxrwsr-x  2 root mail  4096 2007-03-28 23:33 /var/mail
drwxrwxrwt  2 root root  4096 2007-04-29 07:11 /var/tmp

There is an alternative numeric mode to describe file permissions with chmod(1). This numeric mode uses 3 to 4 digit wide octal (radix=8) numbers.

This sounds complicated but it is actually quite simple. If you look at the first few (2-10) columns from "ls -l" command output and read it as a binary (radix=2) representation of file permissions ("-" being "0" and "rwx" being "1"), the last 3 digit of the numeric mode value should make sense as an octal (radix=8) representation of file permissions to you. For example, try:

$ touch foo bar
$ chmod u=rw,go=r foo
$ chmod 644 bar
$ ls -l foo bar
-rw-r--r-- 1 penguin penguin 17 2007-04-29 08:22 bar
-rw-r--r-- 1 penguin penguin 12 2007-04-29 08:22 foo

What permissions are applied to a newly created file or directory is restricted by the umask shell builtin command. See dash(1), bash(1), and builtins(7).

 (file permissions) = (requested file permissions) & ~(umask value)

The Debian system uses a user private group (UPG) scheme as its default. A UPG is created whenever a new user is added to the system. A UPG has the same name as the user for which it was created and that user is the only member of the UPG. UPG scheme makes it is safe to set umask to 0002 since every user has their own private group. (In some Unix variants, it is quite common to setup all normal users belonging to a single users group and is good idea to set umask to 0022 for security in such cases.)

In order to make group permissions to be applied to a particular user, that user needs to be made a member of the group using "sudo vigr".

The hardware devices are just another kind of file on the Debian system. If you have problems accessing devices such as CD-ROM and USB memory stick from a user account, you should make that user a member of the relevant group.

Some notable system-provided groups allow their members to access particular files and devices without root privilege.

Some notable system-provided groups allow their members to execute particular commands without root privilege.

For the full listing of the system provided users and groups, see the recent version of the "Users and Groups" document in "/usr/share/doc/base-passwd/users-and-groups.html" provided by the base-passwd package.

See passwd(5), group(5), shadow(5), newgrp(1), vipw(8), vigr(8), and pam_group(8) for management commands of the user and group system.

There are three types of timestamps for a GNU/Linux file.

Note that ctime is not file creation time.

Use touch(1) command to change timestamps of existing files.

For timestamps, the ls command outputs different strings under the modern English locale ("en_US.UTF-8") from under the old one ("C").

$ LANG=en_US.UTF-8  ls -l foo
-rw-r--r-- 1 penguin penguin 3 2008-03-05 00:47 foo
$ LANG=C  ls -l foo
-rw-r--r-- 1 penguin penguin 3 Mar  5 00:47 foo

There are two methods of associating a file "foo" with a different filename "bar".

See the following example for changes in link counts and the subtle differences in the result of the rm command.

$ echo "Original Content" > foo
$ ls -li foo
2398521 -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:15 foo
$ ln foo bar     # hard link
$ ln -s foo baz  # symlink
$ ls -li foo bar baz
2398521 -rw-r--r-- 2 penguin penguin 17 2007-04-29 08:15 bar
2398538 lrwxrwxrwx 1 penguin penguin  3 2007-04-29 08:16 baz -> foo
2398521 -rw-r--r-- 2 penguin penguin 17 2007-04-29 08:15 foo
$ rm foo
$ echo "New Content" > foo
$ ls -li foo bar baz
2398521 -rw-r--r-- 1 penguin penguin 17 2007-04-29 08:15 bar
2398538 lrwxrwxrwx 1 penguin penguin  3 2007-04-29 08:16 baz -> foo
2398540 -rw-r--r-- 1 penguin penguin 12 2007-04-29 08:17 foo
$ cat bar
Original Content
$ cat baz
New Content

The hardlink can be made within the same file system and shares the same inode number which the "-i" option with ls(1) reveals.

The symlink always has nominal file access permissions of "rwxrwxrwx", as shown in the above example, with the effective access permissions dictated by permissions of the file that it points to.

The "." directory links to the directory that it appears in, thus the link count of any new directory starts at 2. The ".." directory links to the parent directory, thus the link count of the directory increases with the addition of new subdirectories.

If you are just moving to Linux from Windows, it will soon become clear how well-designed the filename linking of Unix is, compared with the nearest Windows equivalent of "shortcuts". Because it is implemented in the file system, applications can't see any difference between a linked file and the original. In the case of hardlinks, there really is no difference.

A named pipe is a file that acts like a pipe. You put something into the file, and it comes out the other end. Thus it's called a FIFO, or First-In-First-Out: the first thing you put in the pipe is the first thing to come out the other end.

If you write to a named pipe, the process which is writing to the pipe doesn't terminate until the information being written is read from the pipe. If you read from a named pipe, the reading process waits until there is nothing to read before terminating. The size of the pipe is always zero --- it does not store data, it just links two processes like the shell "|". However, since this pipe has a name, the two processes don't have to be on the same command line or even be run by the same user. Pipes were a very influential innovation of Unix.

You can try it by doing the following:

$ cd; mkfifo mypipe
$ echo "hello" >mypipe & # put into background
[1] 8022
$ ls -l mypipe
prw-r--r-- 1 penguin penguin 0 2007-04-29 08:25 mypipe
$ cat mypipe
hello
[1]+  Done                    echo "hello" >mypipe
$ ls mypipe
mypipe
$ rm mypipe

Sockets are used extensively by all the Internet communication, databases, and the operating system itself. It is similar to the named pipe (FIFO) and allows processes to exchange information even between different computers. For the socket, those processes do not need to be running at the same time nor to be running as the children of the same ancestor process. This is the endpoint for the inter process communication (IPC). The exchange of information may occur over the network between different hosts. The two most common ones are the Internet socket and the Unix domain socket.

Device files refer to physical or virtual devices on your system, such as your hard disk, video card, screen, or keyboard. An example of a virtual device is the console, represented by "/dev/console".

You can read and write device files, though the file may well contain binary data which may be an incomprehensible-to-humans gibberish. Writing data directly to these files is sometimes useful for the troubleshooting of hardware connections. For example, you can dump a text file to the printer device "/dev/lp0" or send modem commands to the appropriate serial port "/dev/ttyS0". But, unless this is done carefully, it may cause a major disaster. So be cautious.

The device node number are displayed by executing ls(1) as:

$ ls -l /dev/hda /dev/ttyS0 /dev/zero
brw-rw---- 1 root cdrom   3,  0 2007-04-29 07:00 /dev/hda
crw-rw---- 1 root dialout 4, 64 2007-04-29 07:00 /dev/ttyS0
crw-rw-rw- 1 root root    1,  5 2007-04-29 07:00 /dev/zero

Here,

In the Linux 2.6 system, the filesystem under "/dev/" is automatically populated by the udev(7) mechanism.

There are some special device files.

These are frequently used in conjunction with the shell redirection (see Section 1.5.5, “Typical command sequences and shell redirection”).

The procfs and sysfs mounted on "/proc" and "/sys" are the pseudo-filesystem and expose internal data structures of the kernel to the userspace. In other word, these entries are virtual, meaning that they act as a convenient window into the operation of the operating system.

The directory "/proc" contains (among other things) one subdirectory for each process running on the system, which is named after the process ID (PID). System utilities that access process information, such as ps(1), get their information from this directory structure.

The directories under "/proc/sys/" contain interface to change certain kernel parameters at run time. (You may do the same through specialized sysctl(8) command or its preload/configuration file "/etc/sysctrl.conf".)

People frequently panic when they notice one file in particular - "/proc/kcore" - which is generally huge. This is (more or less) a copy of the content of your computer's memory. It's used to debug the kernel. It is a virtual file that points to computer memory, so don't worry about its size.

The directory under "/sys" contains exported kernel data structures, their attributes, and their linkages between them. It also contains interface to change certain kernel parameters at run time.

See "proc.txt(.gz)", "sysfs.txt(.gz)" and other related documents in the Linux kernel documentation ("/usr/share/doc/linux-doc-2.6.*/Documentation/filesystems/*") provided by the linux-doc-2.6.* package.

$ sudo aptitude install mc

In order to make MC to change working directory upon exit and cd to the directory, I suggest to modify "~/.bashrc" to include:

. /usr/share/mc/bin/mc.sh

See mc(1) (under the "-P" option) for the reason. (If you do not understand what exactly I am talking here, you can do this later.)

MC can be started by:

$ mc

MC takes care of all file operations through its menu, requiring minimal user effort. Just press F1 to get the help screen. You can play with MC just by pressing cursor-keys and function-keys.

If you encounter character encoding problem which displays garbage characters, adding "-a" to MC's command line may help prevent problems.

If this doesn't clear up your display problems with MC, see Section 9.6.6, “The terminal configuration”.

The default is two directory panels containing file lists. Another useful mode is to set the right window to "information" to see file access privilege information, etc. Following are some essential keystrokes. With the gpm(8) daemon running, one can use a mouse on Linux character consoles, too. (Make sure to press the shift-key to obtain the normal behavior of cut and paste in MC.)

The internal editor has an interesting cut-and-paste scheme. Pressing F3 marks the start of a selection, a second F3 marks the end of selection and highlights the selection. Then you can move your cursor. If you press F6, the selected area will be moved to the cursor location. If you press F5, the selected area will be copied and inserted at the cursor location. F2 will save the file. F10 will get you out. Most cursor keys work intuitively.

This editor can be directly started on a file:

$ mc -e filename_to_edit
$ mcedit filename_to_edit

This is not a multi-window editor, but one can use multiple Linux consoles to achieve the same effect. To copy between windows, use Alt-F<n> keys to switch virtual consoles and use "File→Insert file" or "File→Copy to file" to move a portion of a file to another file.

This internal editor can be replaced with any external editor of choice.

Also, many programs use the environment variables "$EDITOR" or "$VISUAL" to decide which editor to use. If you are uncomfortable with vim(1) or nano(1) initially, you may set these to "mcedit" by adding these lines to "~/.bashrc":

...
export EDITOR=mcedit
export VISUAL=mcedit
...

I do recommend setting these to "vim" if possible.

If you are uncomfortable with vim(1), you can keep using mcedit(1) for most system maintenance tasks.

Very smart viewer. This is a great tool for searching words in documents. I always use this for files in the "/usr/share/doc" directory. This is the fastest way to browse through masses of Linux information. This viewer can be directly started like so:

$ mc -v path/to/filename_to_view
$ mcview path/to/filename_to_view

Press Enter on a file, and the appropriate program will handle the content of the file (see Section 9.5.11, “Customizing program to be started”). This is a very convenient MC feature.

In order to allow these viewer and virtual file features to function, viewable files should not be set as executable. Change their status using chmod(1) or via the MC file menu.

MC can be used to access files over the Internet using FTP. Go to the menu by pressing F9, then type "p" to activate the FTP virtual filesystem. Enter a URL in the form "username:passwd@hostname.domainname", which will retrieve a remote directory that appears like a local one.

Try "[http.us.debian.org/debian]" as the URL and browse the Debian archive.

You can select your login shell with chsh(1).

In this tutorial chapter, the interactive shell always means bash.

You can customize bash(1) behavior by "~/.bashrc". For example, I added followings to "~/.bashrc":

# CD upon exiting MC
. /usr/share/mc/bin/mc.sh

# set CDPATH to good one
CDPATH=.:/usr/share/doc:~/Desktop/src:~/Desktop:~
export CDPATH

PATH="${PATH}":/usr/sbin:/sbin
# set PATH so it includes user's private bin if it exists
if [ -d ~/bin ] ; then
  PATH=~/bin:"${PATH}"
fi
export PATH

EDITOR=vim
export EDITOR

In the Unix-like environment, there are few key strokes which have special meanings. Please note that on a normal Linux character console, only the left-hand Ctrl and Alt keys work as expected. Here are few notable key strokes to remember.

Unix style mouse operations are based on the 3 button mouse system.

The center wheel on the modern wheel mouse is considered middle mouse button and can be used for middle-click. Clicking left and right mouse buttons together serves as the middle-click under the 2 button mouse system situation. In order to use a mouse in Linux character consoles, you need to have gpm(8) running as daemon.

less(1) is the enhanced pager (file content browser). Hit "h" for help. It can do much more than more(1) and can be supercharged by executing "eval $(lesspipe)" or "eval $(lessfile)" in the shell startup script. See more in "/usr/share/doc/lessf/LESSOPEN". The "-R" option allows raw character output and enables ANSI color escape sequences. See less(1).

You should become proficient in one of variants of Vim or Emacs programs which are popular in the Unix-like system.

I think getting used to Vim commands is the right thing to do, since Vi-editor is always there in the Linux/Unix world. (Actually, original vi or new nvi are programs you find everywhere. I chose Vim instead for newbie since it offers you help through F1 key while it is similar enough and more powerful.)

If you chose either Emacs or XEmacs instead as your choice of the editor, that is another good choice indeed, particularly for programming. Emacs has a plethora of other features as well, including functioning as a newsreader, directory editor, mail program, etc.. When used for programming or editing shell scripts, it intelligently recognizes the format of what you are working on, and tries to provide assistance. Some people maintain that the only program they need on Linux is Emacs. Ten minutes learning Emacs now can save hours later. Having the GNU Emacs manual for reference when learning Emacs is highly recommended.

All these programs usually come with tutoring program for you to learn them by practice. Start Vim by typing "vim" and press F1-key. You should at least read the first 35 lines. Then do the online training course by moving cursor to "|tutor|" and pressing Ctrl-].

Debian comes with a number of different editors. We recommend to install the vim package, as mentioned above.

Debian provides unified access to the system default editor via command "/usr/bin/editor" so other programs (e.g., reportbug(1)) can invoke it. You can change it by:

$ sudo update-alternatives --config editor

The choice "/usr/bin/vim.basic" over "/usr/bin/vim.tiny" is my recommendation for newbies since it supports syntax highlighting.

You can customize vim(1) behavior by "~/.vimrc". For example, I use:

" -------------------------------
" Local configuration
"
set nocompatible
set nopaste
set pastetoggle=<f2>
syn on
if $USER == "root"
 set nomodeline
 set noswapfile
else
 set modeline
 set swapfile
endif
" filler to avoid the line above being recognized as a modeline
" filler
" filler

The output of the shell command may roll off your screen and may be lost forever. It is good practice to log shell activities into the file for you to review them later. This kind of record is essential when you perform any system administration tasks.

The basic method of recording the shell activity is to run it under script(1).

$ script
Script started, file is typescript

$ vim typescript

See Section 9.2.3, “Recording the shell activities cleanly” .

Let's learn basic Unix commands. Here I use "Unix" in its generic sense. Any Unix clone OSs usually offer equivalent commands. The Debian system is no exception. Do not worry if some commands do not work as you wish now. If alias is used in the shell, its corresponding command outputs are different. These examples are not meant to be executed in this order.

Try all following commands from the non-privileged user account:

Please traverse directories and peek into the system using the above commands as training. If you have questions on any of console commands, please make sure to read the manual page. For example, these commands are the good start:

$ man man
$ man bash
$ man builtins
$ man grep
$ man ls

The style of man pages may be a little hard to get used to, because they are rather terse, particularly the older, very traditional ones. But once you get used to it, you come to appreciate their succinctness.

Please note that many Unix-like commands including ones from GNU and BSD will display brief help information if you invoke them in one of the following ways (or without any arguments in some cases):

$ <commandname> --help
$ <commandname> -h

Values of some environment variables change the behavior of some Unix commands.

Default values of environment variables are initially set by the PAM system and then some of them may be reset by some application programs:

Some commands take arguments. Arguments starting with "-" or "--" are called options and control the behavior of the command.

$ date
Mon Oct 27 23:02:09 CET 2003
$ date -R
Mon, 27 Oct 2003 23:02:40 +0100

Here the command-line argument "-R" changes date(1) behavior to output RFC2822 compliant date string.

Often you want a command to work with a group of files without typing all of them. The filename expansion pattern using the shell glob, (sometimes referred as wildcards), facilitate this need.

For example, try the following and think for yourself:

$ mkdir junk; cd junk; .[^.]*touch 1.txt 2.txt 3.c 4.h .5.txt ..6.txt
$ echo *.txt
1.txt 2.txt
$ echo *
1.txt 2.txt 3.c 4.h
$ echo *.[hc]
3.c 4.h
$ echo .*
. .. .5.txt ..6.txt
$ echo .*[^.]*
.5.txt ..6.txt
$ echo [^1-3]*
4.h
$ cd ..; rm -rf junk

See glob(7) for more.

Each command returns its exit status (variable: "$?") as the return value.

Thus:

$ [ 1 = 1 ] ; echo $?
0
$ [ 1 = 2 ] ; echo $?
1

Let's try to remember following shell command idioms.

The Debian system is a multi-tasking system. Background jobs allow users to run multiple programs in a single shell. The management of the background process involves the shell builtins: jobs, fg, bg, and kill. Please read sections of bash(1) under "SIGNALS", and "JOB CONTROL", and builtins(1).

Let's try simple examples of redirection:

$ </etc/motd pager

$ pager </etc/motd

$ pager /etc/motd

$ cat /etc/motd | pager

Although all 4 examples display the same thing, the last example runs an extra cat command and wastes resources with no reason.

The shell allows you to open files using the exec builtin with an arbitrary file descriptor.

$ echo Hello >foo
$ exec 3<foo 4>bar  # open files
$ cat <&3 >&4       # redirect stdin to 3, stdout to 4
$ exec 3<&- 4>&-    # close files
$ cat bar
Hello

Here, "n<&-" and "n>&-" mean to close the file descriptor "n".

The file descriptor 0-2 are predefined:

You can set an alias for the frequently used command. For example:

$ alias la='ls -la'

Now, "la" works as a short hand for "ls -la" which lists all files in the long listing format.

You can list any existing aliases:

$ alias

You can identity exact path or identity of the command using type builtins command. For example:

$ type ls
ls is hashed (/bin/ls)
$ type la
la is aliased to ls -la
$ type echo
echo is a shell builtin
$ type file
file is /usr/bin/file

Here ls was recently searched while "file" was not, thus "ls" is "hashed", i.e., the shell has an internal record for the quick access to the location of the "ls" command.

There are few standard text processing tools which are used very often on the Unix-like system.

If you are not sure what exactly these commands do, please use "man command" to figure it out by yourself.

Regular expressions are used in many text processing tools. They are analogous to the shell globs, but they are more complicated and powerful.

The regular expression describes the matching pattern and is made up of text characters and metacharacters.

The metacharacter is just a character with a special meaning. There are 2 major styles, BRE and ERE, depending on the text tools as described above.

The regular expression of emacs is basically BRE but has been extended to treat "+"and "?" as the metacharacters as in ERE. Thus, there are no needs to escape them with "\" in the regular expression of emacs.

For example, grep(1) can be used to perform the text search using the regular expression:

$ egrep 'GNU.*LICENSE|Yoyodyne' /usr/share/common-licenses/GPL
GNU GENERAL PUBLIC LICENSE
GNU GENERAL PUBLIC LICENSE
Yoyodyne, Inc., hereby disclaims all copyright interest in the program

For the replacement expression, following characters have special meanings:

For Perl replacement string, "$n" is used instead of "\n" and "&" has no special meaning.

For example:

$ echo zzz1abc2efg3hij4 | \
sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/=&=/'
zzz=1abc2efg3hij4=
$ echo zzz1abc2efg3hij4 | \
sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/\2===\1/'
zzzefg3hij4===1abc
$ echo zzz1abc2efg3hij4 | \
perl -pe 's/(1[a-z]*)[0-9]*(.*)$/$2===$1/'
zzzefg3hij4===1abc
$ echo zzz1abc2efg3hij4 | \
perl -pe 's/(1[a-z]*)[0-9]*(.*)$/=&=/'
zzz=&=

Here please pay extra attention to the style of the bracketed regular expression and how the matched strings are used in the text replacement process on different tools.

These regular expressions can be used for cursor movements and text replacement actions in some editors too.

The back slash "\" at the end of line in the shell commandline escapes newline as a white space character and continues shell command line input to the next line.

Please read all the related manual pages to learn these commands.

The ed(1) command can replace all instances of "FROM_REGEX" with "TO_TEXT" in "file" by:

$ ed file <<EOF
,s/FROM_REGEX/TO_TEXT/g
w
q
EOF

The vim(1) command can replace all instances of "FROM_REGEX" with "TO_TEXT" in "file" by using ex(1) commands:

$ vim '+%s/FROM_REGEX/TO_TEXT/gc' '+w' '+q' file

Multiple files ("file1", "file2", and "file3") can be processed with regular expressions similarly with vim(1) or perl(1):

$ vim '+argdo %s/FROM_REGEX/TO_TEXT/ge|update' '+q' file1 file2 file3

$ perl -i -p -e 's/FROM_REGEX/TO_TEXT/g;' file1 file2 file3

In the perl(1) example, "-i" is for in-place editing, "-p" is for implicit loop over files.

Let's consider a text file called "DPL" in which some pre-2004 Debian project leader's names and their initiation days are listed in a space-separated format.

Ian     Murdock   August  1993
Bruce   Perens    April   1996
Ian     Jackson   January 1998
Wichert Akkerman  January 1999
Ben     Collins   April   2001
Bdale   Garbee    April   2002
Martin  Michlmayr March   2003

Awk is frequently used to extract data from these types of files:

$ awk '{ print $3 }' <DPL                   # month started
August
April
January
January
April
April
March
$ awk '($1=="Ian") { print }' <DPL          # DPL called Ian
Ian     Murdock   August  1993
Ian     Jackson   January 1998
$ awk '($2=="Perens") { print $3,$4 }' <DPL # When Perens started
April 1996

Shells such as Bash can be also used to parse this kind of file:

$ while read first last month year; do
    echo $month
  done <DPL

Here, the read builtin command uses characters in "$IFS" (internal field separators) to split lines into words.

If you change "$IFS" to ":", you can parse "/etc/passwd" with shell nicely:

$ oldIFS="$IFS"   # save old value
$ IFS=':'
$ while read user password uid gid rest_of_line; do
    if [ "$user" = "bozo" ]; then
      echo "$user's ID is $uid"
    fi
  done < /etc/passwd
bozo's ID is 1000
$ IFS="$oldIFS"   # restore old value

(If Awk is used to do the equivalent, use "FS=':'" to set the field separator.)

IFS is also used by the shell to split results of parameter expansion, command substitution, and arithmetic expansion. These do not occur within double or single quoted words. The default value of IFS is <space>, <tab>, and <newline> combined.

Be careful about using this shell IFS tricks. Strange things may happen, when shell interprets some parts of the script as its input.

$ IFS=":,"                        # use ":" and "," as IFS
$ echo IFS=$IFS,   IFS="$IFS"     # echo is a Bash builtin
IFS=  , IFS=:,
$ date -R                         # just a command output
Sat, 23 Aug 2003 08:30:15 +0200
$ echo $(date -R)                 # sub shell --> input to main shell
Sat  23 Aug 2003 08 30 36 +0200
$ unset IFS                       # reset IFS to the default
$ echo $(date -R)
Sat, 23 Aug 2003 08:30:50 +0200

The following scripts will do nice things as a part of a pipe.

One-line shell script can loop over many files using find(1) and xargs(1) to perform quite complicated tasks. See Section 10.1.5, “Idioms for the selection of files” and Section 9.5.9, “Repeating a command looping over files”.

When using the shell interactive mode becomes too complicated, please consider to write a shell script (see Section 12.1, “The shell script”).

Here are some key points for package configuration on the Debian system:

The newbie Debian system administrator should stay with the stable release of Debian while applying only security updates. I mean that some of the following valid actions are better avoided, as a precaution, until you understand the Debian system very well:

The non-compatible effects caused by above actions to the Debian package management system may leave your system unusable.

The serious Debian system administrator who runs mission critical servers, should use extra precautions:

Despite my warnings above, I know many readers of this document wish to run the testing or unstable suites of Debian as their main system for self-administered Desktop environments. This is because they work very well, are updated frequently, and offer the latest features.

It takes no more than simply setting the distribution string in the "/etc/apt/sources.list" to the suite name: "testing" or "unstable"; or the codename: "squeeze" or "sid". This will let you live the life of eternal upgrades.

The use of testing or unstable is a lot of fun but comes with some risks. Even though the unstable suite of Debian system looks very stable for most of the times, there have been some package problems on the testing and unstable suite of Debian system and a few of them were not so trivial to resolve. It may be quite painful for you. Sometimes, you may have a broken package or missing functionality for a few weeks.

Here are some ideas to ensure quick and easy recovery from bugs in Debian packages:

(If you can not do any one of these precautionary actions, you are probably not ready for the testing and unstable suites.)

Enlightenment with the following will save a person from the eternal karmic struggle of upgrade hell and let him reach Debian nirvana.

Let's look into the Debian archive from a system user's perspective.

For the typical HTTP access, the archive is specified in the "/etc/apt/sources.list" file as, e.g. for the current stable = lenny system:

deb http://ftp.XX.debian.org/debian/ lenny main contrib non-free
deb-src http://ftp.XX.debian.org/debian/ lenny main contrib non-free

deb http://security.debian.org/ lenny/updates main contrib
deb-src http://security.debian.org/ lenny/updates main contrib

Please note "ftp.XX.debian.org" must be replaced with appropriate mirror site URL for your location, for USA "ftp.us.debian.org", which can be found in the list of Debian worldwide mirror sites. The status of these servers can be checked at Debian Mirror Checker site.

Here, I tend to use codename "lenny" instead of suite name "stable" to avoid surprises when the next stable is released.

The meaning of "/etc/apt/sources.list" is described in sources.list(5) and key points are:

The "deb-src" lines can safely be omitted (or commented out by placing "#" at the start of the line) if it is just for aptitude which does not access source related meta data. It will speed up the updates of the archive meta data. The URL can be "http://", "ftp://", "file://", ….

Here is the list of URL of the Debian archive sites and suite name or codename used in the configuration file:

Each Debian archive consists of 3 components. Components are alternatively called categories in "Debian Policy" or areas in "Debian Social Contract". The component is grouped by the compliance to "The Debian Free Software Guidelines" (DFSG):

Here the number of packages in the above is for the amd64 architecture. Strictly speaking, only the main component archive shall be considered as the Debian system.

The Debian archive organization can be studied best by pointing your browser to the each archive URL appended with dists or pool.

The distribution is referred by two ways, the suite or codename. The word distribution is alternatively used as the synonym to the suite in many documentations. The relationship between the suite and the codename can be summarized as:

The history of codenames are described in Debian FAQ: 6.3.1 Which other codenames have been used in the past?

In the stricter Debian archive terminology, the word "section" is specifically used for the categorization of packages by the application area. (Although, the word "main section" may sometimes be used to describe the Debian archive section which provides the main component.)

Every time a new upload is done by the Debian developer (DD) to the unstable archive (via incoming processing), DD is required to ensure uploaded packages to be compatible with the latest set of packages in the latest unstable archive.

If DD breaks this compatibility intentionally for important library upgrade etc, there is usually announcement to the debian-devel mailing list etc.

Before a set of packages are moved by the Debian archive maintenance script from the unstable archive to the testing archive, the archive maintenance script not only checks the maturity (about 10 days old) and the status of the RC bug reports for the packages but also tries to ensure them to be compatible with the latest set of packages in the testing archive. This process makes the testing archive very current and usable.

Through the gradual archive freeze process led by the release team, the testing archive will be matured to make it completely consistent and bug free with some manual interventions. Then the new stable release is created by assigning the codename for the old testing archive to the new stable archive and creating the new codename for the new testing archive. The initial contents of the new testing archive is exactly the same as that of the newly released stable archive.

Both the unstable and the testing archives may suffer temporary glitches due to:

So if you ever decide to use these archives, you should be able to fix or work around these kinds of glitches.

See Debian Policy Manual for definition of:

The Debian system offers a consistent set of binary packages through its versioned binary dependency declaration mechanism in the control file fields. Here is a bit over simplified definition for them.

The official definition including source dependency can be found in the Policy Manual: Chapter 7 - Declaring relationships between packages.

Here is a summary of the simplified event flow of the package management by APT.

Here, I intentionally skipped technical details for the sake of big picture.

You should read the fine official documentation. The first document to read is the Debian specific "/usr/share/doc/<package_name>/README.Debian". Other documentation in "/usr/share/doc/<package_name>/" should be consulted too. If you set shell as Section 1.4.2, “Customizing bash”, type:

$ cd <package_name>
$ pager README.Debian
$ mc

You may need to install the corresponding documentation package named with "-doc" suffix for detailed information.

If you are experiencing problems with a specific package, make sure to check out these sites first:

Search Google with search words including "site:debian.org", "site:wiki.debian.org", "site:lists.debian.org", etc..

When you file a bug report, please use reportbug(1) command.

Here are package management operations with commandline using aptitude(8) and apt-get(8) /apt-cache(8).

Although it is now safe to mix different package tools on the Debian system, it is best to continue using aptitude as much as possible.

The difference between "safe-upgrade"/"upgrade" and "full-upgrade"/"dist-upgrade" only appears when new versions of packages stand in different dependency relationships from old versions of those packages. The "aptitude safe-upgrade" command will never install new packages nor remove installed packages.

The "aptitude why <regex>" can list more information by "aptitude -v why <regex>". Similar information can be obtained by "apt-cache rdepends <package>".

When aptitude command is started in the commandline mode and faces some issues such as package conflicts, you can switch to the full screen interactive mode by pressing "e"-key later at the prompt.

You may provide command options right after "aptitude".

See aptitude(8) and "aptitude user's manual" at "/usr/share/doc/aptitude/README" for more.

For the interactive package management, you start aptitude in interactive mode from the console shell prompt as:

$ sudo aptitude -u
Password:

This will update the local copy of the archive information and display the package list in the full screen with menu. Aptitude places its configuration at "~/.aptitude/config".

Notable key strokes to browse status of packages and to set "planned action" on them in this full screen mode are:

The file name specification of the command line and the menu prompt after pressing "l" and "//" take the aptitude regex as described below. Aptitude regex can explicitly match a package name using a string started by "~n and followed by the package name.

In the interactive full screen mode of aptitude(8), packages in the package list are displayed like this by default:

idA   libsmbclient                             -2220kB 3.0.25a-1  3.0.25a-2

Here, this line means from the left as:

The candidate version is chosen according to the current local preferences (see apt_preferences(5) and Section 2.7.3, “Tweaking candidate version”).

Several types of package views are available under the menu "Views":

The standard "Package View" categorizes packages somewhat like dselect with few extra features.

Aptitude offers several options for you to search packages using its regex formula:

Here, the string for <package_name> is treated as the exact string match to the package name unless it is started explicitly with "~" to be the regex formula.

The aptitude regex formula is mutt-like extended ERE (see Section 1.6.2, “Regular expressions”) and the meanings of the aptitude specific special match rule extensions are as below:

Here,

Short cuts:

Users familiar with mutt will pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in the "User's Manual" "/usr/share/doc/aptitude/README".

The selection of a package in aptitude not only pulls in packages which are defined in its "Depends:" list but also defined in the "Recommends:" list if the menu "F10 → Options → Dependency handling" is set accordingly. These auto installed packages are removed automatically if they are no longer needed under aptitude.

You can check package activity history in the log files.

In reality, it is not so easy to get meaningful understanding quickly out from these logs. See Section 9.2.9, “Recording changes in configuration files” for easier way.