`Topal': GPG/Pine integration
Copyright (C) 2001--2003 Phillip J. Brooke
Contents
Topal is a `glue' program that links
GnuPG
and
Pine. It offers
facilities to encrypt, decrypt, sign and verify emails. See the list
of features below.
- In-place decryption/verification, dealing with multiple blocks
embedded in text.
- Caching of output to reduce need for passphrase (at expense of
storing decrypts and verification output).
- Receiving of MIME RFC2015 multipart/signed and
multipart/encrypted messages. (Top-level multipart items need some help from a script,
topal-fix-email, invoked by procmail.) These features are available
to any program that uses .mailcap files.
- Sending of MIME RFC2015 multipart/signed and multipart/encrypted
messages. (Needs a patch to Pine.)
- Sending and receiving of the old application/pgp content-type
(sending requires the same patch as the previous item).
- Offers user the opportunity to check output before sending it.
- Rich configuration options.
- Shortcuts for selecting keys, as well as general key selection
routines when sending email.
- Few arbitrary limits.
The previous stable releases were 0.7.2, 0.7.8 and 0.7.9.
Important Changes in Version 0.7.10
The recommended procmail recipe has been changed.
Important Changes in Version 0.7.8
topal-fix-email and topal-fix-folder have been replaced by the
main topal binary. Change topal-fix-email in your .procmailrc to be
topal --fix-email. (Or add symlinks: the binary checks what it has been called as.)
You must clear your cache otherwise the changes made for
inline-separate-output (added in version 0.7.8) will break (this occurs regardless of whether
the option is on or off). This new feature shows the GnuPG/Topal
output separately, then hands back the decrypted or verified output
without any wrappers. This makes it more suitable for dealing with
attachments (but you need to set it manually via topal
-config).
Finally, the send menu has a new option: `Pass through unchanged'.
This does nothing to the message (except for forcing the content-type
to be text/plain when invoked with -sendmime; this is useful for
dealing with an obscure bug in Pine 4.44). Moreover, you can
always have Topal invoked for sending.
You need a working C compiler and the GNU Ada Compiler (GNAT). There
is a makefile: simply type `make'. Type `make install' to see what
files need copying (it doesn't actually do the copy).
Assuming that the topal binary is installed in /usr/local/bin, set up
the Pine sending & display filters as follows:
display-filters=_BEGINNING("-----BEGIN PGP ")_ /usr/local/bin/topal -display _TMPFILE_ _RESULTFILE_
sending-filters=/usr/local/bin/topal -send _TMPFILE_ _RESULTFILE_ _RECIPIENTS_,
/usr/local/bin/topal -sendmime _TMPFILE_ _RESULTFILE_ _MIMETYPE_ _RECIPIENTS_
You can choose either or both of the sending filters. The -sendmime
option allows the user to choose the MIME type of the outbound
email. (Legacy fixes are in place that make -decrypt and -verify
behave the same as -display.)
For -sendmime to work, you will need to patch Pine. There are patches
for versions
4.44,
4.50
and
4.53
of Pine. (They're all more-or-less the same patch. It's pretty easy
to apply them against Pine versions 4.51 and 4.52 if you feel so
inclined.)
`cd' into the pine4.xx directory and use the command
`patch -p1 < wherever/the/patch/is'.
It doesn't seem to have broken anything else.... It seems to work for
sending via an SMTP server - it might break for sending via
/usr/lib/sendmail (if it does, please send me a debug trace by
invoking pine with `-d 9').
To decode MIME RFC2015 multipart/signed and /encrypted messages
requires the assistance of metamail. Add in either the user mailcap
configuration (.mailcap) or the system configuration
(/etc/mailcap) the lines
multipart/signed; /usr/local/bin/topal -mime '%s' '%t'; needsterminal
multipart/encrypted; /usr/local/bin/topal -mime '%s' '%t'; needsterminal
application/pgp; /usr/local/bin/topal -mimeapgp '%s' '%t'; needsterminal
In your procmailrc, add the recipe:
:0fw
| /usr/local/bin/topal --fix-email
I strongly advise that you also use one of the backup
recipes from the procmail manual. See also the notes in
additional programs.
Create a directory called `${HOME}/.topal'. This is currently
hard-coded into Topal. Create the basic configuration file by running
topal with the -dump or -default option. This file should be named `config'.
All .topal files are silently ignored if they cannot be found.
Parsing errors cause an exception.
If you want to include strings with spaces, you'll need to quote them
with double-quotes ("). Double-quotes themselves can be
included by `stuffing' ("").
-help as the first argument dumps a help message.
The help message is derived from the help.txt file (included at
compile time).
See help.txt for information on non-Pine use of Topal.
Send email to me if you're really stuck.
-config as the first argument brings up the configuration menu.
This menu is also available when sending (so that the signing key can
be changed).
Depending on configuration, Topal will either ignore the file
altogether, ask you what you want to do with it, or proceed to
process the file automatically.
GPG will ask you for your passphrase when it needs it.
Caching is in place; the results of decryption and verification are
(subject to configuration) saved in ~/.topal/cache. The results of
caching mean that you won't be repeatedly asked for your passphrase,
at the expense of storing decrypts in the clear.
Be warned: Topal often invokes `less' to view something. So you'll
need to use `q' to get out of it. `Metamail' is called for anything
after MIME processing.
A new option (for version 0.7.8) called `inline-separate-output'
concerns inlined (i.e., not MIME) messages. If the option is on, then
the Topal/GnuPG output will be shown to you by less. Then the
decrypted or verified output will be handed back to Pine. This is the
way to approach attachments. However, you will normally want to keep
this option off, because if you're reading (for example) BugTraq
mailings, then it will want you to hit `q' an awful lot....
If you choose the `Topal' filter when sending, you will be offered a
menu
(lkr) List/edit current recipient keys (@) Add own key
(n) Pass through unchanged (o) Configuration (q) Abort
Execute GPG: (e) Encrypt (s) Sign-encrypt (c) Clearsign
Above that, it will indicate picking up keys for each recipient email
address. It will also add a key `for self'.
`Abort' tells Pine you don't want Topal to process the email anymore.
`Pass through unchanged' does nothing to the message (except for forcing the
content-type to be text/plain when invoked with -sendmime; this is useful for dealing with an
obscure bug in Pine 4.44). This means that you can always have Topal
invoked for sending.
`Add own key' adds an `encrypt to self' key. (It is added by default,
but if you remove it, this is a quick way to restore it.)
The three `execute GPG' options use the current settings and execute
GPG. You will be asked to confirm the command-line, and after
processing, less is invoked to visually check that the desired result
has been achieved. Again, a confirmation is asked for.
If -sendmime was used, then a menu will ask
(p) Inline plain text (a) application/pgp (m) multipart/*
(q) Abort
offer a choice of three MIME types. Don't use (a) unless you really
know what you're doing. If you are signing and encrypting, a further
choice, (e) multipart/* encapsulated
will be offered.
This encapsulates a MIME signed message inside an encrypted message.
Otherwise, we do both operations at once. (If you choose `clearsign'
and `multipart/*', then all trailing blank lines will be deleted.
Note also that Pine appears to delete trailing whitespace in trailing
blank lines.)
`Configuration' offers the same menu that is available from the
-config option.
`List current recipient keys' offers a list of recipients:
Select key, or (dq) to quit and return to main send menu
or (s) to select a key after searching in the main keyring
or (ak) to add keys from the main keyring (not recommended, use `s')
Displaying choices 1 to 2 of 1 to 2 (<,) page up (>.) page down
1 - Details: pub 1024D/50973B91 2000-12-19 Dr Phil Brooke (at home) <P.J.Broo
2 - Details: pub 1024D/16BE903A 2001-02-25 Malcolm Gray <malcolm.gray@jobstre
`Quit and return to main send menu' sends you back to the first menu.
`Add key from main keyring' prompts you for a search pattern. It will
do a general search on your GPG keyring. Beware of just pressing
enter - it will select all keys on your keyring.
A better alternative is to use the `select after search' option. This
also does a search on your GPG keyring, but then you must select
one key to be added to your list of recipients.
Selecting a key will offer a third menu (a similar menu is offered
when selecting a single key):
Key: Details: pub 1024D/50973B91 2000-12-19 Dr Phil Brooke (at home) <P.J.Broo
(d) Display details of key with less, (v) Verbosely
(r) Remove key from list (kql) Return to key list
`Return to key list' takes you back to the second menu.
`Display details of key (less)' simply uses GPG to list the
key details via less. You'll need to use `q' to get out of less.
`Verbose details of key (less)' pipes verbose output from GPG for this
key into gpg. You'll need to use `q' to get out of less.
`Remove key from list' removes the key from this recipient list.
If you invoke Topal on the command-line with a filename as an
argument, it will offer the sending functions on that file. It
doesn't actually send anything: instead it allows you to encrypt,
sign, etc. the message. You have a choice of overwriting or
preserving the original file (this bit is case-sensitive).
The main purpose of this mode is for encrypting or signing attachments
before they are attached to the message in Pine. Beware that Pine
does not feed the attachments to a sending filter.
MIME functions are not available in this mode: it makes no sense.
Two scripts used to be included with topal: topal-fix-email and
topal-fix-folder. They have been replaced by the --fix-email
and --fix-folder command-line options to the main binary.
topal --fix-email modifies any email that is (at the top level) a
multipart/signed or multipart/encrypted message. It creates a
multipart/alternative message. The revised message contains the first
part of the original message as one alternative, then the entire
original message as an alternative part. Pine can cope with invoking
Topal on the (original) subpart. Confused? (It probably isn't clever
for /encrypted mail, but at least Topal can get at it.)
Usage:
- topal --fix-folder <folder> ...
- This fixes the old
email folders you may have.
- topal --fix-email
- Takes no arguments; it accepts a single
email on stdin. Ideally, it should be invoked by procmail (see the
configuration section above).
topal --fix-email has a simpler mode (--simple) where it
pretends that there are two MIME content types:
`application/x-topal-encrypted' and `application/x-topal-signed'. You
might prefer using this.
What does the patch to Pine do? It removes some of the safety
checking when changing the content-type (_MIMETYPE_) in a filter.
Normally, if the returned content-type is not text/*, then the entire
content-type is dropped.
The patch instead adds a flag, `topal_hack', and sets this if the
returned content-type is not text. From time-to-time, we
pretend that the body is normal text. We take a little care to check
if this message is already a multipart message, so hopefully, the normal
sending of attachments still works.
Topal internally lists keys by their fingerprint. It uses GPG to look
up key fingerprints by using whatever GPG can cope with.
Duplicate keys are silently suppressed. Removing a key only removes
one instance, if somehow you've coerced Topal to list duplicates
(which is quite easy, since adding a key with its short key ID, and
the same key with its fingerprint will add two identical keys).
The way that Topal chooses the keys is as follows:
- For each recipient email address (supplied by Pine)
- For each matching line in keylist, use the key ID to get a fingerprint, and add the key to the list.
- In there are no matching lines in keylist, try to get a
fingerprint via just that email address (but exclude `xk'
configuration entries).
The keylist is a way to say, `for this particular email address, use
this particular key'.
Bad things happening should result in Topal setting its exit status to
`failed', so Pine should detect this and not send your email.
Bug reports are welcome: send them by email to me (contact details below).
If an attachment is a plaintext PGP ASCII-armoured message, then Topal
will be invoked by Pine. You probably want to say `no' when asked
here (beware of the configuration options here). Otherwise, you'll
get a decrypted file with the original attachment filename, plus the
various Topal headers.
You might want to run something like
find ${HOME}/.topal/cache -mtime +7 | xargs rm
to remove all the cache files that are a bit old (in this example, 7
days old or older).
To be notified of new releases of Topal, send an email to me....
Phil Brooke wrote this, partially out of boredom, but mostly because
he wanted a GPG/Pine add-on to do exactly what he wants. There are
many similar programs.
If you like this program, please tell me. If you'd like it better
with changes, please tell me what changes you want. If particular
items on the `To do' list are important to you, let me know. In
particular, if you find bugs, feel free to tell me the details by
email.
This package is released under the GPL: see the file COPYING.
I can be emailed on
pjb@lothlann.freeserve.co.uk
My key ID is 0x50973B91; the key is available from web pages and public key
servers.
If you want to send snailmail to me, email me for my (physical) address.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- Fix these known/reported bugs:
- `Pine indicating both text elements as shown' problem -
no obvious reason for it, though.
- Very rarely, some Topal-munged multipart/signed messages aren't
handed off to Pine. Not obvious why, yet.
- Add signal handlers.
- Catch GPG keyboard interrupt.
- Should we check that the infile matches the cache file even if
the MD5 hash matches? (We'd need to store the infile in the cache as well.)
- Check through code: all external calls should check return
values. General clean-up.
- Add pkcs7 signatures via SSL?
- Use mimeconstruct to attach attachments that are then signed/encrypted/whatever?
- Add interrupt option at very beginning of execution? (which
would bring up the configuration menu?)
- Associate extra options with particular keys?
- Configuration routine for managing keys/config/keylist?
- Implement rest of configuration menu.
- Make a much nicer interface all round....
- Separate out all the constant strings -- so that we can have internationalization.
- Context-sensitive help throughout (modify mkhelp to create multiple
procedures, or do it by number?); add COPYING option?
- More receiving/decrypt options: include both plaintext and
ciphertext.
- Add periodic cache cleanup when Topal is invoked?
- Add logging for workaround mode (report time of email processing (include PID); indicate if the file was changed or not)?
Look in release for the current release number.
- 06/2001, 0.1
- First alpha release.
- 06/2001, 0.2
- Minor changes.
- 06/2001, 0.3
- Major changes to how keys are identified and
looked up.
- 06/2001, 0.4
- Adding more customization features.
- 11/2001, 0.4.4
- Cleaned up some error messages; added -nps
mode.
- 11/2001, 0.4.5
- Added `gpg-options' config item with
default `--no-options'. (Forgot to add this note as well....)
- 11/2001, 0.5.0
- Dumped -verify and -decrypt modes in
favour of the multiple-block `-display' mode. Added -help. Added
caching. Added more switches relating to caching. Better output
formatting.
- 11/2001, 0.5.1
- Improved menus. Tidied up some of the
interface. Added -s, which does the same as -nps.
- 12/2001, 0.5.2
- Tidied disclaimer. Added synonyms for
-help (-h, -?, --help, --h) Cleaned up menus; keypresses aren't echoed
any longer.
- 12/2001, 0.5.3
- Altered packaging to include version in
directory name. Changed names of some -clear options to be a bit more
sensible. Changing config settings method (big change). Making -s
the default operation. Some rearrangement of code, constants. Some
configuration editing possible via Topal. Send has access to
configuration menu.
- 12/2001, 0.5.4
- Bug fix; one-off error in the sending
menus.
- 12/2001, 0.5.5
- Removed redundant examples directory.
Changed over to HTML documentation. Tweaked the RELEASE stuff. Use
space instead of enter when waiting to continue: this looks forward to
offering a help option at every prompt. The receive/blocks
stuff now uses an expanding array. The GPG return value is checked
when receiving: if it's bad, then some bits of the output are omitted;
the cache file is not written. The date bit of Topal output moved
onto the previous line (echo -n blah blah).
- 12/2001, 0.5.6
- Adding installation instructions. Using
tee and PIPESTATUS to get stderr on screen during receiving while also
saving that output and recording gpg's exit status. Changed RELEASE
filename to release. Tidied up the Makefile. Invalid passphrase
messages are grep'd out of the output. Added `fast continue' options.
Key lists in the configuration section now use expanding arrays.
Changed key details selection message. Secret key selection now
offers a menu of secret keys on the secret keyring. Initial recipient
search excludes keys in XK list. Added key search/selection menu
choice - much nicer to use than the add menu. More configuration
stuff added (still more to do, although the config file can always be
used). Partial documentation update.
- 2/2002, 0.5.7
- Adding limited RFC2015/MIME decoding of
multipart email.
- 2/2002, 0.5.8
- Adding mime-construct to configuration in
expectation of more RFC2015 features. Put test for the config file
existing before actually attempting to read it (oops). Added -O2
-Wall and the TOPALDEBUG variable for compiling. Put up WWW page via
own Freeserve site. Announcing via Freshmeat. Automating output WWW
site generation (all the grunge in the Makefile).
- 3/2002, 0.6.0
- Distribution uses a gzip'd binary now....
Added a pre-built binary that is statically linked against the GNAT
stuff so that people don't need to acquire GNAT first (this, I
believe, complies with the GNAT licence).
Added the scripts
topal-fix-email and topal-fix-folder. This makes it a lot easier to
work with other people's multipart/signed or /encrypted email.
Procmail recipe added to this README.
Added display of
application/pgp messages. Including the text of one of these in a
reply might be difficult, but then, it was difficult without topal's
mangling. At least they can be verified and read now.
-sendmime
option added. Hack needed (in topal-pine-patch [now pine-4.44-patch])
to allow non-text/blah content-types in Pine. RFC2015 send and
received done (including micalg detection when sending clearsigned
messages: list used from RFC3156.). Ditto for application/pgp, but
I'm not sure of some of the parameters, since I've only ever seen
signed emails of this form.
Removed some of the waits for execution,
since it seems reliable. Added error checking on return value of GPG
in sends.
- 3/2002, 0.6.1
- The Content-Type for MIME sending is
displayed on the screen using `cat' rather than `less', which was
getting to be annoying.
Two changes that are related to how I
manage the source code: Slight tweak to makefile for keeping track
of RCS files; and using rcs -n<symbolic-name> to tag the
released files.
- 3/2002, 0.6.2
- MIME clear-signed messages: trailing blank
lines are now deleted before signing (this would cause BAD signature
when verifying on some other MTAs). Added remarks to documentation
about the patch to Pine and attachments.
- 4/2002, 0.6.3
- RFC1847 multipart encapsulation added.
(See section 6.1 of RFC3156.) Cleaned up related receiving/caching
behaviour.
Another MIME clear-signed messages bugfix. This one
sorts out line-end conventions correctly.
New patch for Pine: this
stops a SEGFAULT when using RFC2015 stuff and other attachments at the
same time.
Updated documentation; added man pages for the two scripts.
- 4/2002, 0.6.4
- New patch for Pine. Adds a workaround for
the problem where some versions of MS Exchange would silently lose
inbound MIME clearsigned email. It turns out that a slight formatting
change stops the problem.
- 5/2002, 6/2002; 0.6.5, 0.6.6, 0.6.7, 0.6.8
- Adding more debugging,
mostly to the menus code. Used for tracking down a nasty problem
causing exceptions. Many thanks to Felix Madlener for pointing this
out and testing the revised code.
- 7/2002, 0.6.9
- Renamed the Pine patch for when new versions
come out. (It's still the same patch as for Topal 0.6.4.) Added trap
for non-existent file when using `-s'. Cache directory as well as
.topal directory is also chmod'd to 700. Added README.txt to package
file (even though it's generated from the .html) so that those who
just want to `less' it (instead of firing up a HTML reader) can do so.
- 8/2002, 0.7.0
- Changed email address in man page. Lots more
exception handling for extra info when something goes wrong. Moderate
code reorganisation: mostly splitting blocks of code out for future
work. Fixed `bug' (feature?) where send fails if a public key is
unusable (although this may risk sending plaintext through; we assume
that if an output file was generated, then the GPG errors weren't
fatal). Now we check instead if the output file exists. Checking all
source files for any similar bugs in menus (cf. the 5/2002 entry).
Modified MIME RFC2015 receiving function so that it isn't so reliant
on shell calls of sed (which can fall over with nasty characters in an
incoming emails boundary). Moreover, it can now cope with MIME parts
that don't end with a newline. Tweaking MIME/verify cache handling:
we shouldn't actually get an output file from GPG (since we're only
verifying one part with the other); we put a vague warning if this
happens, and trap when reading the cache. Added content-type to
plaintext for MIME/encrypted. Documentation update.
- 8/2002, 0.7.1
- Fixed minor bug with inverted return code
(`-s' trap). Doc update.
- 9/2002, 0.7.2
- Fixed minor bug in key list handling code
(dealing with key selection).
- 9/2002; 0.7.3, 0.7.4 (BETA)
- Disposed of the dependency on a shell by
introducing Ada bindings for fork/exec/dup/pipe/glob, etc.. Several
external binaries are no longer needed (cat, echo). Most return codes
are now properly checked (although still need to do a better audit).
Followed Eduardo Chappa's advice and changed Pine patch version
letter. Miscellaneous cleanups and fixes. Many thanks to Peter
Losher for giving me the incentive to sort out the external calls.
- 9/2002; 0.7.5 (BETA)
- Tidying up structure of external calls, and
how the various messages are built up and torn down. Changed the lynx
switches at the suggestion of Felix Madlener (many thanks!). When
receiving MIME encrypted attachments, the output is not included in
the Topal output, but only in the metamail invocation.
- 10/2002; 0.7.6 (BETA)
- Explicitly noted which versions are
not intended for general use (beta versions). Rearranged command line
parsing for more flexibility in future.
- 10/2002; 0.7.7 (BETA)
- Re-implementing topal-fix-email and
topal-fix-folder as part of the main topal binary. This removes the
(script) dependency on munpack, but adds formail and diff to the main
binary. Fixed some missing bits for particular binaries in
configuration handling. Adding `important changes from last stable
version' documentation. Tweaked the body extraction procedure.
Tweaked some output messages. Major changes to menus: they now use
enumerated types rather than integers.... Tweaking cl_menu some
more. Added `pass-thu' option to send menu (so you can always use the
Topal filter. This might also fix the minor problem with text/html
occasionally being sent when it shouldn't be....) Fixed bug where
MIME decrypt failure would still cause metamail to be invoked, but
that's a waste of time.
- 10/2002; 0.7.8
- Clearing out case statements with `when
others'. Tidying up sending.adb. Fixed problem in MIME output where
a leading blank line was added. Finally implemented `topal
--fix-folders' functionality added. No longer need the two old
scripts (I hope)! Another documentation tidy-up. Added
`inline-separate-output' option: this effectively turns off the GnuPG/Topal
wrappers in output. However, the side-effect is that the cache must
be cleared when upgrading to this version.
- 11/2002; 0.7.9
- Added some infrastructure for
encrypting/signing attachments (but this is nowhere near working yet).
Documentation and manpage update (again). Seems stable, will release.
- 2/2003; 0.7.10, 0.7.11
- Tweaking distribution pages (mkdistrib).
Including patches against Pine versions 4.50 and 4.53. (They're all
more-or-less the same patch. It's pretty
easy to apply them against 4.51 and 4.52 if you feel so inclined.)
Further doc clean up (particular the stuff about important changes
from previous stable versions). Implemented Felix M.'s suggestion for
handling non-existant command-line options: things that aren't valid
options, but are prefixed with a `-' get a more helpful error
message. --fix-email workaround also writes out the original input in
the exception handler. Changed recommended procmail recipe so that
Topal's exit code is checked.
- 2/2003; 0.7.12
- Adding `workaround-error-log' file to
.topal. This accepts output from topal --fix-email when it fails to
exit cleanly. Not quite clear if this bit works yet (was tracking
down other problem). It appears that when running without a real
terminal, the call to set_echo fails. Odd. Nasty workaround
implemented.
- 2/2002; 0.7.13
- Added missing includes to ada-echo-c.c.
Perhaps related to issue in the previous entry.
- 4/2003; 0.7.13b
- Bug fix release only - backported from
(not-yet-released 0.8.0). Fixed bug when
changing own signing key using the -config option - thanks to Stewart
James for the bug report.
- 10/2003; 0.7.13.2
- Bug fix release only - backported from
(not-yet-released 0.8.0). Changed bug fix versioning scheme.
Makefile now links properly against static GNAT runtime. Fixed
problem which manifests as: `relocation error: /lib/libreadline.so.4:
undefined symbol: BC' (needed instruction to link against ncurses) -
thanks to Marty Hoff for the bug report. Added patch against Pine
version 4.58.
- 10/2003; 0.7.13.3
- Now use -gnatwa and -gnato for all Ada
compilation. It was omitted from the main binary build command
before. Fixed all the resulting warnings.