-.\" $Id: t-prot.1,v 1.144 2010/01/08 11:08:50 jochen Exp $
+.\" $Id: t-prot.1,v 1.182 2010/04/09 06:45:35 jochen Exp $
.\"
-.TH T-PROT "1" "January 2010" "T-PROT"
+.TH T-PROT "1" "April 2010" "T-PROT"
.SH NAME
-t-prot \- TOFU Protection - Display Filter for RFC 2822 messages
+t-prot \- TOFU Protection - Display Filter for RFC 5322 messages
.SH SYNOPSIS
.BI "t\-prot [" OPTIONS "]..."
.SH DESCRIPTION
.PP
-This program is a filter to improve the readability of
+This program is a filter to improve the readability of internet
messages (emails and usenet posts) by *hiding* some annoying parts,
e.g. mailing list footers, signatures, and TOFU (see definition below),
as well as squeezing sequences of blank lines or punctuation.
+The program also detects TOFU or a high quoting ratio in a message (so
+you may take appropriate action, e.g. when submitting messages to a mailing
+list or a news server).
.br
The filter is written in Perl and relies on input to be a single
-message conforming to RFC 822 or its successor, RFC 2822. Messages
-conforming to RFCs 2045-2049 should be treated reasonably correct.
+message conforming to RFC 822 or its successors, RFC 2822 and RFC 5322.
+In messages conforming to MIME (RFCs 2045-2049) t\-prot handles text/plain
+parts, others are not touched.
.PP
Already reformatted messages are handled well: the script was
initially designed to cope with the output of the MUA mutt (which
is the reason for not using standard CPAN modules for handling
messages).
.PP
-For easy usage, you can include the file muttrc.t\-prot in your
-~/.muttrc. Also coming with the t-prot package is the example S-Lang
-macro t\-prot.sl for using t-prot from within slrn.
+T\-prot offers example configuration files for mutt, Heirloom mailx
+and metamail. Also coming with the t-prot package is the example
+S-Lang macro t\-prot.sl for using t-prot from within slrn. There is
+a proof\-of\-concept filter for INN2, which you will have to adapt
+to the needs of the news site you host. For use with sendmail's
+alias(5) file, please see below (the option
+.BR "\-p" " provides an example line)."
.SH OPTIONS
+.PP
+If you do not specify any options, t\-prot does ... nothing. Every
+feature you want must be turned on explicitly.
+Admittedly, we have quite a number of options for t\-prot. To limit
+confusion they are grouped into five sections: Input/Output Options,
+Advertisement And Mailing List Footers, Filtering Options, Detection
+Options, and Other Options. While the others should be quite clear,
+filtering and detection might deserve a word (or two).
+.PP
+If you want to tune the appearance of your mail from within your MUA
+(or news messages from within your NUA), then go for the Filtering
+Options section.
+.PP
+If you want to use t\-prot to check on mails before they are submitted
+to mailing lists, fed to your news server, or delivered by your MDA,
+then have a peek at the Detection Options section. You may accept or
+reject/bounce messages depending on t\-prot's result.
+.SH INPUT/OUTPUT OPTIONS
.TP
-.BR "\-A" =DIRECTORY
-"ad footer directory":
-Defines the directory which contains the advertisement list footers (one
-footer per file) which are to be tested when removing them with options
-.B \-a
-or
-.BR \-\-ftr\-ad .
-.br
-This option is also needed if you do not want signature lengths to be
-counted wrong or fullquotes get undetected when an ad footer is
-appended at the bottom of the message (especially when using
-.B "\-S"
-or
-.BR "\-t" ).
+.BR "\-i" " FILE"
+Defines an input file; default is '\-' i.e. STDIN.
+.TP
+.BR "\-o" " FILE"
+Defines the output file; default is STDOUT.
+.TP
+.B "\-\-body"
+Input consists just of the message's body. There are no RFC 5322 header
+lines.
+.IP
+.IR NOTE :
+This does not work with \-\-pgp\-short, and multipart messages will not
+be detected due to missing headers.
+.TP
+.B "\-\-lax\-security"
+Allow insecure writing method. DO NOT USE UNLESS YOU REALLY KNOW WHAT
+YOU ARE DOING. (This ugly workaround is needed for some early mutt versions
+and should NEVER be used as a default, otherwise it will probably turn into
+a security issue.)
+.sp
+You can use this option safely to enable
+.B "\-o"
+/dev/null (or other files which cannot be changed with the user's
+privileges).
+.TP
+.BR "\-\-max\-lines" =x
+Maximum number of lines a message may count (with headers). If the message
+is longer than x lines, the message will not be processed but printed
+unmodified. Exit status will be EX_DATAERR except when called with \-Mmutt.
+.SH ADVERTISEMENT AND MAILING LIST FOOTERS
.TP
.B "\-a"
"commercial signature":
.br
This option compares the last lines of the message body with any
footer file found in the directory specified with
-.BR "\-A" =DIRECTORY
+.BR "\-A" " DIRECTORY"
(which is mandatory for this option). The comparison is done by perl's
.B index()
function (please try
.B \-\-ftr\-ad
is specified.
.TP
-.BR "\-\-bigq" [=n[,x]]
-"shrink big quotes":
-Blocks of quotes with more than n lines will be shrunk to x lines.
-Defaults are 30 for n and 10 for x.
-.TP
-.B "\-\-body"
-Input consists just of the message's body. There are no RFC 2822 header
-lines.
-.IP
-.IR NOTE :
-This does not work with \-\-pgp\-short, and multipart messages will not
-be detected due to missing headers.
-.TP
-.B "\-c"
-"compress":
-Squeezes a sequence of blank lines to two blank lines at maximum.
-.TP
-.BR "\-\-check" [=FLAGS]
-Run checks. If successful, print an error message and quit with
-an appropriate exit code. Useful e.g. for rejecting messages from
-within INN2.
-.IP
-Flags are separated by commas (no whitespaces), and can be the
-following (right now just one flag):
-.IP
-.IR ratio [=n]
-.br
-If the quoting ratio is n or more, the message is rejected. Must be
-between 0 and 1, or else it is entirely disabled. Default is 0.75
-(i.e., three quarters are quoted).
-.TP
-.B "\-d, \-\-debug"
-"debug":
-Print envelope info to syslog when bouncing TOFU contaminated email.
-Default syslog facility is mail.debug. Requires
-.BR \-p .
-.TP
-.B "\-\-diff"
-"tolerate diff":
-Tolerate unified diff (see
-.BR diff (1)
-and
-.BR patch (1))
-appended *after* the signature (which usually makes the signature too long
-to be valid). Not entirely consequent but sometimes useful, e.g. at the
-Linux kernel mailing list.
-.TP
-.B "\-e"
-"ellipsis":
-Squeezes a sequence of four or more dots, exclamation marks, or question marks
-to only three dots or marks, respectively.
-.TP
.B "\-\-ftr\-ad"
"enable aggressive ad footer matching":
With this option enabled, t-prot makes footer detection really greedy: We
.sp
.IR NOTE :
This requires a directory with footer files to be given with option
-.BR "\-A" =DIRECTORY.
+.BR "\-A" " DIRECTORY".
+.TP
+.BR "\-A" " DIRECTORY"
+"ad footer directory":
+Defines the directory which contains the advertisement list footers (one
+footer per file) which are to be tested when removing them with options
+.B \-a
+or
+.BR \-\-ftr\-ad .
+.br
+This option is also needed if you do not want signature lengths to be
+counted wrong or fullquotes get undetected when an ad footer is
+appended at the bottom of the message (especially when using
+.B "\-S"
+or
+.BR "\-t" ).
+.TP
+.B "\-l"
+"list signature":
+Hides "footers" (signatures) from mailing lists. Footer detection works like the
+.B "\-a"
+option.
+.sp
+.IR NOTE :
+This requires a directory with footer files to be given with option
+.BR "\-L" " DIRECTORY".
+.B \-l
+is not needed if
+.B \-\-ftr\-ml
+is specified.
.TP
.B "\-\-ftr\-ml"
"enable aggressive mailing list footer matching":
.sp
.IR NOTE :
This requires a directory with footer files to be given with option
-.BR "\-L" =DIRECTORY.
+.BR "\-L" " DIRECTORY".
.TP
-.B "\-\-groupwise"
-"delete Groupwise style TOFU":
-Hides TOFU as produced by Novell Groupwise.
+.BR "\-L" " DIRECTORY"
+"list footer directory":
+Defines the directory which contains the mailing list footers (one footer
+per file) which are to be tested when removing them with the options
+.B \-l
+or
+.BR \-\-ftr\-ml .
+.br
+This option is also needed if you do not want signature lengths to be
+counted wrong or fullquotes get undetected when a mailing list footer is
+appended at the bottom of the message (especially when using
+.B "\-S"
+or
+.BR "\-t" ).
+.SH FILTER OPTIONS
.TP
-.B "\-h, \-\-help"
-"help":
-Displays a short help text with a summary on all options, and exits.
+.BR "\-\-bigq" [=n[,x]]
+"shrink big quotes":
+Blocks of quotes with more than n lines will be shrunk to x lines.
+Defaults are 30 for n and 10 for x.
+.TP
+.BR "\-c" " [n]"
+"compress":
+Squeezes a sequence of blank lines to just n blank lines. n defaults to 2.
.TP
-.BR "\-i" =FILE
-"input file":
-Defines a file for input; the default input is from '\-' i.e. STDIN.
+.B "\-\-diff"
+Tolerate unified diff (see
+.BR diff (1)
+and
+.BR patch (1))
+appended after the signature (which usually makes the signature too long
+to be valid).
+.sp
+Also, protect diff standard output from hiding (which would otherwise be easy
+prey for
+.BR "\-t" ).
+.TP
+.B "\-e"
+"ellipsis":
+Squeezes a sequence of four or more dots, exclamation marks, or question marks
+to only three dots or marks, respectively.
+.TP
+.B "\-\-groupwise"
+Hides TOFU as produced by Novell Groupwise.
.TP
.B "\-k"
"anti Kammquote":
.IP
Please also note that enabling this option is quite a performance hit.
.TP
-.BR "\-\-kminl" =n
-Minimum line length for wrapped line detection on Kammquotes. For
-details, please see the source code.
+.BR "\-\-kdiff" =n
+Minimum length difference between two lines for wrapped line detection on
+Kammquotes. For details, please see the source code.
.br
Anyway, lower values make the algorithm more aggressive, higher values
-make Kammquotes harder to detect. Default is 65.
+make Kammquotes harder to detect. Default is 20.
.sp
Requires
.BR "\-k" .
Requires
.BR "\-k" .
.TP
-.BR "\-\-kdiff" =n
-Minimum length difference between two lines for wrapped line detection on
-Kammquotes. For details, please see the source code.
+.BR "\-\-kminl" =n
+Minimum line length for wrapped line detection on Kammquotes. For
+details, please see the source code.
.br
Anyway, lower values make the algorithm more aggressive, higher values
-make Kammquotes harder to detect. Default is 20.
+make Kammquotes harder to detect. Default is 65.
.sp
Requires
.BR "\-k" .
.TP
-.BR "\-L" =DIRECTORY
-"list footer directory":
-Defines the directory which contains the mailing list footers (one footer
-per file) which are to be tested when removing them with the options
-.B \-l
-or
-.BR \-\-ftr\-ml .
-.br
-This option is also needed if you do not want signature lengths to be
-counted wrong or fullquotes get undetected when a mailing list footer is
-appended at the bottom of the message (especially when using
-.B "\-S"
-or
-.BR "\-t" ).
-.TP
-.B "\-l"
-"list signature":
-Hides "footers" (signatures) from mailing lists. Footer detection works like the
-.B "\-a"
-option.
-.sp
-.IR NOTE :
-This requires a directory with footer files to be given with option
-.BR "\-L" =DIRECTORY.
-.B \-l
-is not needed if
-.B \-\-ftr\-ml
-is specified.
-.TP
-.B "\-\-lax\-security"
-Allow insecure writing method. DO NOT USE UNLESS YOU REALLY KNOW WHAT
-YOU ARE DOING. (This ugly workaround is needed for some early mutt versions
-and should NEVER be used as a default, otherwise it will probably turn into
-a security issue.)
-.TP
.BR "\-\-locale" =LOCALE
-"locale to use":
-Here you can specify which locale to use. Right now this option is only
-used when
+Specify which locale to use for correct parsing of your MUA's formatting
+of the displayed message (usually it is the locale your MUA uses). Right
+now this option is only used when
.I "\-Mmutt"
is specified, but this may change in future. You need the Perl module
.I "Locale::gettext"
and some header lines...)
.TP
.B "\-\-ms\-smart"
-Burn CPU cycles trying to be smart with MS style TOFU.
+Burn CPU cycles trying to be smart with MS style TOFU: If there are PGP
+signed parts inside the TOFU, the text still might conceal other message
+parts and therefore should not be deleted.
.sp
Please note that this is probably just a waste of time because most
MS Outlook users who do produce this kind of TOFU won't care about
.BR "\-Mmutt" " and"
.BR "\-m" .
.TP
-.BR "\-\-max\-lines" =x
-Maximum number of lines a message may count (with headers). If the message
-is longer than x lines, the message will not be processed but printed
-unmodified. Exit status will be EX_DATAERR except when called with -Mmutt.
-.TP
-.BR "\-o" =FILE
-"output file":
-Define the file to be written *to*; the default output is to STDOUT.
-.sp
-If "NONE" is specified as file name, there will be no output at all (useful
-if all you need is t-prot's exit code). Use something like "./NONE" if you
-really want to write to a file of this name.
-.TP
-.BR "\-p" [=ADDRESS]
-"picky delivery":
-If we really find some TOFU, abort with exit code
-.IR EX_UNAVAILABLE .
-Otherwise redirect the message to ADDRESS if given.
-.sp
-Intended for use from within mail delivery agents (MDAs) or mail transport
-agents (MTAs), or even from with INN, so the message bounces if TOFU is
-detected, and does not get on *your* nerves. :)
-.sp
-As an example for usage with
-.BR sendmail ,
-put this line into your alias file and invoke
-.BR newaliases :
-.sp
-notofu: |"/usr/local/bin/t\-prot \-mt \-p=user@mydomain"
-.sp
-This will bounce messages for <notofu@domainname> if any TOFU is detected
-inside the message, and deliver it to <user@mydomain> otherwise.
-.I Note
-that TOFU is only detected if you specify
-.B "\-t"
-respectively
-.BR "\-m" .
-.sp
-.B PLEASE be careful not to bounce messages to mailing lists!
-.TP
-.BR "\-P" =MESSAGE
-"user defined bounce message for picky delivery":
-You may specify your own bounce message to be returned when we try to deliver
-an email and bounce it because there is TOFU inside.
-.TP
.B \-\-pgp\-move
Move PGP verification output to bottom; requires
.BR "\-Mmutt" .
.TP
.B \-\-pgp\-move\-vrf
-Move PGP verification outout to bottom only if verification shows a good signature
+Move PGP verification output to bottom only if verification shows a good signature
and the signature could be verified as authentic (using a trust path). If there is
any problem with the signature, the PGP output should not be moved so the user is
more likely to notice. Requires
"rip header off":
Hides all mail header lines.
.TP
-.BR "\-S" [=n]
+.B "\-\-reply"
+Subject lines with multiple reply prefixes (Re: and translations into other
+languages) get squeezed to only one prefix.
+.TP
+.BR "\-S" " [n]"
"supression of overlong signatures":
Signatures are to be n lines (not including the one containing dash-dash-space)
or less. If there are more, it is probably not that spirited after all.
Hides signatures, i.e. all lines after a "signature dashes" line,
i.e. a line with three characters: dash-dash-space (no more, no less).
.TP
+.B "\-\-sani"
+Sanitize headers "To:", "From:" and "Subject:": Quoted-printable gets
+fixed to the corresponding chars. German Umlauts are translated to their
+"ae", "oe", "ue" pendants.
+.br
+Useful e.g. for searching by subject within MUAs like Berkeley mailx.
+.TP
.BR "\-\-sigsmax" [=n]
"maximum number of tolerated signatures":
Here you can define how many signatures you accept to be treated as such.
Hides trailing whitespace (sequences of space and tab).
CAVEAT: This may lead to interesting effects with crossposts
between mailing lists or with undetected signature attempts.
+.SH DETECTION OPTIONS
+.TP
+.BR "\-P" " MESSAGE"
+"user defined bounce message for picky delivery":
+You may specify your own bounce message to be returned when we try to deliver
+an email and bounce it because there is TOFU inside. See
+.BR "\-p" .
+.TP
+.BR "\-p" " [ADDRESS]"
+"picky delivery":
+If we really find some TOFU, abort with exit code
+.IR EX_UNAVAILABLE .
+Otherwise redirect the message to ADDRESS if given.
+.sp
+Intended for use from within mail delivery agents (MDAs) or mail transport
+agents (MTAs), or even from within INN, so the message bounces if TOFU is
+detected, and does not get on *your* nerves. :)
+.sp
+As an example for usage with
+.BR sendmail ,
+put this line into your alias file and invoke
+.BR newaliases :
+.sp
+notofu: |"/usr/local/bin/t\-prot \-mt \-p=user@mydomain"
+.sp
+This will bounce messages for <notofu@domainname> if any TOFU is detected
+inside the message, and deliver it to <user@mydomain> otherwise.
+.I Note
+that TOFU is only detected if you specify
+.B "\-t"
+respectively
+.BR "\-m" .
+.sp
+.B PLEASE be careful not to bounce messages to mailing lists!
+.TP
+.BR "\-\-check" [=FLAGS]
+Run checks. If successful, print an error message and quit with
+an appropriate exit code. Useful e.g. for rejecting messages from
+within INN2.
+.IP
+Flags are separated by commas (no whitespaces), and can be the
+following (right now just one flag):
+.IP
+.IR ratio [=n]
+.br
+If the quoting ratio is n or more, the message is rejected. Must be
+between 0 and 1, or else it is entirely disabled. Default is 0.75
+(i.e., 75% of the message lines are quotes).
+.TP
+.B "\-d, \-\-debug"
+Print envelope info to syslog when bouncing TOFU contaminated email.
+Default syslog facility is mail.debug. Requires
+.BR \-p .
+.SH OTHER OPTIONS
+.TP
+.B "\-h, \-\-help"
+Displays a short help text with a summary on all options, and exits.
.TP
.B "\-v, \-\-version"
-"version info":
-Prints the current version number and release date, and exits.
+Prints the current version number and exits.
.SH ENVIRONMENT
The environment variables
.IR LC_ALL ,
.SH IDEAS AND INSPIRATION
Many good ideas, bug reports and support from (in alphabetical order) Bjoern
Buerger, Bjoern Laessig, Christian Borss, Gerfried Fuchs, Martin Neitzel, Martin
-Dietze, Ralf Doeblitz, Sven Guckes and many more (see the ChangeLog for active
-contributors). Many thanks to all of them!
+Dietze, Matthias Kilian, Ralf Doeblitz, Sven Guckes and many more (see the
+ChangeLog for active contributors). Many thanks to all of them!
.sp
Many thanks to Gerhard H. Wrodnigg who uses a TOFU protection script
in order to keep the responses to his cancel bot reasonably short. The
.BR perl (1),
.BR aliases (5),
.sp
-RFCs 2045-2049 and 2822,
+RFCs 2045-2049 and 5322,
.sp
+.I http://freshmeat.net/articles/t\-prot/
+(a nice, solid introduction),
+.br
.I http://got.to/quote/
(German language),
.br
projects / pkg / t-prot.git / blobdiff