1 .\" $Id: t-prot.1,v 1.148 2010/02/07 14:31:14 jochen Exp $
3 .TH T-PROT "1" "February 2010" "T-PROT"
5 t-prot \- TOFU Protection - Display Filter for RFC 5322 messages
7 .BI "t\-prot [" OPTIONS "]..."
10 This program is a filter to improve the readability of
11 messages (emails and usenet posts) by *hiding* some annoying parts,
12 e.g. mailing list footers, signatures, and TOFU (see definition below),
13 as well as squeezing sequences of blank lines or punctuation.
15 The filter is written in Perl and relies on input to be a single
16 message conforming to RFC 822 or its successors, RFC 2822 and RFC 5322.
17 Messages conforming to RFCs 2045-2049 should be treated reasonably correct.
19 Already reformatted messages are handled well: the script was
20 initially designed to cope with the output of the MUA mutt (which
21 is the reason for not using standard CPAN modules for handling
24 For easy usage, you can include the file muttrc.t\-prot in your
25 ~/.muttrc. Also coming with the t-prot package is the example S-Lang
26 macro t\-prot.sl for using t-prot from within slrn.
30 "ad footer directory":
31 Defines the directory which contains the advertisement list footers (one
32 footer per file) which are to be tested when removing them with options
37 This option is also needed if you do not want signature lengths to be
38 counted wrong or fullquotes get undetected when an ad footer is
39 appended at the bottom of the message (especially when using
45 "commercial signature":
46 Hides "footers" (signatures) from commercial email providers.
48 This option compares the last lines of the message body with any
49 footer file found in the directory specified with
51 (which is mandatory for this option). The comparison is done by perl's
58 This option is not needed if
62 .BR "\-\-bigq" [=n[,x]]
64 Blocks of quotes with more than n lines will be shrunk to x lines.
65 Defaults are 30 for n and 10 for x.
68 Input consists just of the message's body. There are no RFC 2822 header
72 This does not work with \-\-pgp\-short, and multipart messages will not
73 be detected due to missing headers.
77 Squeezes a sequence of blank lines to just two blank lines.
79 .BR "\-\-check" [=FLAGS]
80 Run checks. If successful, print an error message and quit with
81 an appropriate exit code. Useful e.g. for rejecting messages from
84 Flags are separated by commas (no whitespaces), and can be the
85 following (right now just one flag):
89 If the quoting ratio is n or more, the message is rejected. Must be
90 between 0 and 1, or else it is entirely disabled. Default is 0.75
91 (i.e., 75% of the message lines are quotes).
94 Print envelope info to syslog when bouncing TOFU contaminated email.
95 Default syslog facility is mail.debug. Requires
100 Tolerate unified diff (see
104 appended *after* the signature (which usually makes the signature too long
105 to be valid). Not entirely consequent but sometimes useful, e.g. at the
106 Linux kernel mailing list.
110 Squeezes a sequence of four or more dots, exclamation marks, or question marks
111 to only three dots or marks, respectively.
114 "enable aggressive ad footer matching":
115 With this option enabled, t-prot makes footer detection really greedy: We
116 assume that commercial email providers aren't even frightened to append
117 changing texts *under* their ads which are appended to the message body.
118 Because these texts even have changing lengths we simply detect the
119 lines of the footer *anywhere* in the body of the message and assume that
120 everything below belongs to the footer. (Man, if life where always that
123 With this option even GMX ads should be easy to hide -- you buy this with
124 a slight performance hit (which is the reason this option is disabled by
125 default), and with the possibility that sometimes the algorithm is just a
129 This requires a directory with footer files to be given with option
130 .BR "\-A" =DIRECTORY.
133 "enable aggressive mailing list footer matching":
134 With this option enabled t-prot makes footer detection really greedy: Should
135 be helpful with broken list servers, or even if your email provider munges
136 the bodies of your messages.
138 Works similar to \-\-ftr\-ad, just that it is intended for mailing list footers.
141 This requires a directory with footer files to be given with option
142 .BR "\-L" =DIRECTORY.
145 Hides TOFU as produced by Novell Groupwise.
148 Displays a short help text with a summary on all options, and exits.
152 Defines a file for input; the default input is from '\-' i.e. STDIN.
156 Tries (not too aggressively) to fix those broken zig-zag-shaped lines
157 wrapped around by some MUAs which are known as "Kammquoting" in German.
160 This option is considered stable by now. However, sometimes Kammquotes
161 should have been removed but weren't. Please send a bug report if this
162 happens to you (after carefully reading the BUGS and REPORTING BUGS
163 section of this man page, that is).
165 Please also note that enabling this option is quite a performance hit.
168 Minimum line length for wrapped line detection on Kammquotes. For
169 details, please see the source code.
171 Anyway, lower values make the algorithm more aggressive, higher values
172 make Kammquotes harder to detect. Default is 65.
178 Maximum line length for wrapped line detection on Kammquotes. For
179 details, please see the source code.
181 Anyway, higher values make the algorithm more aggressive, lower values
182 make Kammquotes harder to detect. Default is 80.
188 Minimum length difference between two lines for wrapped line detection on
189 Kammquotes. For details, please see the source code.
191 Anyway, lower values make the algorithm more aggressive, higher values
192 make Kammquotes harder to detect. Default is 20.
198 "list footer directory":
199 Defines the directory which contains the mailing list footers (one footer
200 per file) which are to be tested when removing them with the options
205 This option is also needed if you do not want signature lengths to be
206 counted wrong or fullquotes get undetected when a mailing list footer is
207 appended at the bottom of the message (especially when using
214 Hides "footers" (signatures) from mailing lists. Footer detection works like the
219 This requires a directory with footer files to be given with option
220 .BR "\-L" =DIRECTORY.
226 .B "\-\-lax\-security"
227 Allow insecure writing method. DO NOT USE UNLESS YOU REALLY KNOW WHAT
228 YOU ARE DOING. (This ugly workaround is needed for some early mutt versions
229 and should NEVER be used as a default, otherwise it will probably turn into
232 .BR "\-\-locale" =LOCALE
234 Here you can specify which locale to use. Right now this option is only
237 is specified, but this may change in future. You need the Perl module
246 with locales, t-prot will only work correctly if you specify the corresponding
247 locale string. Alternatively, you can use the environment variables
252 to specify the locale string.
255 You also have to make sure you are running t-prot
260 versions. T-prot detects
264 locales of the recent stable versions of those programs, earlier versions
265 might not work well with a recent version of t-prot. There are patches available
266 to make t-prot fit into environments with some other
272 .BR "\-M, \-\-mua" =MUA
274 Turn on special treatment for some mail user agents. (Right now only
276 is supported, but more might be added in future.)
278 If your MUA is supported by this feature you must ensure t-prot makes
279 use of it when called from within your MUA to work as desired.
283 Hides TOFU as given by some Microsoft mailers. (You all surely know these
284 fullquotes beginning with
286 "\-\-\-\-\- Original Message \-\-\-\-\-"
288 and some header lines...)
291 Burn CPU cycles trying to be smart with MS style TOFU.
293 Please note that this is probably just a waste of time because most
294 MS Outlook users who do produce this kind of TOFU won't care about
295 making their messages the least bit readable or even predictable. So
296 this option will probably just be interesting for mutt message hooks
297 (to activate it on demand when you
298 .BR know " the sender tries to write legible messages)."
304 .BR "\-\-max\-lines" =x
305 Maximum number of lines a message may count (with headers). If the message
306 is longer than x lines, the message will not be processed but printed
307 unmodified. Exit status will be EX_DATAERR except when called with -Mmutt.
311 Define the file to be written *to*; the default output is to STDOUT.
313 If "NONE" is specified as file name, there will be no output at all (useful
314 if all you need is t-prot's exit code). Use something like "./NONE" if you
315 really want to write to a file of this name.
319 If we really find some TOFU, abort with exit code
321 Otherwise redirect the message to ADDRESS if given.
323 Intended for use from within mail delivery agents (MDAs) or mail transport
324 agents (MTAs), or even from with INN, so the message bounces if TOFU is
325 detected, and does not get on *your* nerves. :)
327 As an example for usage with
329 put this line into your alias file and invoke
332 notofu: |"/usr/local/bin/t\-prot \-mt \-p=user@mydomain"
334 This will bounce messages for <notofu@domainname> if any TOFU is detected
335 inside the message, and deliver it to <user@mydomain> otherwise.
337 that TOFU is only detected if you specify
342 .B PLEASE be careful not to bounce messages to mailing lists!
345 "user defined bounce message for picky delivery":
346 You may specify your own bounce message to be returned when we try to deliver
347 an email and bounce it because there is TOFU inside.
350 Move PGP verification output to bottom; requires
353 .B \-\-pgp\-move\-vrf
354 Move PGP verification outout to bottom only if verification shows a good signature
355 and the signature could be verified as authentic (using a trust path). If there is
356 any problem with the signature, the PGP output should not be moved so the user is
357 more likely to notice. Requires
361 If gpg is terminated before finished (e.g. hitting Ctrl-C, or using
363 we cannot always detect if the check was interrupted. Though t-prot tries
364 to be smart, there will be false positives.
367 Hide non-relevant PGP key uids; requires
372 Hides all mail header lines.
375 "supression of overlong signatures":
376 Signatures are to be n lines (not including the one containing dash-dash-space)
377 or less. If there are more, it is probably not that spirited after all.
378 So with this option you trade it for a
382 If no n is given, default is 4. (We do not recommend using a value other
383 than 4. Consider this old-fashioned, but we actually do *like* RFC
387 The line containing "\-\- " ist not counted when testing for an overlong
388 signature, but it is included when displaying how many lines were deleted.
391 "signature deletion":
392 Hides signatures, i.e. all lines after a "signature dashes" line,
393 i.e. a line with three characters: dash-dash-space (no more, no less).
395 .BR "\-\-sigsmax" [=n]
396 "maximum number of tolerated signatures":
397 Here you can define how many signatures you accept to be treated as such.
398 (Most significant behaviour is when microsoft style quotes are removed.
399 Experts please see the code for the more subtle implications of this
402 Leave empty or specify zero to have an unlimited number of sigs.
406 "SpamAssassin workaround":
407 SpamAssassin (available at
408 .IR http://spamassassin.org/ ") often is configured that it adds some"
409 lines to the message body containing information about the spam criteria
410 which were found matching for the message. This option enables an extra
411 test to avoid false positives for Microsoft style TOFU on such messages.
415 Hides "traditional style" TOFU, where each line begins with an
416 indent string like "> ".
418 (You may edit the indent pattern in the script itself to suit your needs,
424 "whitespace deletion":
425 Hides trailing whitespace (sequences of space and tab).
426 CAVEAT: This may lead to interesting effects with crossposts
427 between mailing lists or with undetected signature attempts.
429 .B "\-v, \-\-version"
430 Prints the current version number and release date, and exits.
432 The environment variables
437 are read and respected when interpreting output by mutt or gnupg
438 (unless they are overruled by the \-\-locale option). T-prot's own
439 output is English regardless of any locale setting.
441 On program exit, t-prot uses exit codes from
442 .I /usr/include/sysexits.h
443 and thus behaves in a manner that sendmail and others understand when
446 Currently, the codes used are
464 If, however, perl fails to compile and execute t-prot, perl's normal
465 exit codes will be returned.
467 TOFU is an abbreviation which mixes German and English words;
468 it expands to "text oben, full-quote unten" which means
469 "text above - full quote below" and describes the style of so
470 many users who let their mailer or newsreader quote everything
471 of the previous message and just add some text at the top;
472 obviously they think that quoted text must not be changed at all.
473 This is quite annoying as it needlessly sends a lot of data
474 even when it is not required. Some editing of messages is desired.
475 Please point these people to the page
476 .I http://www.river.com/users/share/etiquette/edit.html
479 There are several ways to fine-tune t-prot's performance:
481 Some command line options are quite grave a performance hit -- do not
482 use \-k and especially \-\-ms\-smart if you are content without them.
484 Checking for special footers is very costly as well. Put as few footer
485 files as absolutely needed in any footer directory.
487 All PGP related options are eating up lots of CPU time. Try to avoid them
488 on unsigned and unencrypted messages.
490 When calling t-prot from within mutt, you might use mutt's folder-hook
491 and message-hook facilities to turn options on only when needed, e.g. to
492 set up a different footer directory for each mailing list folder.
496 I want to make my mailing list footer files match more different mailing
497 list footers. Can I use regular expressions, or how can I accomplish
501 Nope, regexp's do not work here. The comparison is made by the perl
506 for more detailed info), so you must exactly match the beginning of the
507 line. The longer the line you specify, the more precise the match; if
508 your line is empty you match unconditionally.
512 I use the options \-l and \-L to supress mailing list footers when
513 displaying messages in
515 This does work sometimes, but sometimes it does not: the footer is not
516 detected, and therefore full quotes are not deleted and signatures are
517 detected as too long (which they aren't).
520 This might occur if the message is badly encoded, so mutt cannot resolve
521 all encoded characters, e.g. if you have an encoded message on a mailing
522 list, and majordomo appends a mailing list footer in a different encoding
523 (or even plain us-ascii). "\-\- " simply does not match "\-\-=20".
525 Another problem are non-us-ascii characters. Just avoid them, and
526 everything should work fine.
528 See the preceding Q+A for a solution.
532 I want to write a message which contains parts that should *not* be
533 deleted even when filtered with t-prot. Is this possible?
536 Yes, but please do not spread word of it. Make unobstrusive use of the
543 This line is protected from being filtered by t\-prot !!!!!!!
547 Text coming now is not.
549 Written by Jochen Striepe <t\-prot@tolot.escape.de>.
551 All of the documentation and software included in the t-prot releases
552 is copyrighted by Jochen Striepe (except when explicitly stated otherwise).
554 Copyright \(co 2001-2010 Jochen Striepe. All rights reserved.
556 Redistribution and use, with or without modification, are permitted
557 provided that the following conditions are met:
559 1. Redistributions of source code must retain the above copyright notice,
560 this list of conditions and the following disclaimer.
562 2. All advertising materials mentioning features or use of this software
563 must display the following acknowledgement:
565 This product includes software developed by Jochen Striepe and others.
567 3. Neither the name of the author nor the names of any contributors may
568 be used to endorse or promote products derived from this software without
569 specific prior written permission.
571 THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ANY
572 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
573 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
574 DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY
575 DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
576 (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
577 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
578 ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
579 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
580 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
581 .SH IDEAS AND INSPIRATION
582 Many good ideas, bug reports and support from (in alphabetical order) Bjoern
583 Buerger, Bjoern Laessig, Christian Borss, Gerfried Fuchs, Martin Neitzel, Martin
584 Dietze, Matthias Kilian, Ralf Doeblitz, Sven Guckes and many more (see the
585 ChangeLog for active contributors). Many thanks to all of them!
587 Many thanks to Gerhard H. Wrodnigg who uses a TOFU protection script
588 in order to keep the responses to his cancel bot reasonably short. The
589 entire inspiration for this hack came from the "TOFU protection" line of
590 his script on many usenet postings.
592 You can get the latest version from
593 .IR http://www.escape.de/users/tolot/mutt/ .
595 There is a problem when mutt gives a PGP verified or even a multipart
596 message to t-prot: The information where the PGP encrypted/signed data
597 or even attachments begin and end is plainly embedded in the text, not
598 really cleanly recognizable for t-prot. The problem should be worked
599 around by now, please send a bug report if it does not work for you.
601 Please note that t-prot development happens on
602 .BR "current stable perl versions only" .
603 If you do run t-prot on earlier (or unstable) perl versions, you might
604 encounter perl compiler bugs (or funny t-prot behaviour). One solution
605 is to upgrade your perl, another is simply to write a bug report. If
606 you do not run a current perl version, please include this information
613 * you found it in the TODO file coming with the distribution. We do
614 know those and try to fix them as soon as possible.
616 * you have an old t-prot version. If you encounter a problem, first
617 see if there is a new t-prot version which fixes the issue. If you
618 upgraded to the latest version and it still occurs, a bug report is
621 If you noticed a bug when processing a message and want to provide the
622 t-prot team with some useful info, please:
624 * if invoking t-prot by mutt's
626 facility, just set display_filter to something like
628 "tee ~/foobar | t\-prot <your options>"
630 and include ~/foobar in the bug report -- this way we might reproduce
631 the bug much easier if you are using a different environment than we do.
633 * provide information on what command line options you use t-prot
634 with, what perl version t-prot runs on your system, and what else might
635 be important to enable us reproducing the bug.
637 Send your bug report to
638 .IR <t\-prot\-bugs@tolot.escape.de> .
643 section). Beside that, all main features should be implemented by now.
644 See the TODO file for more information.
648 and the part about "display_filter",
652 RFCs 2045-2049 and 5322,
654 .I http://got.to/quote/
657 .I http://www.river.com/users/share/etiquette/edit.html
658 (the Learn To Edit Messages HowTo has found a new home).