Subject: Text canonicalization in RFC 2440 - backwards compatibility?

From: Thomas Roessler <[email-masked]>
Date: Wed, 4 Oct 2000 12:36:38 +0200

[This thread got started since PGP signatures on the bugtraq list
did not verify correctly.  However, these signatures are (1)
text-mode, and (2) the modification apparently concerns trailing
whitespace.]

This smells like a discrepancy between RFC 2440 and the classical
PGP implementation has crept in, and gone unnoticed for quite some
time.

Essentially, RFC 2440 says that we shouldn't observe the problems
occuring on bugtraq: Peter is generating "canonical text"
signatures.  RFC 2440 says:

   0x01: Signature of a canonical text document.
            Typically, this means the signer owns it, created it, or
	    certifies that it has not been modified.  The signature
	    is calculated over the text data with its line endings
	    converted to <CR><LF> and trailing blanks removed.

However, when experimenting with PGP 2.6.3in, I'm observing that
canonical text signatures _do_ take trailing whitespace into
account.

Now, let's look at the older docs: RFC 1991 doesn't seem to define
canonical text mode at all.  pgpdoc2.txt from the PGP 2
distribution, however, just says this: "Canonical text has a
carriage return and a linefeed at the end of each line of text."

This is a precise description of the behaviour, and actually matches
the expectation which seems to have been implicit to RFC 2015, which
only deals with line-end canonicalization, but not with the
signature mode to be used.

To make things worse, the "clearsign" signatures of pgp2 _do_
correspond with what RFC 2440 says about canonical text documents in
general.

Now, what are the recent implementations (PGP 5/6/7, GnuPG) doing
about all this?  Are they compatible:

- to each other?
- to PGP 2.6?

Or am I just confused?

On 2000-10-04 11:55:29 +0200, Peter J . Holzer wrote:
> Date: Wed, 4 Oct 2000 11:55:29 +0200
> From: "Peter J . Holzer" <[email-masked]>
> To: Lars Hecking <[email-masked]>
> Cc: Thomas Roessler <[email-masked]>,
> 	[email-masked]
> Subject: Re: rcp file transfer hole (was: scp file transfer hole)
> X-Mailer: Mutt 0.95.3i
> 
> [I have included Elias in the Cc, because I think he might want to know
> about this "feature" of his mailing-list software]
> 
> 
> On 2000-10-04 10:06:34 +0100, Lars Hecking wrote:
> >  Any idea why half the signatures on bugtraq don't check out? Do you
> >  have an fcc copy of your posting that you can compare with the posted
> >  message?
> 
> Yes.
> 
> Looks like Bugtraq strips trailing spaces from each line. This will
> garble the signature separator ("-- ") and therefore the pgp signature
> won't check out any more. 
> 
> I guess I should add 
> 
> send-hook '~t [email-masked]' 'set pgp_strict_enc'
> 
> to my .muttrc.
> 
> For people which don't use PGP signatures, but want their normal
> signatures to be separated correctly, it would be nice if mutt could be
> forced to use quoted printable even for ascii messages. Thomas?
> 
> 	hp
> 
> -- 
>    _  | Peter J. Holzer      | Any setuid root program that does an
> |_|_) | Sysadmin WSR / LUGA  | exec() somewhere is just a less
> | |   | [email-masked]        | user friendly version of su.
> __/   | http://www.hjp.at/   |    -- Olaf Kirch on bugtraq 2000-08-07

-- 
Thomas Roessler                         <[email-masked]>

---

From: "Peter J . Holzer" <[email-masked]>
Date: Wed, 4 Oct 2000 14:20:01 +0200

> about all this?  Are they compatible:

I think that GnuPG is not compatible with PGP 2.6. Lars (using GnuPG)
couldn't verify a message (signed with PGP 2.6) I sent to him in a
tarred and compressed folder (to avoid any corruption in transit). I'll
try PGP 6.5 (which I have installed at home) this evening.

	hp

---

From: Lars Hecking <[email-masked]>
Date: Wed, 4 Oct 2000 13:25:46 +0100

> try PGP 6.5 (which I have installed at home) this evening.
 
 Just re-tried with pgp 2.6.3i, and yes, this time it works.

 -l

-- 
Be free and open and breezy!  Enjoy!  Things won't get any better so
get used to it.

---

From: Ulf Möller <[email-masked]>
Date: Thu, 5 Oct 2000 20:13:06 -0400

> PGP implementation has crept in, and gone unnoticed for quite some

Sometimes I have the impression that most of my comments on the
OpenPGP drafts fall in the "gone unnoticed" category (I first pointed
it out on September 16th, 1998, again on October 13th -- RFC 2440 was
published in November --, once more on February 10th, 1999, and again
on December 23rd after a cryptic note on the subject appeared in
2440bis-00); but we actually have the following text in the latest
draft:

     * PGP 2.6.X and 5.0 do not trim trailing whitespace from a
       "canonical text" signature. They only remove it from cleartext
       signatures. These signatures are not OpenPGP compliant --
       OpenPGP requires trimming the whitespace. If you wish to
       interoperate with PGP 2.6.X or PGP 5, you may wish to accept
       these non-compliant signatures.

---

From: Jon Callas <[email-masked]>
Date: Thu, 5 Oct 2000 20:52:40 -0700

>       these non-compliant signatures.

You're welcome.

I've said this a number of times, and I think it bears repeating.

If I get a request about OpenPGP that is vague, I may not do what the
requester intends. The ways in which I can fall down include putting the
wrong thing in the draft, but they are also likely to be that I'll do
nothing when you wanted something.

There are a couple of reasons I might do nothing. The usual one is that I
don't know what you want me to do. OpenPGP has small quantities of
sub-optimal things in it. Usually, those things are there because of
backwards compatibility.

However, at the start of this working group, we decided that backwards
compatibility is important and desirable, but not essential. So there are
things we can, and have, broken because it makes a better standard moving
forward.

So sometimes I interpret someone's complaint as a complaint, not as a
suggestion for a change in the document.

I know that in your case, I've made a number of errors of this sort. I am
truly sorry for this, and I've been trying to make a special effort to pay
closer attention to what you want.

However, it makes it *much* easier for me if you tell me what you want. The
best thing is something like:

Please change XXX to YYY.

For example, I am presently working on getting the PKCS1 updates done.
Thank you for you help. But changing them correctly means that I have to go
dredge out the PKCS1 draft, read it, interpret it, and figure out what to
do. This will take a couple of hours, most likely. In the last few days
I've had several ten minute opportunities to edit. I've had a couple
hour-long ones. I haven't yet had that much time.

If you know what you want text to be, tell me. If I have to do research not
only will I do it slower, but there's a chance I may decide that the best
thing to do is nothing.

In short -- if you make me think, I may end up concluding something other
than you want.

Again, though, I'm sorry I've screwed up. It isn't that I'm ignoring you.

	Jon