Subject: confidential subject lines -> use content description field w/ pgp/mime?

From: [email-masked]
Date: Fri, 02 Jun 2000 18:16:11 +0900

problem
-------

imo, as the number of messages using pgp (or pgp/mime) for a user
increases, the need for a mechanism for protecting the confidentiality
of subject lines (i.e. what a user sees as a subject in a mail client)
becomes greater.

important information is often stored in the subject line and not
having a meaningful subject which was specified by the author of the
message can be a serious inconvenience to users (e.g. searching,
referring to past messages in communication, etc.) -- which may likely
lead to a decision on the user's part to go ahead and place sensitive
information in the subject line.

suggestion
----------

i think it would be a good idea to address this by defining a common
specification.  afaik, there isn't one yet -- please tell me if you
know of one already.

i've asked around on a few mailing lists for ideas about this, and,
imho, the most promising one i heard was to give meaning to the
Content-Description field for a particular mime part.

the basic idea is to store the information which would normally be the
value of a Subject header as the value of a Content-Description header
which is placed in a "to-be-encrypted" text message part.

Example:

        From: Michael Elkins <[email-masked]>
        To: Michael Elkins <[email-masked]>
        Subject: encrypted in CD:
        Mime-Version: 1.0
        Content-Type: multipart/encrypted; boundary=foo;
           protocol="application/pgp-encrypted"

        --foo
        Content-Type: application/pgp-encrypted

        Version: 1

        --foo
        Content-Type: application/octet-stream

      & Content-Type: text/plain
      & Content-Description: this is the subject the user will see
      &
      & here is some message text
      &
      & :-)

        --foo--

where, as in the (open)pgp/mime specification, the ampersands indicate
which portion of the text is encrypted.

upon/after decryption, mail clients would make an effort to display
the contents of the aforementioned Content-Description field in
relevant user interface portions (e.g. title of a message window,
where subject text might be displayed in a view that listed
information about a collection of message [one message per line],
etc.).  mail clients might also make use of these Content-Description
values for the purposes of searching and filtering.

how does the basic idea sound?

p.s. this idea was originally suggested by Kazu Yamamoto
<[email-masked]>.

---

From: Ian BROWN <[email-masked]>
Date: Fri, 02 Jun 2000 11:28:41 +0100

This is a really important thing we need to sort out. There was extensive 
discussion of it almost two years ago (!) but I don't know if that resulted in 
anything in the PGP/MIME draft.

Ned Freed's suggestion seemed to be widely agreed upon: I've attached his last 
e-mail on the subject, but you can read the discussion in the archives at 
www.imc.org

Ian :)
--
Date: Thu, 17 Sep 1998 11:04:15 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Encrypting RFC822 headers in S/MIME or PGP/MIME messages
To: Ian Bell <[email-masked]>
Cc: [email-masked], [email-masked]

> (Apologies if this has been done to death in the past - I can imagine
> Ned sighing about protracted discussions prior to RFC1847 - but I
> couldn't find any discussion in the archives)

> RFC2311 (SMIME) and RFC1847 (upon which PGP/MIME has been based) only
> allow MIME headers to be protected by the encryption process. Was there
> any discussion during the preparation of RFC1847 about the possibility /
> desirability / feasibility of allowing general RFC822 headers to be
> included in the encrypted part of the message?

There were extensive discussions of this. I probably have several hundred
messages archived on this topic, and there were also WG discussions and several
private meetings where the design team covered this area exhaustively.

The basic conclusion was (and is) that this dog doesn't hunt and cannot be made
to hunt. The problem is that message headers are routinely rewritten by
intervening MTAs as messages are transferred via SMTP. They are refolded,
reordered, addresses are converted from one form to another, encoded
words are added or removed, entire fields are added or deleted.

And all these activities are the rule, not the exception. And it is far more
likely to happen to fields it makes sense to sign, such as subject, to/cc/bcc,
comments, etc. than to any others.

In short, if you want to keep the transports from modifying things, you cannot
put them in the primary header. You either have to put them in the body. And
given this, MIME provides the obvious means to do this if that's what you're
after -- nested messages.

> The most obvious candidates for headers to be encrypted along with the
> MIME headers would be Subject: and Disposition-Notification-To: (the
> subject the sender may have intended to use may be too sensitive to use
> as the subject of the open message: the sender may want any MDN to be
> sent only when the message is decrypted), though cases could probably be
> made for just about any RFC822 header.

> Could (and should) any replacements for RFC2015 and RFC2311 be amended
> to allow RFC822 headers to be sent encrypted, and for the decryption
> process to swap any encrypted headers found with the corresponding
> headers in the actual message?

There is no need to do this. All you need to do is use MIME encapsulation
to put the real headers into the body of the message.

One thing that would be nice is if there was a way to say in the signature
information that the inner message is actually supposed to replace the outer
container message. (This could also be done as a parameter to message/rfc822
if need be, although there are arguments for doing it in the signature.

There is one other thing that is missing, however, and that is a means of
encapsulating the entire message including the envelope. There are many
applications where the envelope itself is sensitive and needs end-to-end
protection.

And there is a proposal on the table to deal with this:
draft-freed-bsmtp-02.txt. This defines envelope encapsulation and the semantics
that necessarily follow from doing it.

> As availability of encryption software becomes more widespread, many
> end-users may find SMIME/PGP most useful as simply a transport security
> mechanism rather than a way of securely storing messages. In any case,
> MUAs implementing PGP or S/MIME probably already allow the user to save
> the decrypted version of a message.

Sure.

> It would be good if there were an interoperable way of making the
> stored, decrypted message reflect the message the author would have
> liked to send in the first place. It would be particularly nice if the
> author could transmit the intended subject of a message when this may be
> too sensitive to put in the open message headers.

See above.

				Ned

---

From: [email-masked]
Date: Fri, 02 Jun 2000 19:44:26 -0700 (PDT)

> e-mail on the subject, but you can read the discussion in the archives at

Please note that since this discussion the BSMTP document has come out as
RFC 2442.

				Ned

---

From: [email-masked]
Date: Mon, 05 Jun 2000 11:54:19 +0900

From: [email-masked]
Subject: Re: confidential subject lines -> use content description field w/ pgp/mime?
Date: Fri, 02 Jun 2000 19:44:26 -0700 (PDT)
Message-ID: <[email-masked]>

> Please note that since this discussion the BSMTP document has come out as

so, if i understand the idea correctly, header protection can be had
by having mail clients support bsmtp processing:

  sending side

  1) create a message
  2) create a bsmtp object out of the message (encapsulate)
  3) encrypt (and optionally sign)
  4) send the message

  receiving side

  4) receive the message
  5) decrypt (and optionally verify signature)
  6) extract (recreate) the original message from the bsmtp object
  7) read as usual

does that sound about right?

---

From: [email-masked]
Date: Sun, 04 Jun 2000 21:11:38 -0700 (PDT)

> does that sound about right?

Yes, except that the protection extends to the envelope when you use BSMTP.
If all you want to protect is the header it is easier to just use a
message/rfc822 MIME encapsulation.

				Ned

---

From: [email-masked]
Date: Mon, 05 Jun 2000 14:08:12 +0900

From: [email-masked]
Subject: Re: confidential subject lines -> use content description field w/ pgp/mime?
Date: Sun, 04 Jun 2000 21:11:38 -0700 (PDT)
Message-ID: <[email-masked]>

...

> message/rfc822 MIME encapsulation.

so to protect the confidentiality of the Subject header:

  sending side

  1) create a rfc 822 message (contains Subject)
  2) mime encapsulate the message
  3) encrypt (and optionally sign)
  4) send the message

  receiving side

  5) receive the message
  6) decrypt (and optionally verify signature)
  7) extract (recreate) the original message
  8) read as usual (can see the Subject)

right?

if there is no better method yet, imo, it would be a good idea for
this method of protecting headers to receive some kind of
"endorsement" (e.g. mentioned in the openpgp/mime or some other spec
for instance -- a separate document is may be too much, but who
knows).

also, i noticed the following statement in a past (septemeber 17th
1998) post by you in the archives:

  There are many applications where the envelope itself is sensitive and 
  needs end-to-end protection.

but i did not notice any concrete examples of such applications.  would you
mind mentioning a few?

http://www.imc.org/ietf-open-pgp/mail-archive/msg01941.html

---

From: [email-masked]
Date: Tue, 06 Jun 2000 18:11:55 -0700 (PDT)

> but i did not notice any concrete examples of such applications.  would you

Two words: traffic analysis. Knowing who's getting a message is often enough to
tell you something even if the contents are obscured.

				Ned

---

From: [email-masked]
Date: Wed, 07 Jun 2000 10:39:20 +0900

thanks for the clarification.

From: [email-masked]
Subject: Re: confidential subject lines -> use content description field w/ pgp/mime?
Date: Tue, 06 Jun 2000 18:11:55 -0700 (PDT)
Message-ID: <[email-masked]>

> Two words: traffic analysis. Knowing who's getting a message is
> often enough to tell you something even if the contents are
> obscured.

sure.  so the "There are many applications" statement referred to
applications where the traffic analysis of envelope information is
relevant.  is that a reasonable interpretation?  also, did you have
other things in mind?

assuming envelope information is protected using rfc 2442, is there a
recommended mechanism for actually getting the encapsulated message
to the receiving end?  not anonymous remailers...

<humor>
i suppose one of these days we'll end up seeing all anonymous remailer
functionality provided via several standards documents...i think that
leaves: splitting messages up into equal size pieces (+ reassembly on
the receiving end of course), sending cover traffic, pooling message 
pieces, making message pieces traverse different paths...am i missing 
anything?  :-)
</humor>

---

From: [email-masked]
Date: Tue, 06 Jun 2000 20:48:38 -0700 (PDT)

> relevant.  is that a reasonable interpretation?  also, did you have

Signing the envelope also has its applications -- it prevents some forms
of envelope tampering that might otherwise be used in some sorts of
attacks.

> to the receiving end?  not anonymous remailers...

Recommended, no. In practice regular SMTP is often used, in effect creating a
variety of secure tunnel.

				Ned

---

From: [email-masked]
Date: Thu, 08 Jun 2000 13:48:40 +0900

From: [email-masked]
Subject: Re: confidential subject lines -> use content description field w/ pgp/mime?
Date: Tue, 06 Jun 2000 20:48:38 -0700 (PDT)
Message-ID: <[email-masked]>

...

> variety of secure tunnel.

i see that the MAIL FROM info for the container message can be made
different from the corresponding information in the bsmtp object w/o
having adverse affects on delivery.

for RCPT TO though, the best i've come up w/ so far is to specify an
address that forwards received messages to other forwarders or to the
ultimate recipient.  is it possible to do better than this?

what am i missing?

thanks for bearing w/ me.

p.s. is this an appropriate place to discuss rfc 2442?  if not, is there
some other place?

---

From: [email-masked]
Date: Wed, 07 Jun 2000 22:15:09 -0700 (PDT)

> ultimate recipient.  is it possible to do better than this?

The RCPT TO needs to be the address of the BSMTP processor. You can have a
single processor for a large number of individual users, and if you do
this, you cannot tell from the outer envelope what user is being sent
to. Not knowing the specific user makes traffic analysis a heck of a lot
harder.

				Ned

---

From: [email-masked]
Date: Fri, 09 Jun 2000 12:37:52 +0900

From: [email-masked]
Subject: Re: confidential subject lines -> use content description field w/ pgp/mime?
Date: Wed, 07 Jun 2000 22:15:09 -0700 (PDT)
Message-ID: <[email-masked]>

> The RCPT TO needs to be the address of the BSMTP processor. 

hmm, so making the RCPT TO (of the outer envelope) an address that
will forward to the address of the BSMTP processor won't work?  i
don't understand why.  what am i missing?

> You can have a single processor for a large number of individual
> users, and if you do this, you cannot tell from the outer envelope
> what user is being sent to. Not knowing the specific user makes
> traffic analysis a heck of a lot harder.

yes, i suppose it does increase the amount of work that must be done
-- i guess that amount is related to how much use the bsmtp processor
is getting and by how many users.  so there should be incentive to use
a processor that is heavily used and by a lot of users.

i suppose one can place encrypted bsmtp objects inside of bsmtp
objects and do chaining of bsmtp processors...

thanks very much for the explanations.

p.s. does anyone know of any bsmtp processor implementations?

---

From: [email-masked]
Date: Thu, 08 Jun 2000 21:27:45 -0700 (PDT)

> don't understand why.  what am i missing?

Nothing; it can be any address that gets to the processor. Any amount
of aliasing or forwarding is fine. But the more actual recipients
associated with a single outer address, the better.

> p.s. does anyone know of any bsmtp processor implementations?

There are a bunch, although many of them are likely not in full compliance with
the RFC, which specifies support for certain aspects of ESMTP. Remember, the
entire BITNET network was implemented using BSMTP to get around the 8X8 address
limit of NJE. In its heyday BITNET amounted to several thousand systems running
on a variety of hardware and software platforms.

One implementation that is up to date is the one in PMDF, which is descended
from our BITNET handling code. The same code is also in SIMS.

I also doubt if we would have written the specification had we not had an
actual implementation in hand. The BITNET experience showed that turning an
interactive protocol into a batch protocol has some subtle spots to it.

				Ned

---

From: Ian Bell <[email-masked]>
Date: Fri, 9 Jun 2000 13:55:44 +0100

>e-mail on the subject, but you can read the discussion in the archives at

So far in this thread, two methods of encrypting headers have been 
discussed: encapsulating a complete message in a message/rfc822 and 
encapsulating an SMTP session in BSMTP. Both can be used for end-to-end 
encryption and the first probably inter-operates quite well amongst 
existing PGP/MIME or S/MIME clients.

However, encrypting message headers could also be potentially useful for 
protecting stored messages as well, but here the message/rfc822 solution 
is not so useful (and BSMTP not appropriate at all).

For instance, an IMAP client may want to allow its users to encrypt all 
messages being archived on an IMAP server in order to protect the user 
against any server security event. It could do this by encapsulating 
existing messages in a message/rfc822 and holding the resulting 
encrypted message in a vanilla outer message header (e.g. no subject, 
all addressees set to the user themselves etc...).

However if this is done to lots of messages, the problem arises that the 
user won't be able to distinguish between messages without the client 
downloading and decrypting all messages (including bodies) first!

Two things seem to be needed to usefully support this sort of stored 
message encryption: the ability of a client to know that there are some 
replacement headers in an encrypted message and the ability of the 
client to decrypt those headers independently of the body.

This could be accomplished using an unencrypted multipart message, where 
the first part could hold the headers (text/rfc822-headers, encrypted 
using PGP/MIME or S/MIME) and the second part could hold the body (again 
encrypted). If a new type were defined (multipart/rfc822?), an IMAP 
client could recognize the top level Content-Type, download and decrypt 
the secure headers and thereafter display the decrypted header 
information to the user.

Just an idea, but without something like this, re-encrypting messages 
for storage involves either making the user enter alternative header 
details, or not securing the headers in the first place.

-- 
Ian Bell                                           T U R N P I K E  Ltd

---

From: [email-masked]
Date: Mon, 12 Jun 2000 10:15:49 +0900

From: [email-masked]
Subject: Re: confidential subject lines -> use content description field w/ pgp/mime?
Date: Thu, 08 Jun 2000 21:27:45 -0700 (PDT)
Message-ID: <[email-masked]>

...

> There are a bunch, although many of them are likely not in full compliance 
> with the RFC, which specifies support for certain aspects of ESMTP. 
> Remember, the entire BITNET network was implemented using BSMTP to get 
> around the 8X8 address limit of NJE. In its heyday BITNET amounted to 
> several thousand systems running on a variety of hardware and software 
> platforms.

i was an email-only user (i.e. not a user w/ administrative knowledge)
when i was using bitnet, so i cannot claim to have any in-depth
knowledge.  thanks for the explanation though.

> from our BITNET handling code. The same code is also in SIMS.

thank you very much for the pointers.

> interactive protocol into a batch protocol has some subtle spots to it.

in your opinion, are those subtle spots adequately and comprehensively
covered in the rfc?  i ask because if bsmtp encapsulation is to become
widely used (assuming here that it isn't at the moment), it seems to
me that there may have to be a few more implementations.

---

From: [email-masked]
Date: Mon, 12 Jun 2000 10:40:40 +0900

From: Ian Bell <[email-masked]>
Subject: Re: confidential subject lines -> use content description field w/          pgp/mime?
Date: Fri, 9 Jun 2000 13:55:44 +0100
Message-ID: <[email-masked]>

> is not so useful (and BSMTP not appropriate at all).

i used to think this might be a good idea, but i started thinking that
encrypting filesystems or directories where the messages are stored
(on the server end in your imap example) might be a better solution.

on the other hand, messages are sometimes stored w/in
databases...unless most databases support the notion of encrypting
records, i guess this could be problematic in that case, but may be
databases should support encryption/decryption.

> against any server security event. 

right, so you can protect against having the message contents revealed
and w/ signing against message content being tampered w/.  not to say
this idea isn't useful, but i don't know how to protect against
deletion.

> It could do this by encapsulating existing messages in a
> message/rfc822 and holding the resulting encrypted message in a
> vanilla outer message header (e.g. no subject, all addressees set to
> the user themselves etc...).
>

another approach:

encrypted messages a user receives could initially be decrypted on the
client end, modified to a decrypted form and placed again on the
server (whether they replace the original messages could be a policy
decision).

the server could implement confidentiality services through local
store (e.g. filesystem, directory, etc.) encryption -- a key for
decryption could be requested from the user during imap authentication
(or computed using authentication info).

all messages for a given user on the server end could then be
protected using this key.  before imap (or pop) processing occurs,
messages are decrypted.  when the processing is complete (e.g. imap
session ends), messages are encrypted.

---

From: [email-masked]
Date: Sun, 11 Jun 2000 19:26:42 -0700 (PDT)

> covered in the rfc?

Yes they are. Indeed, this was the entire point of writing the RFC -- to
explain how this can be done and what the issues are with doing it.

> i ask because if bsmtp encapsulation is to become

Frankly, I doubt if it will ever be widely used. But writing the document
seemed to be useful given that there have been a fair number of implementations
in the past and they didn't get it right without a specification.

				Ned

---

From: [email-masked]
Date: Fri, 16 Jun 2000 04:46:36 +0900

> existing PGP/MIME or S/MIME clients.

at this point, i'd like to see the methods mentioned/referred to in
some specification so i can point mail client developers at such a
document for the purposes of having them work on implementations that
will interoperate.

even w/ some kind of endorsement in the form of being present in a
specification, it may take a while for header (and/or envelope)
protection to be implemented at the user-level -- i.e. there is a single
command a user can use to create such header-protected messages.

so, any plans or takers on this point?