Subject: Re: [saag] I-D on PKCS#11 URI Scheme for naming objects

From: Stef Walter <[email-masked]>
Date: Mon, 02 Aug 2010 17:44:42 +0200

On 08/02/2010 04:04 PM, Jan Pechanec wrote:
> 	I'm also cc'ing Nikos who raised the similar question on the 
> encoding just a few days ago.
> 
>> I work on gnome-keyring which is based around PKCS#11. We're working on
>> an effort to use PKCS#11 as a common key and certificate storage
>> standard for the GNOME Desktop.
>>
>> I'm interested in the PKCS#11 URI [1] RFC that you proposed. However,
>> several questions come up about encoding.
> 
> 	I'm very glad that you might be interested in using this. I hope 
> we can agree on a common specification.

Yes that would be great.

> 	the BNF code in the I-D uses UTF-8 for most of the attribute 
> values so all other characters were supposed to stand for themselves. 
> However, I think that using the percent encoding is better, please see 
> below.

Cool.

>> Secondly, would it perhaps make sense to use percent encoding for the
>> id= attribute? PKCS#11 URIs as proposed are introducing a colon style
>> encoding. I noticed it's not defined whether there may be one hex
>> character between colons or not. Percent encoding is a well defined
>> encoding which removes such ambiguities.
> 
> 	it seems to me it would be logical to use the percent encoding 
> for the 'id' attribute as well.
> 
> 	there is a minor issue though. From RFC 2068 I can see that in 
> the HTTP url, '%' can not stand for itself as $ in shell can, for 
> example. So, "http://www.abc.com/page%" is not a correct URL. What do we 
> do here?
> 
> 	we have two options. (a) We can say that in those attributes 
> with UTF-8 character set '%' is supposed to be an escape if followed by 
> two hexa characters. That way I could just reference 'UTF8-octets' 
> defined in RFC3629. It would mean that:
> 
> 	pkcs11:object=abc%
> 
> 	would constitute a valid PKCS#11 URI. An application would have 
> to choose what to do with that. For example, Firefox even accepts 
> "http://www.abc.com/page%" but Apache returns BAD_REQUEST no matter that 
> there is a page literally named "page%".
> 
> 	alternatively, (b) we could exclude such URIs using the BNF code 
> and give '%' a special meaning. I'd have to do something similar to what 
> I do now - put the UTF-8 definition there and modify it.

I'd propose the third option (c) that a percent character on its own is
invalid, and that percent characters must be encoded using percent
encoding. This would mirror the above apache behavior. This is far
easier and consistent for an application or library to implement.
Perhaps this is similar to what you meant by option (a) but is further
defined.

> 	I assume that %hh would be allowed only in values so if there is 
> literal ';' it is an attribute delimiter. 

Yes correct.

We could also mention that
> using the percent encoding for anything else than ';' and possibly '%' 
> is optional, mostly based on what other characters need to be escaped 
> from the system using the URI instances.

What about spaces and new lines? It seems to me that there are many
characters which would require encoding. Or am I misunderstanding?

> 	as mentioned, I'd also replace the current ck_id encoding with 
> the %hh encoding.

Perfect.

I'm currently working on implementing this in gnome-keyring, and
hopefully will have some code to try out PKCS#11 URIs soon. Another
question to follow in another thread...

Thanks again.

Stef

---

From: Stef Walter <[email-masked]>
Date: Mon, 02 Aug 2010 20:45:31 +0200

On 08/02/2010 06:03 PM, Jan Pechanec wrote:
> On Mon, 2 Aug 2010, Stef Walter wrote:
> 	that would be fine with me but if we say that a percent 
> character MUST NOT be present in the URI, the UTF-8 BNF definition must 
> be copied over, changed and put it into the I-D. That's how the BNF code 
> is there now and it brings more complexity without really giving any 
> real benefit. Given that it's encoding of the attribute value, I'm 
> wondering if we could get away with just stating that the percent 
> encoding SHOULD be used for ';', '%' and other characters like new lines 
> or spaces but the final value interpretation is up to the consumer.

It seems to me that the complexity in the BNF definition is traded for
complexity in a conforming URI parser.

RFC 3986 does indicate that percent should not be allowed unless percent
encoded as %25. [1]

My opinion on this matter is not a strong one, but I had planned to use
url unescape functions available in glib which expect properly encoded
URI segments, and as such reject raw percent signs.

> 	strictly speaking, we should also limit what %hh could be used 
> for. It should not be anything above 128 otherwise it could corrupt the 
> UTF-8 encoding. That's why I'd like to avoid this in the BNF code.

Well, in cases of escaped CKA_ID it would be any arbitrary byte value
from 0 - 256.

In addition UTF-8 characters are typically encoded one byte at a time in
URIs. When encoded as percent encoding, the french word 'défaut' would
be encoded as 'd%C3%A9faut'. More info toward the end of section 2.5 in
RFC 3986 [2].

Therefore accepting percent encoded characters higher than the byte 128
is necessary. Such URIs would be produced by the GNOME implementation.

> 	no, just those characters do not have to be escaped. In shell, 
> we can put a newline into the string easily but it could be used. Of 
> course, they MAY be escaped.

Okay, so this is different than RFC 3986, where whitespace (unless
percent encoded) is to be ignored by a parser. That said, in practice
many (most?) URI parsers preserve whitespace, so I agree that we can be
more lenient in the encoding of newlines or spaces.

>>> 	as mentioned, I'd also replace the current ck_id encoding with 

FWIW, the hex encoding of a URI is defined as upper case in RFC 3986 [3].

I realize I keep pointing to RFC 3986, but that's because we're calling
it a 'URI'. If we choose to differ from the definition of URI in the
above aspects, I think it would be important to note these differences
in the PKCS#11 URI specification.

Cheers,

Stef

[1] http://tools.ietf.org/html/rfc3986#section-2.4

[2] http://tools.ietf.org/html/rfc3986#section-2.5

[3] http://tools.ietf.org/html/rfc3986#section-2.1

---

From: Stef Walter <[email-masked]>
Date: Wed, 04 Aug 2010 08:25:38 +0200

>> Jan Pechanec wrote:

I've been thinking about the CKA_ID encoding further, and because it's
arbitrary binary data percent encoding may not be the best fit... I
realize I suggested percent encoding, and perhaps percent encoding could
work, but it may be out of the ordinary to use percent encoding for
predominantly binary data.

For example the data: URI uses base64 encoding instead. It seems that a
straight forward hex encoding or base64 encoding is more common to use
in a URI when raw binary data is encoded.

But I'm fine with any of these approaches to encoding the CKA_ID.
Technically they would all work.

Cheers,

Stef

---

From: Jan Pechanec <[email-masked]>
Date: Mon, 02 Aug 2010 16:04:38 +0200 (CEST)

On Sun, 1 Aug 2010, Stef Walter wrote:

	hi Stef,

>The standard states that semi-colons and backslashes should be escaped
>with back slashes. That fixes the problem of allowing semi-colons to be
>specified in object=xxx labels.
>
>However if there are spaces, tabs, or other special characters in the
>PKCS#11 URI should these be percent encoded? This is not made clear in
>the spec, but perhaps it's an assumption because of the reference to
>other URI RFCs.

>That said, one implementation I found in OpenSolaris libcryptoutil does
>not handle percent encoding or even the backslash style escaping [2].

	correct. It was a dependency for another project I've been 
working on so I just wrote the code in a way that was enough for the URI 
use in Solaris.

>I question the need for backslash style escaping of semi-colon
>suggested in the PKCS#11 URI proposal. In my opinion PKCS#11 URIs should
>just use percent encoding for semi-colons, similar to how you would
>percent encode a '?' character or space in a http: URL.

	I agree.

	pkcs11:object=abc%

	I prefer (a). We can clearify that if % is not followed by two 
hexa characters it's up to the consumer of the URI to decide what to do. 
We do not have to say more since in UTF-8, any character <=128 stands 
for itself so the consumer would be expected to reparse the attribute 
values and replace %hh with the correct characters and possibly leave 
values ending with sequences like '%h' as they are (or return an error).

literal ';' it is an attribute delimiter. We could also mention that 

	what do you think?

	thanks for the feedback, J.

>Looking forward to your comments.
>
>Cheers,
>
>Stef
>
>
>[1] http://tools.ietf.org/html/draft-pechanec-pkcs11uri-01
>
>[2]
>http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/lib/libcryptoutil/common/pkcs11_uri.c#126
>

-- 
Jan Pechanec
http://blogs.sun.com/janp

---

From: Jan Pechanec <[email-masked]>
Date: Mon, 02 Aug 2010 18:03:18 +0200 (CEST)

given that "http://www.abc.com/page%" is an invalid http URL I'd 
expect Firefox to refuse it right away. That's why I don't think we need 
to do that in the BNF code.

	"Anything with an ASCII code 1-128 MAY be escaped using the % 
character."

>question to follow in another thread...

	cool. Cheers, J.

---

From: Jan Pechanec <[email-masked]>
Date: Tue, 03 Aug 2010 02:51:29 +0200 (CEST)

hi Stef,

>URI segments, and as such reject raw percent signs.

	they must be able to return an error on an improperly encoded 
string :-) Anyway, I understand the problem, it's when to catch the 
errors. The closer to the user, the better. I'll have a look what would 
the BNF code look like.

>Well, in cases of escaped CKA_ID it would be any arbitrary byte value

	sure, CKA_ID is easy.

>be encoded as 'd%C3%A9faut'. More info toward the end of section 2.5 in

	aha, got it. The percent encoding would be representing the 
UTF-8 encoding so any UTF-8 character above 128 if represented by the 
percent encoding would be encoded in at least two %hh sequences. That's 
fine with me.

>more lenient in the encoding of newlines or spaces.

	I think I have to take a closer look at the RFC but I think we 
should allow spaces as spaces.

>FWIW, the hex encoding of a URI is defined as upper case in RFC 3986 [3].

	ok

>in the PKCS#11 URI specification.

	I think we should follow RFC 3986 as much as possible, I'll read 
through it properly and get back to you.

	thanks, J.

---

From: Jan Pechanec <[email-masked]>
Date: Tue, 10 Aug 2010 15:36:12 +0200 (CEST)

hi Stef,

	I'm fine with all the suggestions and would like to generate a 
new draft that I'd sent here before filing it. I'm not sure about one 
thing though - the space. I hope we can use it literarily in the PKCS#11 
URI. However, you mention it should be ignored by the parser unless 
encoded as %20.

	when reading RFC 3968, it's not in reserverd characters (section 
2.2), it's not in unreserved (2.3). I found this:

Appendix C. Delimiting a URI in Context

In some cases, extra whitespace (spaces, line-breaks, tabs, etc.) may 
have to be added to break a long URI across lines. The whitespace should 
be ignored when the URI is extracted.

<snip>

Using <> angle brackets around each URI is especially recommended as a 
delimiting style for a reference that contains embedded whitespace.

	does "embedded whitespace" means a space that is part of the 
attribute value? BTW, Firefox allows to use a literal space in URL. It 
encodes all spaces in the HTTP request though.

	given that the space is not in unreserved characters, I tend to 
think it must be used as %20...

	cheers, J.

---

From: Jan Pechanec <[email-masked]>
Date: Tue, 10 Aug 2010 15:37:52 +0200 (CEST)

>Technically they would all work.

	hi Stef, I'd prefer the percent encoding since that can be done 
by hand. One needs another tool to encode data into the base64 form.

	J.