Subject: Review of ordering draft, version 05

From: "Lisa Dusseault" <[email-masked]>
Date: Sat, 8 Feb 2003 09:45:46 -0800

Since we're getting close to last call on the ordering draft, I reviewed
it all again (it's been years since I did so before).  Here are all my
comments.

Section 3:
"This document uses the terms "precondition" as "postcondition" as
   defined in [RFC3253]. "

Should be "and", not "as"

I don't think it's sufficient to just import the definitions of
precondition and postcondition. These are concepts that can clarify
what's going on, but unless it's accompanied by really plain English,
future readers of the specification won't be able to extract the
information they need. Long-timer WG participants may know the meaning,
but I'm learning more and more about people who never come to the
working groups, who may only read little pieces of our specifications
(e.g. a developer who doesn't usually do protocol implementation but is
assigned to fix a bug).  Server implementors won't bother verifying
whether some condition is one they have to enforce, or simply ignore it.
I strongly recommend including the standard MUST/MAY/SHOULD language
that specification readers are familiar with, particularly in the
postconditions which are not just error code definitions but also
specify behavior.

As an example, I'll jump ahead to the first precondition and
postcondition in section 5.1.  I'm not even sure what they mean, but
I'll try to guess in order to restate them. Here's what I'd like to see:

 Additional Preconditions:

      (DAV:ordered-collections-supported): Error used when the client
attempts to create an ordered collection, if the server does not support
the creation of ordered collections in this namespace.

  Additional Postconditions:

      (DAV:ordering-type-set): The server MUST set the ordering-type
property on the new resource to the value specified in the Ordering-Type
header by the client. If it cannot, this error should be used.

That's just a sample; all of the postconditions and preconditions need
to be full sentences explaining what must or may happen.

Section 5.1:
"If the Ordering-Type header is not present, the collection will be
unordered."

"Will be" is ambiguous standards language.  It implies MUST but doesn't
really require anything.  "... the server MUST create the collection as
unordered" is one possibility. However, it's fine to say "... the
collection MAY be created as unordered" if we want to give servers the
freedom to make new collections ordered by default.

Section 5.1: I don't see how the ordering-type-set postcondition can be
implemented. How can the server tell the difference between an ordering
type that it must support, or an ordering type that it's not supposed to
know about but just advertise?  
  Or is the server just supposed to set the value, period?  If that's
the case, then why isn't this simply a MUST requirement, rather than a
postcondition?
I suspect I don't really understand the ordering-type-set postcondition,
thus it probably just has to be explained more in the specification.

Section 6.1
The "segment" definition in BNF is imported from RFC2396.  But that
definition says that a segment can contain a "param":
	RFC2396
	  "segment       = *pchar *( ";" param )"

What would the param be or mean?  I suspect we'll have to define our own
here.

" The server MUST insert the new member into the ordering at the
      location specified in the Position header, if one is present (and
      if the collection is ordered)."

What does the "one" refer to in "if one is present"?  I think this means
"If the collection is ordered and a new member is inserted with the
Position header, the server MUST insert the new member into the ordering
at the location specified in the header."

" (DAV:collection-must-be-ordered): the target collection must be
      ordered."

Does this mean that the request to PUT a resource *fails* if the client
includes a Position header and the collection isn't ordered?  That's
expensive for a client to have fail.  I don't know if that's what client
implementors are going to want to happen.

"    (DAV:segment-must-identify-member): the referenced segment must
      identify a resource that exists and is different from the affected
      resource."

Again, does this mean the server MUST fail the PUT request if the
segment in the Position header doesn't exist?

"     (DAV:position-set): the newly created collection member was
      created at the specified position."

Shouldn't the server always create the new member at the specified
position (a MUST)? Under what circumstances would this ever not happen?

Specification organization: how about putting the Position and
Ordering-Type header definitions in their own sections, titled "Position
Header" and "Ordering-Type Header". That makes it easier for people to
use the specification as a reference.

Section 7.

"     <!ELEMENT orderpatch (ordering-type?, ordermember*) >"

This definition makes the orderpatch element (thus the ORDERPATCH
request body) non-extensible unless we say otherwise.  We could put in
the "elements not understood by the server MUST be ignored" language, or
redefine certain elements as

      <!ELEMENT orderpatch (ordering-type?, ordermember*, ANY) >

We need to say that the list of <ordermember> elements in the orderpatch
body is itself order-dependent -- I believe the server must process them
in the order they appear, first to last.  Actually, this is specified in
section 7.1 under the example, but it should be said in section 7.

"      (DAV:orderding-modified): if the request body contained
      DAV:ordermember elements, the ordering of internal member URIs in
      the collection identified by the request-URI has been changed
      based on instructions in the DAV:ordermember elements."

Isn't this a simple MUST rather than a postcondition?  "The server MUST
order internal member URIs in the collection identified by the
request-URI based on instructions in the DAV:ordermember elements in the
ORDERPATCH request body." Under what conditions would this
ordering-modified error be returned to the client?

Section 7.2.

This example shows what happens if the fourth <segment> element in the
request body does not name a valid segment.  By analogy, I can see what
would be returned if the second <segment> element were invalid. However,
I'm not clear what would happen if the first or third <segment> element
were invalid.  What would be returned in the <href> element in the error
response?

"Because ORDERPATCH is an atomic method, the request to
   reposition nunavut.desc (which would otherwise have succeeded) failed
   as well, but doesn't need to be expressed in the multistatus response
   body."

One of the things we've learned from RFC2518 interoperability is that's
not necessarily true. When a server does a MOVE on a collection, the
client has a hard time figuring out which things moved and which didn't.
Some responses have the failed dependency error, others don't.  Should
we consider being more specific here? The server could simply include a
424 Failed Dependency error for each ordering that failed because of the
atomicity of the method.

Section 9.1.

"The property DAV:version-controlled-binding-set ([RFC3253], section
   14.2.1) records the set of version-controlled bindings in the
   collection. For ordered collections, the DAV:version-controlled-
   binding elements MUST appear in the ordering defined for the checked-
   in ordered collection."

This is true only for working collections when versioned collections are
supported. At a minimum, change to:

"The property DAV:version-controlled-binding-set ([RFC3253], section
   14.2.1) records the set of version-controlled bindings in a
   working collection (part of the versioned collections feature). 
   For an ordered working collections, the 
   DAV:version-controlled-binding elements MUST appear in the ordering 
   defined for the associated checked-in ordered collection."

Doesn't that mean that the user can't reorder a collection that they've
checked out into a working collection?  

Section 10 (Discovery).

"  Furthermore, RFC 3253 [RFC3253] introduces the live properties
   DAV:supported-method-set (section 3.1.3) and DAV:supported-live-
   property-set (section 3.1.4). Servers MUST support these properties
   as defined in RFC 3253."

Do you mean that any server that supports this specification must
support these two RFC3253 properties as well? If so, that could be made
a little more clear like this:

"  Furthermore, RFC 3253 [RFC3253] introduces the live properties
   DAV:supported-method-set (section 3.1.3) and DAV:supported-live-
   property-set (section 3.1.4). Servers supporting ordered-collections
   MUST support these properties as defined in RFC 3253 even if the
   server does not support any RFC3253 features."

Section 16. Acknowledgements

"Lisa Lippert" was me, but my name is now "Lisa Dusseault" again...  I'd
much prefer to be listed as Dusseault!

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 8 Feb 2003 20:55:16 +0100

Lisa,

thanks for the review of the document. My comments are inline.

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Lisa Dusseault
> Sent: Saturday, February 08, 2003 6:46 PM
> To: 'Julian Reschke'
> Cc: Webdav WG
> Subject: Review of ordering draft, version 05
>
>
>
>

Fixed.

> whether some condition is one they have to enforce, or simply ignore it.

Well, the simple answer is, there *are* not pre/postconditions a server can
ignore. Quoting from RFC3253, section 1.6:

---> A "precondition" of a method describes the state of the server that
must be true for that method to be performed. A "postcondition" of a method
describes the state of the server that must be true after that method has
been completed. If a method precondition or postcondition for a request is
not satisfied, the response status of the request MUST be either 403
(Forbidden) if the request should not be repeated because it will always
fail, or 409 (Conflict) if it is expected that the user might be able to
resolve the conflict and resubmit the request.<---

> postconditions which are not just error code definitions but also

Actually the goal was to do that by normatively referring to RFC3253.

> to be full sentences explaining what must or may happen.

All of them are MUSTs. Thinking of it (and remembering the WG discussion),
the ACL spec may want to clarify that it's using the terminology in a way
that is not fully compatible to RFC3253.

> freedom to make new collections ordered by default.

That's a good question. Is there any specific behaviour we want to mandate?
Are there use cases where a server may want to default to a specific
ordering type when none was given?

> know about but just advertise?

Correct.

All we want to say that the ordering type was set as given:

(DAV:ordering-type-set): the collection was created with the specified
ordering type.

> the case, then why isn't this simply a MUST requirement, rather than a

Again, all conditons are MUSTs.

> What would the param be or mean?  I suspect we'll have to define our own

No, we don't. We just inherit. It means whatever RFC2396 wants it to mean.

> at the location specified in the header."

Rewritten as:

"When the Position header is present, the server MUST insert the new member
into the ordering at the specified location."

> implementors are going to want to happen.

It's what the spec has been saying for a long time.

I also don't see why that's a problem. It will only be a problem if

- the collection was ordered at time x
- a client does a PUT at time y and specifies a location header (if it
doesn't care, it can let it default to "last") *and*
- between x and y the collection was set to unordered by somebody else.

Seems like a case where I really *want* the request to fail. The client can
always use a sequence of PUT (without Position) and ORDERPATCH if this is
really an issue.

> segment in the Position header doesn't exist?

Yes.

> position (a MUST)? Under what circumstances would this ever not happen?

Yes, it's a MUST.

> use the specification as a reference.

However it also leads to text duplication. People can use both the TOC and
the index to locate the relevant section.

> the "elements not understood by the server MUST be ignored" language, or

No, it doesn't. It uses DTD fragments in *exactly* the same way as RFC2518
and RFC3253, that is the standard WebDAV extensibility rules apply.

> section 7.1 under the example, but it should be said in section 7.

Correct.

Section 7 changed to say: "The ordering of internal member URIs in the
collection identified by the Request-URI is changed based on instructions in
the order-member XML elements in the order they appear in the request."

Also the explanations for example 7.1 were still assuming the old "href"
based syntax, I've updated that as well.

> ordering-modified error be returned to the client?

That would be a server error (it's MUST).

> were invalid.  What would be returned in the <href> element in the error

The URI that identifies the member that could not be repositioned. As per

"Since multiple changes can be requested in a single ORDERPATCH request, if
any problems are encountered, the server MUST return a 207 (Multi-Status)
response (defined in [RFC2518]), containing DAV:response elements for either
the request-URI (when the DAV:ordering-type could not be modified) or URIs
of collection members to be repositioned (when an individual positioning
request expressed as DAV:order-member could not be fulfilled)."

> client has a hard time figuring out which things moved and which didn't.

That's because MOVE in RFC2518 is not atomic.

> 424 Failed Dependency error for each ordering that failed because of the

How would that help? A client can rely on the request being atomic.

> supported. At a minimum, change to:

The property DAV:version-controlled-binding-set is only defined for
collection versions. So yes, this only applies to servers that support
versioned collections.

> ...

Why would it mean that? A working collection doesn't have that property.

What indeed is missing here is some words about how the ordering in
DAV:version-controlled-binding-set affects UPDATE and CHECKOUT of collection
versions.

> support these two RFC3253 properties as well? If so, that could be made

Yes.

>    server does not support any RFC3253 features."

Well, in that case it supports RFC3253 features (namely these two live
properties). I'd say that normatively referring to a specific part of
another RFC should really be clear enough.

> much prefer to be listed as Dusseault!

Done.

Again,

thanks for the review!

Julian

--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 13 Feb 2003 21:25:14 +0100

Hi.

I have adressed some of Lisa's issues, in particular:

- how to interpret the DTD fragments [1]
- reference to RFC3253 pre/postcondition definition [2]

I've also tried to clarify the relation to versioned collection ([3]).

To those who revieved the 05 draft: please check out the current version
(all changes are highlighted) -- I intend to submit this draft as "last
call" version early next week.

Julian

[1]
<http://greenbytes.de/tech/webdav/draft-ietf-webdav-ordering-protocol-latest
.html#rfc.section.1>
[2]
<http://greenbytes.de/tech/webdav/draft-ietf-webdav-ordering-protocol-latest
.html#rfc.section.3>
[3]
<http://greenbytes.de/tech/webdav/draft-ietf-webdav-ordering-protocol-latest
.html#rfc.section.9>

> -----Original Message-----

---

From: "Lisa Dusseault" <[email-masked]>
Date: Thu, 13 Feb 2003 17:24:12 -0800

I still feel strongly that postconditions (and ideally also
preconditions) should be explained as full English sentences

For example, replace

  "(DAV:ordering-type-set): the collection was created with the
specified ordering type. "

With

  "(DAV:ordering-type-set): The server MUST set the ordering-type
  property on the new resource to the value specified in the
Ordering-Type
  header by the client. If it cannot, this error should be used."

I will propose specific text for every postcondition if necessary.

Lisa

> [mailto:[email-masked]] On Behalf Of Julian Reschke
> Sent: Thursday, February 13, 2003 12:25 PM
> To: Webdav WG
> Subject: RE: Review of ordering draft, version 05
> 
> 
> 
> Hi.
>

---

From: "Julian Reschke" <[email-masked]>
Date: Fri, 14 Feb 2003 10:30:17 +0100

> Sent: Friday, February 14, 2003 2:24 AM
> To: 'Julian Reschke'; 'Webdav WG'

(that *is* a full sentence, isn't it?)

>   header by the client. If it cannot, this error should be used."

I agree that the condition itself can be expressed in a more stringent
manner. However, I strongly disagree with the second sentence: all
pre/postconditions are MUSTs (by definition), and therefore the statement is
(a) redundant and (b) - for consistency - it would have to be repeated on
each an every condition.

Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Fri, 14 Feb 2003 09:25:09 -0800

> (that *is* a full sentence, isn't it?)

My suggestion isn't really about grammar, it's about readability.  This
is a passive phrase, which requires the reader to have a lot of context
and experience to understand what behavior is required of whom.  

> (a) redundant and (b) - for consistency - it would have to be 

Yes, I agree that consistency helps make a spec readable.  What I'm
proposing is to put a phrase such as "the server MUST..." on each and
every condition. I will propose specific sentences if necessary, but I'm
not sure I even understand each postcondition (which is itself clearly a
problem).

The reason I suggest this redundancy is because I talk to a lot of
people who implement WebDAV yet who haven't read every word of every
spec.  I remember my first job working on WebDAV in 1998 and I just
browsed through the WebDAV and HTTP specs lightly. Then when a developer
would ask me a specific question about something like how range headers
worked, I'd look up that specific section. Usually, I could find the
answer without having to integrate massive amounts of text from various
sources, because in fact RFC2616 and RFC2518 are fairly well-written. 

Since this is the way specifications are used, a well-written
specification *does* have a lot of redundancy. In writing essays,
students are told to say what you're going to say, say it, and then say
what you said. A specification is not a piece of code, to be executed by
a computer which can load every line into memory and compile before
executing. It's a puzzle that's too big to be seen at once, so instead
each reader picks up a small piece of the puzzle at a time and tries to
understand some of the picture just from that piece. A pragmatic
approach to writing a successful specification takes that into account
and provides enough context in each puzzle piece so that piece makes
sense.

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 15 Feb 2003 01:05:21 +0100

> From: Lisa Dusseault [mailto:[email-masked]]
> Sent: Friday, February 14, 2003 6:25 PM

I will look into rewriting the condition explanations. If there's a specific
one which you currently don't understand (after having clarified that all of
them are MUSTs), I would be interested to hear which.

> and provides enough context in each puzzle piece so that piece makes

All true, but...

On the other hand, a spec must be precise. RFC2518 currently is very
verbose, but neglects to precisely define everything. For example: the
various PROPFIND types are mainly documented by example. A similiar problem
exists for the definition of error handling. So I really don't accept the
current version of RFC2518 as good example.

RFC3253 on the other hand is very precise, but definitively lacks examples
(the price for adding examples would have been a much much bigger spec,
which is a problem in itself).

The current ordering draft tries to adopt the more precise approach of
RFC3253. In particular, by inheriting from RFC3253 it gets

- (IMHO) a better error marshalling as RFC2518,
- precise definitions of what must happen upon executing a method
(pre/postconditions).

In addition, the draft tries to give more examples than RFC3253 does. In
this particular case, this can easily be afforded because the spec is so
much smaller.

So, the ordering spec tries to use the same approach as RFC3253, ACL and the
bindings spec. If there's something fundamentelly wrong with this approach,
it also applies to these other RFCs and drafts, and I'd like to see a more
general discussion about that.

Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Fri, 14 Feb 2003 16:28:26 -0800

> - precise definitions of what must happen upon executing a method

Actually, I agree that RFC2518 has a better error marshalling mechanism
*and* that precise definitions of what must happen upon executing a
method are all good things. I think the concept of preconditions and
postconditions is a good concept and helps make a specification more
complete.

But, here's what I definitely don't want to *lose* in the process.
 - the ability of a specification reader to digest a piece of the
specification, and basically "get it"
 - the clarity of the standard MUST, MAY, SHOULD language
I think we're in agreement on this now and I look forward to seeing your
revised language-- I will review it all to see if it all makes sense to
me then.

A couple more subtle topics now...

 1. I'm a little worried that we're starting to lose the richness of
having error codes from 100 to 999 available by using only 403/409.  For
example, when ordered collections aren't supported in a given location,
why aren't we using 501?  If a segment listed in the ordering is not
found, why would the server return 409 with
(DAV:segment-must-identify-member) rather than a 404 Not Found response
code (either on its own or in a body)?

 2. Are we possibly making things confusing by introducing a
postcondition "code" like (DAV:ordering-type-set) for something that
MUST be done?  It makes it seem like the server might choose not to set
the ordering type, create the collection anyway, and respond with 409
containing <DAV:ordering-type-set/>.  Wouldn't a total failure to do
some required behavior normally be defined as a 500 response?

Thanks,
Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Sat, 15 Feb 2003 10:40:54 +0100

> Sent: Saturday, February 15, 2003 1:28 AM

I assume that's a typo :-)

> revised language-- I will review it all to see if it all makes sense to

OK.

> why aren't we using 501?  If a segment listed in the ordering is not

Because 501 means:

	"The server does not support the functionality required to fulfill the
request. This is the appropriate response when the server does not recognize
the request method and is not capable of supporting it for any resource."

In general I'm sympathic to allow DAV:error response bodies for *any* failed
request (I think I actually proposed this a while back on this list).

> code (either on its own or in a body)?

We can't simple return a 404 because the Request-URI identified a collection
which in fact *does* exist. The request failed because we tried to
manipulate the state of the collection in a way that failed because one of
the bindings we tried to position did not exist. We did *not* try to
manipulate the resource identified by that internal member name at all.

> some required behavior normally be defined as a 500 response?

Yes, and I think that's something that should be possibly fixed in RFC3253.
Failure to meet a postcondition (after all preconditions were verified)
always is a server bug and thus would belong into the 5xx range. The main
question is: which spec should fix that? I'd really like to see RFC2518bis
to pick up (and clarify) this kind of error typing (maybe in a way that
makes it optional?).

Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Sat, 15 Feb 2003 10:09:18 -0800

> I assume that's a typo :-)

Yeah, oops, that was a typo :)

> to pick up (and clarify) this kind of error typing (maybe in 

Seems reasonable to me, as long as it is optional. Other opinions?

---

From: "Julian Reschke" <[email-masked]>
Date: Mon, 17 Feb 2003 14:02:02 +0100

Hi.

I have tried to clarify the postconditon descriptions in the current
"latest" draft ([1]). Those who already reviewed it and found them to be
inprecise may want to review the changes.

Julian

[1]
.html>

> Sent: Saturday, February 15, 2003 7:09 PM
> To: 'Julian Reschke'; Webdav WG