Subject: [core] block transfer cancellation

From: "Peter A. Bigot" <[email-masked]>
Date: Sat, 19 Oct 2013 22:13:37 -0500

If an endpoint involved in a block transfer as sender or receiver needs
to cancel the transmission, how can this be done?

Consider a request with a block-transmitted payload, during transfer of
which the client determines that the request needs to be aborted.  The
server has already received part of the information.  The client can
simply fail to transfer the remainder, but because each message-level
fragment was completed no existing timeouts apply to tell the server to
give up expecting any of the remainder.  When can the server discard
pending material?

 From the other direction, if the sender is using confirmable messages
the recipient may reject one of the blocks.  I can't find text in
block-12 that requires the sender to abort transmission of the remainder
of the blocks.

The same issue exists for responses, assuming server-initiated is not
completely removed as the resolution of #253.  (I found Matthias' recent
comments on this, but no discussion with a clear description of what
will be incorporated into the rewrite).

Note that only for observe notifications is it required that block
transfers be confirmable, so if a receiver rejects a non-confirmable
block fragment there is no obligation for the server to react (following
observe where only rejecting a confirmable notification is guaranteed
to stop future notifications).

Peter

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 20 Oct 2013 10:55:09 +0200

On Oct 20, 2013, at 05:13, "Peter A. Bigot" <[email-masked]> wrote:

> (I found Matthias' recent

Indeed, this is difficult to discuss further without text to look at.
I have submitted -13 so we have a common reference of how these comments will be resolved.

Grüße, Carsten

---

From: "Peter A. Bigot" <[email-masked]>
Date: Sun, 20 Oct 2013 12:44:33 -0500

> I have submitted -13 so we have a common reference of how these comments will be resolved.

Thanks; this does appear to remove server initiated transfers, so part
of one of my questions is moot, but the bulk of the issues I raised are
still open for discussion.

Specific comments:

block-13 section 2.5 has:

    ...  (Note that a 4.13 response to a
    request that does not employ Block1 is a hint for the client to try
    sending Block1, and a 4.13 response with a smaller SZX in its Block1
    option than requested is a hint to try a smaller SZX.)

This implies Block1 may appear in a 4.13 response, which I don't think
is the intent.  I think it's meant to be more like:

    ...  (Note that a 4.13 response to a
    request that does not employ Block1 is a hint for the client to try
    sending Block1.  As described in [I-D.ietf-core-coap] the 4.13
    response SHOULD include a Size1 option to help the client select
    an acceptable SZX for the Block1 option.)

Two issues with 2.7 which has:

    continues to send requests similar to the requests in the Block1
    phase, bute leaves out the Block1 options and includes a Block2
    request option with non-zero NUM.

s/bute/but/

    Block2 transfers that retrieve the response body for a request that
    used Block1 MUST be performed in sequential order.

Why is this requirement restricted to responses where the request used
Block1?  Is it reasonable to allow non-sequential order if the request
did not carry a large representation, but the response does?  Perhaps:

    Block2 transfers that retrieve the response body MUST be performed
    in sequential order.

and move it to the discussion of Block2 rather than the section that
discusses combining Block1 with Block2.

Or, if the intent is to allow non-sequential order, is the server
required to support such requests, or does it need a 4.09 Response
Entity Incomplete to tell the client it insists on in-order transfer?
(I'm thinking here of a case where it's easy to incrementally produce
the response representation, but expensive to randomly access pieces of
it, e.g. if it's compressed or encrypted.)

A general comment:

As discussed in an earlier email, [I-D.ietf-core-coap] generically
imposes strict requirements on which options may appear in which
requests and responses, though it does not rigorously define the
specifics.  block-13 appears to define two new response codes: 2.31
(Continue) and 4.08 (Request Entity Incomplete).  The introduction of
these new codes is deep within 2.5 "Using the Block1 Option", and their
introduction is not apparent from the table of contents.

It would be helpful to future readers of CoAP extension RFCs if each new
option, request method, and response code was documented in a subsection
of its own, paralleling the structure of [I-D.ietf-core-coap], and
providing complete information regarding appearance in requests and
responses, repeatability, etc.  This should be done so that the
introduction of these new options/codes is visible in the index, and so
an implementer can locate the relevant information without having to do
a text search through the document.  Something like:

2. Options
2.1. Block1
2.2. Block2
2.3. Size1
2.4. Size2
2.5 ... interactions
3. Response Code Definitions
3.1. 2.31 Continue
3.2. 4.08 Request Entity Incomplete

I understand this would require some significant document restructuring
so it may be too late for block.

Peter

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 21 Oct 2013 00:22:48 +0200

Hi Peter,

thanks for the quick review of -13.

> is the intent.  

That actually *is* the intent ("control usage").
(This is about SZX, not total size of the request body, which would be indicated in the Size1 option.)

(Thanks for the editorial fix.)

>   Block2 transfers that retrieve the response body MUST be performed

All Block2 transfers retrieve the response body.
There are good reasons to leave out the MUST for Block2/GET.
I do think it is appropriate to extend the MUST for Block2/all requests that involve request bodies.

> it, e.g. if it's compressed or encrypted.)

Good point; we don't have a defined error code for that yet.
While the server can always send a 4.00, something more specific would be useful.
(Similarly, we don't have a good way to indicate that the client's block number has overshot the size of the representation.
This can happen even with sequential access in case of dynamically changing representations.)

We need to work on the Block2 error codes.

> (Continue) and 4.08 (Request Entity Incomplete).  

Yes, that list is in section 6.  See below.

> The introduction of

That doesn't work very well, because you can't define the meaning of the options without mentioning response codes.

I don't mind restructuring the text at all (even though I'm not sure after the recent simplifications that we still need the "grand rewrite") but it should not make the text harder to read for the sole purpose of making the table of contents more immediately accessible -- most people do know how to search in documents at this point.

But I do like the idea of giving the response codes a subsection of their own.
Also, we can make the table of contents more directly useful:

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Block-wise transfers  . . . . . . . . . . . . . . . . . . . .   4
     2.1.  The Block2 and Block1 Options . . . . . . . . . . . . . .   5
     2.2.  Structure of a Block Option . . . . . . . . . . . . . . .   6
     2.3.  Block Options in Requests and Responses . . . . . . . . .   8
     2.4.  Using the Block2 Option . . . . . . . . . . . . . . . . .   9
     2.5.  Using the Block1 Option . . . . . . . . . . . . . . . . .  11
     2.6.  Combining Blockwise Transfers with the Observe Option . .  12
     2.7.  Combining Block1 and Block2 . . . . . . . . . . . . . . .  13
     2.8.  Combining Block2 with Multicast . . . . . . . . . . . . .  13
     2.9.  Additional Response Codes . . . . . . . . . . . . . . . .  13
       2.9.1.  2.31 Continue . . . . . . . . . . . . . . . . . . . .  13
       2.9.2.  4.08 Request Entity Incomplete  . . . . . . . . . . .  13
   3.  Examples  . . . . . . . . . . . . . . . . . . . . . . . . . .  14
     3.1.  Block2 Examples . . . . . . . . . . . . . . . . . . . . .  14
     3.2.  Block1 Examples . . . . . . . . . . . . . . . . . . . . .  17
     3.3.  Combining Block1 and Block2 . . . . . . . . . . . . . . .  19
     3.4.  Combining Observe and Block2  . . . . . . . . . . . . . .  20
   4.  The Size2 and Size1 Options . . . . . . . . . . . . . . . . .  23
   5.  HTTP Mapping Considerations . . . . . . . . . . . . . . . . .  24
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  26
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  27
     7.1.  Mitigating Resource Exhaustion Attacks  . . . . . . . . .  27
     7.2.  Mitigating Amplification Attacks  . . . . . . . . . . . .  28
   8.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  28
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  29
     9.1.  Normative References  . . . . . . . . . . . . . . . . . .  29
     9.2.  Informative References  . . . . . . . . . . . . . . . . .  29
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  29

I propose adding a section 2.9 that reads like this:

2.9.  Additional Response Codes

   Two response codes are defined by this specification beyond those
   already defined in [I-D.ietf-core-coap].

2.9.1.  2.31 Continue

   This success status code indicates that the transfer of this block of
   the request body was successful and that the serve encourages sending
   further blocks, but that a final outcome of the whole block-wise
   request cannot yet be determined.  No payload is returned with this
   response code.

2.9.2.  4.08 Request Entity Incomplete

   This client error status code indicates that the server has not
   received the blocks of the request body that it needs to proceed.
   The client has not sent all blocks, not sent them in the order
   required by the server, or has sent them long enough ago that the
   server has already discarded them.

Grüße, Carsten

---

From: "Peter A. Bigot" <[email-masked]>
Date: Sun, 20 Oct 2013 21:27:25 -0500

> (This is about SZX, not total size of the request body, which would be indicated in the Size1 option.)

That makes sense.  I'm still hung up by the very clear requirements in 
coap-18 section 5.4 that options not explicitly defined for response 
codes are disallowed.

So this means Block1 is defined for 4.13 as a mechanism to convey to the 
client the (maximum?) SZX value the server would accept in another 
request.  And that only the SZX portion of the option value is defined 
and the M and NUM fields should be zero, perhaps.

> All Block2 transfers retrieve the response body.

Indeed; again Response-Block as the option name might reduce confusion.

> There are good reasons to leave out the MUST for Block2/GET.

An informational note explaining what those might be would be helpful, 
e.g. if the intent is to allow access to subresource content (not just 
out-of-order, but selective retrieval).  It should also make clear 
whether the recipient is required to support this practice, and if it 
does not how it informs the sender.

> I do think it is appropriate to extend the MUST for Block2/all requests that involve request bodies.

I don't understand this statement unless you mean the text as written is 
what you intended.  It would be worth documenting why this is an 
appropriate restriction that is not appropriate when request bodies 
aren't transferred in blocks.

In general one thing that frequently throws me in these documents is 
where a MUST/SHOULD statement is made for a special case but the 
expected or permitted behavior in the general case, or parallel special 
cases, lacks similar clarity.  A clear general rule of "Block transfers 
MUST be made in sequential order" or "Block transfers MAY be made in 
arbitrary and incomplete order" (whichever it is), applying to both 
Block1 and Block2, would be helpful.  Then follow that with deviations 
from the special rule (perhaps noting explicitly what they deviate from 
and the reasoning that supports the deviation).

> We need to work on the Block2 error codes.

I hope they could be rolled up into one or two 4.xx codes, e.g. 4.08 
Block Violation with a diagnostic payload as explanation.  There are 
only 32 codes possible and spending more than one or two on diagnostics 
on any particular feature is going to use them up quickly.

Just throwing this out: maybe an error code "Option Use Violation". 
Could support a payload holding the encoded option(s) that were not 
acceptable, though this would have to have a content format and 
therefore could not be a diagnostic payload.  The bad options could be 
transferred in an opaque BadOption option instead but that's getting 
pretty meta.

>     Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  29

This would go a long way to making things easier for the way I use IETF 
documents, which is to navigate to the RFC's HTML tools page and follow 
links from the index to the details of what I'm trying to clarify.  I 
also like to have links to the standard definition in API documentation, 
which also works if there's an index entry.

>     server has already discarded them.

Ignoring whether 4.08 should be generalized to cover other block 
protocol violations, I like this.  Per coap-18 section 5.4 I'd like it 
even more if 2.9.1 and 2.9.2 were explicit about which of Size1, Size2, 
Block1, and Block2 are defined for each response.  (I believe the 
proposed text is adequately explicit for coap-18's requirement in 
section 5.5 about whether payloads are permitted, since 4.xx has a 
general rule allowing diagnostic payloads.)

Peter