Subject: [core] WGLC observe comment: notification supersedure

From: "Peter A. Bigot" <[email-masked]>
Date: Wed, 16 Oct 2013 12:54:38 -0500

Although I raised this before, for WGLC I'm hoping for comments from
others.  One reviewer "dislike" and one author "like" is not much
of a consensus.

observe-11 section 4.5 adds a requirement that if the state of an
observed resource changes while a confirmable message has not been
acknowledged, the server MUST replace the content of the message for any
remaining retransmissions (superseding the previous message).

I believe this is an inappropriate expansion of functionality by a CoAP
protocol extension.  Here goes the argument:

CoAP (per coap-18) specifies the behavior of the protocol with respect
to message-layer (CON/NON/ACK/RST) and transaction-layer
(Request/Response) behavior.  The process for a confirmable message
transmission is well-defined, including a limited number of BEBO (binary
exponential back off) retransmissions.

coap-18 (section 4.2) permits a sender to stop retransmitting a
confirmable message in certain circumstances.

In section 5.10.5 coap-18 permits, but does not require, a sender to
modify an option value prior to retransmission if the message contains a
representation of a resource with strict tolerances on Max-Age.

Overall coap-18 describes how protocol extensions may specify new
options, as long as they obey critical/unsafe/no-cache-key standards,
and may define new request methods and response codes, so an implementer
can take that into account.

But there is nothing in base CoAP that suggests that other options or
content of a message may be changed prior to retransmission, or that
right-to-retransmit may be "donated" for the purposes of sending a new
message earlier than would normally be permitted by congestion control
(viz., upon acknowledgement or elapse of EXCHANGE_LIFETIME).

To meet the MUST requirement proposed by observe-11 4.5 an implementer
would have to provide an interface to the message layer that provides
supersedure functionality that is not envisioned by coap-18.

I believe it is inappropriate for an extension to require an implementer
to change the documented message-layer behavior of the base protocol in
order to support that extension.

To fix this I propose: Change step 3 in section 4.5 to something like:

    3.  If the entry is still in the list of observers, the server MAY
        replace the message with a new notification with a representation
        of the current resource state.  Should the resource have changed
        its state more than once in the meantime, the notifications for
        the intermediate states are silently skipped.

This does not obligate the implementation to support message
supersedure, as the current text does even if updates have no "strict 
tolerance" for timeliness for a particular resource.

It keeps the expectation that if a client fails to acknowledge a 
confirmable notification after BEBO completes all transmissions the 
client is removed from the list of observers.

And it allows the supersedure if the implementation chooses to support it.

Everybody wins?

Peter

---

From: Zach Shelby <[email-masked]>
Date: Fri, 18 Oct 2013 14:41:37 -0700

Peter,

I think you're making a completely reasonable suggestion here. +1.

Zach

On Oct 16, 2013, at 10:54 AM, Peter A. Bigot <[email-masked]> wrote:

> supersedure, as the current text does even if updates have no "strict tolerance" for timeliness for a particular resource.
> 
> It keeps the expectation that if a client fails to acknowledge a confirmable notification after BEBO completes all transmissions the client is removed from the list of observers.
> 

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://www.sensinode.com @SensinodeIoT
Mobile: +358 40 7796297
Twitter: @zach_shelby
LinkedIn: http://fi.linkedin.com/in/zachshelby
6LoWPAN Book: http://6lowpan.net

---

From: Thomas Fossati <[email-masked]>
Date: Mon, 21 Oct 2013 12:46:25 +0100

>        the intermediate states are silently skipped.

+1

---

From: Klaus Hartke <[email-masked]>
Date: Fri, 25 Oct 2013 23:33:03 +0300

> Peter A. Bigot wrote:

So the idea is to slightly relax the requirement to let a client see
the new state after a state change as soon as possible, but not to
dispose the goal of eventual consistency.

To be clear, this means the options are as follows:

Current (paraphrased):

   If a server generates a new notification while the transmission
   process for a previous notification is still active, then the
   server aborts the transmission process for the old notification
   (but not before the current transmission attempt completes) and
   starts a new transmission process for the new notification (but
   with the retransmission counter and retransmission timer of the
   aborted process instead of the default values).

Invalid change:

   If a server generates a new notification while the transmission
   process for a previous notification is still active, then the
   server has the choice between the following two options:

   - It aborts the transmission process for the old notification
     (but not before the current transmission attempt completes)
     and starts a new transmission process for the new notification
     (but with the retransmission counter and retransmission timer
     of the aborted process instead of the default values).

   or

   - It silently ignores the new notification.

Possible change:

   If a server generates a new notification while the transmission
   process for a previous notification is still active, then the
   server has the choice between the following two options:

   - It aborts the transmission process for the old notification
     (but not before the current transmission attempt completes)
     and starts a new transmission process for the new notification
     (but with the retransmission counter and retransmission timer
     of the aborted process instead of the default values).

   or

   - It waits for the transmission process for the old notification
     to complete. If the transmission process completed successfully,
     it starts a new transmission process for the new notification
     (with default values).

I'm not overly convinced that we gain much from this change, but if it
eases the implementation for some people I wouldn't oppose it.

Best regards,
Klaus

---

From: "Peter A. Bigot" <[email-masked]>
Date: Fri, 25 Oct 2013 17:51:18 -0500

> dispose the goal of eventual consistency.

No.  It's to relax the requirement that all implementations of observe 
consider it "possible" to supersede an in-progress confirmable 
transmission with a new message that takes over remaining 
rights-to-retransmit.

>     aborted process instead of the default values).
[...]
>     - It silently ignores the new notification.
[...]
>       it starts a new transmission process for the new notification

The difference is only in the case where supersedure is not supported. 
Summarizing, the "invalid" one overspecifies by unnecessarily 
introducing "ignores the new notification", while the "possible" one 
overspecifies by explicitly requiring immediate transmission of the 
updated representation after the previous notification completes.  We 
can eliminate the overspecification by relying on more fundamental 
principles.

When I look at observe-11 there is no clear description of timeliness 
requirements for updates under observe.  "As soon as possible" is not a 
requirement: it's a guiding principle which is in direct conflict with 
principles related to limiting message traffic.  "Every change MUST be 
provided as a notification" would be a requirement, but it's not in the 
text (IMO the protocol overview in section 1.2 doesn't specify 
requirements).  If that's what you want, let's add it, then figure out 
if it makes sense.  (What if the resource is updated 100 times per second?)

Then let's see the argument that says observe's timeliness requirements 
are so strong they require that supersedure be a "possible" technique at 
all.  Supersedure now appears to interfere with one use of MID 
deduplication, unless I misunderstood Matthias' dependence on it in the 
token lifetime discussion.  This is one of those things that happen when 
the model of how the message layer works lacks a consensus understanding.

But assume the requirements do clearly state that every change has to 
produce a notification, and assume supersedure is worth accepting as an 
option within the model of what a CoAP message/exchange/transaction is. 
  We still don't need your "possible" rule as an explicit special case. 
  Just describe the functional behavior of observe by saying that each 
change to the resource conceptually adds a potential notification to a 
queue, and the observe infrastructure must transmit or discard the 
notifications in a manner consistent with (a) the balance between 
timeliness and congestion requirements, and (b) availability of 
supersedure as a message-layer transmission optimization.  Once you've 
done that, treating this as a special case is unnecessary.

In fact, I think if the text I proposed as a compromise is accepted then 
the entire conditional:

      If a server generates a new notification while the transmission
      process for a previous notification is still active....

can disappear and the steps at the end of 4.5 become a straightforward 
description of how the observe server transmits notifications whether 
there is or is not a change to the resource during that process.

That's a lot simpler to understand than having detailed instructions for 
a special case, and leaving the reader to try to figure out exactly what 
differences it might have from the less clearly described common case.

Peter

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Sun, 27 Oct 2013 12:49:59 +0000

When the implementation complies with the eventual consistency and stores all new or the latest notification in a queue for subsequent transmission, it is a feasible option.

+1

However, I would then add the text on queuing to get the eventual consistency right, maybe as implementation note.

Ciao
Matthias

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Peter A. Bigot
> Sent: Samstag, 26. Oktober 2013 00:51
> To: [email-masked]
> Subject: Re: [core] WGLC observe comment: notification supersedure
> 
> consider it "possible" to supersede an in-progress confirmable transmission
> with a new message that takes over remaining rights-to-retransmit.
> 
> >
> Summarizing, the "invalid" one overspecifies by unnecessarily introducing
> "ignores the new notification", while the "possible" one overspecifies by
> explicitly requiring immediate transmission of the updated representation
> after the previous notification completes.  We can eliminate the
> overspecification by relying on more fundamental principles.
> 
> requirement: it's a guiding principle which is in direct conflict with principles
> related to limiting message traffic.  "Every change MUST be provided as a
> notification" would be a requirement, but it's not in the text (IMO the
> protocol overview in section 1.2 doesn't specify requirements).  If that's what
> you want, let's add it, then figure out if it makes sense.  (What if the resource
> is updated 100 times per second?)
> 
> Then let's see the argument that says observe's timeliness requirements are
> so strong they require that supersedure be a "possible" technique at all.
> Supersedure now appears to interfere with one use of MID deduplication,
> unless I misunderstood Matthias' dependence on it in the token lifetime
> discussion.  This is one of those things that happen when the model of how
> the message layer works lacks a consensus understanding.
> 
>   Just describe the functional behavior of observe by saying that each change
> to the resource conceptually adds a potential notification to a queue, and the
> observe infrastructure must transmit or discard the notifications in a manner
> consistent with (a) the balance between timeliness and congestion
> requirements, and (b) availability of supersedure as a message-layer
> transmission optimization.  Once you've done that, treating this as a special
> case is unnecessary.
> 
> In fact, I think if the text I proposed as a compromise is accepted then the
> entire conditional:
> 
> description of how the observe server transmits notifications whether there
> is or is not a change to the resource during that process.
> 
> That's a lot simpler to understand than having detailed instructions for a
> special case, and leaving the reader to try to figure out exactly what
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core