Subject: [core] Multicast response suppression, random back-off (ticket #177)

From: "Dijk, Esko" <[email-masked]>
Date: Wed, 14 Dec 2011 12:59:25 +0000

Dear all,

If I'm correct it was mentioned in the Taipei CoRE meeting to add a mechanism such as a random back-off period for responses to multicast CoAP requests, in the base CoAP spec.  (An additional mechanism is not sending a response at all in certain cases; see ticket #177 text).

Considering the wide range of (1) multicast use cases and (2) constrained network topologies/characteristics that CoAP may be used in , it would be best probably to make a back-off period configurable to adapt to the needs. If multiple multicast applications/end-points reside on a CoAP node (say e.g., lighting control and parameters update) it should even be possible I think to configure different intervals per application - in the example a short interval for lighting control and a longer interval for parameters update.

Anyone thoughts on what functionality we need to specify for responding to multicast requests?

regards
Esko

Esko Dijk

Philips Corporate Technologies, Research
High Tech Campus 34, Eindhoven, The Netherlands
[email-masked]

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: Guido Moritz <[email-masked]>
Date: Wed, 14 Dec 2011 17:22:01 +0100

What is the idea how to negotiate the back-off period to be used for the
response? Is it to be sent together with the request? Otherwise, to stay
compliant, implementers may choose always the lowest possible period as
defined in the spec, which leads to a higher independency over different
applications.

Guido

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von
Dijk, Esko
Gesendet: Mittwoch, 14. Dezember 2011 13:59
An: [email-masked]'
Betreff: [core] Multicast response suppression, random back-off (ticket
#177)

Dear all,

If I'm correct it was mentioned in the Taipei CoRE meeting to add a
mechanism such as a random back-off period for responses to multicast CoAP
requests, in the base CoAP spec.  (An additional mechanism is not sending a
response at all in certain cases; see ticket #177 text).

Considering the wide range of (1) multicast use cases and (2) constrained
network topologies/characteristics that CoAP may be used in , it would be
best probably to make a back-off period configurable to adapt to the needs.
If multiple multicast applications/end-points reside on a CoAP node (say
e.g., lighting control and parameters update) it should even be possible I
think to configure different intervals per application - in the example a
short interval for lighting control and a longer interval for parameters
update.

Anyone thoughts on what functionality we need to specify for responding to
multicast requests?

regards

Esko

Esko Dijk

[email-masked]

 

  _____  

The information contained in this message may be confidential and legally
protected under applicable law. The message is intended solely for the
addressee(s). If you are not the intended recipient, you are hereby notified
that any use, forwarding, dissemination, or reproduction of this message is
strictly prohibited and may be unlawful. If you are not the intended
recipient, please contact the sender by return e-mail and destroy all copies
of the original message.

---

From: "Dijk, Esko" <[email-masked]>
Date: Fri, 16 Dec 2011 14:39:24 +0000

Hi Guido, all,

at the very least the back-off period should be configurable at a server, which in itself does not require negotiation features in the CoAP protocol.

Preferably CoAP would also provide a way for the client to include, if required, a preference for the maximum back-off value inside a multicast request CoAP message (e.g. as a CoAP Option "Backoff-Period" as below, value in milliseconds). 
  +-----+----------+----------------+--------+---------+-------------+
  | No. | C/E      | Name           | Format | Length  | Default     |
  +-----+----------+----------------+--------+---------+-------------+
  | 16  | Elective | Backoff-Period | uint   | 0-4 B   | TBD         |

Furthermore, different multicast use cases typically require different response suppression behaviour. Preferably a CoAP client can request i.e. indicate preference for suppression of specific responses. As an example below an 8-bit field in an Option is used to indicate such preferences for non-confirmable multicast requests:

[Default value: No additional suppression e.g. do as in current CoAP spec]
Bit 0: If set, suppress 2.xx success responses
Bit 1: If set, suppress 4.xx client error responses 
Bit 2: If set, suppress 5.xx server error responses
Bit 3: If set, suppress query responses with a size zero result set
Bit 4: If set, suppress any responses with empty payload
[Of course any combination of bits/flags is thus possible.]

  +-----+----------+----------------------+----------+---------+-------------+
  | No. | C/E      | Name                 | Format   | Length  | Default     |
  +-----+----------+----------------------+----------+---------+-------------+
  | 18  | Elective | Response-Suppression | bitfield | 1 B     | 0b00000000  |

As additional configuration (rather than negotiation) mechanisms on top of the base spec we could have a CoAP MIB that exposes all configurable parameters and enables local and remote configuration of these.  The MIB could be exposed also as a specific resource e.g. under .well-known/core , allocated for storage/retrieval of multicast related parameter values.

Any feedback would be most welcome! We should discuss how to best accommodate scalability/multicast control functions in CoAP, given the current state of the CoAP base spec.

regards,
Esko

------------
From: Guido Moritz [mailto:[email-masked]] 
Sent: Wednesday 14 December 2011 17:22
To: Dijk, Esko; [email-masked]
Subject: AW: [core] Multicast response suppression, random back-off (ticket #177)

What is the idea how to negotiate the back-off period to be used for the response? Is it to be sent together with the request? Otherwise, to stay compliant, implementers may choose always the lowest possible period as defined in the spec, which leads to a higher independency over different applications.

Guido

Von: [email-masked] [mailto:[email-masked]] Im Auftrag von Dijk, Esko
Gesendet: Mittwoch, 14. Dezember 2011 13:59
An: [email-masked]'
Betreff: [core] Multicast response suppression, random back-off (ticket #177)

Dear all,

 
 
regards
Esko

Esko Dijk

 

________________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: Guido Moritz <[email-masked]>
Date: Fri, 16 Dec 2011 15:59:39 +0100

Esko,

I see two problems here.

First, neither the client nor the server knows how long the message
transport took. So what is a good value for the back-off?

Second, from a client perspective, how long should I wait for an incoming
response? When is the right time to consider the request/response as
finished and go with whatever follows. It's up to the server. In worst case,
as a client I would always have to wait for the maximum back-off time
defined in the spec. Hence, if the back-off is variable, the client has to
indicate the maximum amount of time for receiving responses.

I am not sure if a response suppression mechanism would be way to complex
from the implementation perspective. I am not sure, it's just a feeling. If
no response is received at the client side, was the response suppressed or
got the response lost on the path? So the client has no knowledge about the
result of the request. Further I guess response suppression is against a
main design criteria, which is the request/response pattern based on the
REST principals. 

Best,
Guido

> -----Ursprüngliche Nachricht-----
> Von: Dijk, Esko [mailto:[email-masked]]
> Gesendet: Freitag, 16. Dezember 2011 15:39
> An: Guido Moritz; [email-masked]
> Betreff: RE: [core] Multicast response suppression, random back-off
(ticket
> #177)
> 
> Hi Guido, all,
> 
> at the very least the back-off period should be configurable at a server,
which
> in itself does not require negotiation features in the CoAP protocol.
> 
> Preferably CoAP would also provide a way for the client to include, if
required,
> a preference for the maximum back-off value inside a multicast request
CoAP
> message (e.g. as a CoAP Option "Backoff-Period" as below, value in
> milliseconds).
> Furthermore, different multicast use cases typically require different
response
> suppression behaviour. Preferably a CoAP client can request i.e. indicate
> preference for suppression of specific responses. As an example below an
8-bit
> field in an Option is used to indicate such preferences for
non-confirmable
> multicast requests:
> As additional configuration (rather than negotiation) mechanisms on top of
the
> base spec we could have a CoAP MIB that exposes all configurable
parameters
> and enables local and remote configuration of these.  The MIB could be
> exposed also as a specific resource e.g. under .well-known/core ,
allocated for
> storage/retrieval of multicast related parameter values.
> 
> Any feedback would be most welcome! We should discuss how to best
> accommodate scalability/multicast control functions in CoAP, given the
current
> state of the CoAP base spec.
> 
> regards,
> Esko
> 
> ------------
> Subject: AW: [core] Multicast response suppression, random back-off
(ticket
#177)
> If I'm correct it was mentioned in the Taipei CoRE meeting to add a
mechanism
> such as a random back-off period for responses to multicast CoAP requests,
in
> the base CoAP spec.  (An additional mechanism is not sending a response at
all
> in certain cases; see ticket #177 text).
> 
best
> probably to make a back-off period configurable to adapt to the needs. If
> multiple multicast applications/end-points reside on a CoAP node (say
e.g.,
> lighting control and parameters update) it should even be possible I think
to
> configure different intervals per application - in the example a short
interval
> for lighting control and a longer interval for parameters update.
> 
> addressee(s). If you are not the intended recipient, you are hereby
notified that
> any use, forwarding, dissemination, or reproduction of this message is
strictly
> prohibited and may be unlawful. If you are not the intended recipient,
please
> contact the sender by return e-mail and destroy all copies of the original
> message.

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 16 Dec 2011 18:33:37 +0100

On Dec 16, 2011, at 15:59, Guido Moritz wrote:

> response? 

Exactly.  We had a couple of situations already where conveying that information from the client to the server is useful in a unicast situation as well.

So we would have an elective "Reponse-most-useful-within" option (measured in, say, seconds), and (for the unicast case) a "sorry I'm not that quick" response code in case the server understands that option and can say that it won't make that deadline.

For the multicast case, the same option would provide the upper bound for the response delay randomly chosen by each server within the range [0, RMUW[ (the randomness need not and should not be restricted to entire seconds).

The duration expressed in the option would be approximate; the client would have to factor in network latencies.

For more sampling-oriented applications of multicast, it would also be useful to provide a Bolot-Turletti-Wakeman (Sigcomm '94) key/keysize to control the probability of responding/the subset responding.  I'm not sure we need to go that far.

Grüße, Carsten

---

From: Likepeng <[email-masked]>
Date: Mon, 19 Dec 2011 01:35:51 +0000

Hi Esko,

Your proposed "Backoff-Period" Option seems to be similar with what I proposed in the request-timeout draft:
http://tools.ietf.org/id/draft-li-core-coap-request-timeout-option-00.txt

Maybe we can work together to add your use case to the draft?

Thanks,
Kind Regards
Kepeng
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Dijk, Esko
Sent: Friday, December 16, 2011 10:39 PM
To: Guido Moritz; [email-masked]
Subject: Re: [core] Multicast response suppression, random back-off (ticket #177)

Hi Guido, all,

regards,
Esko

------------

Guido

Dear all,

 
 
regards
Esko

Esko Dijk

 

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: "Dijk, Esko" <[email-masked]>
Date: Mon, 19 Dec 2011 13:23:30 +0000

Hello Kepeng,

indeed let's try to merge the proposals for Request-Timeout and Backoff-Period. Carsten also suggested that one option could be used for both purposes. However Carsten suggested both the whole second units and fractional second unit formats. Using two different formats for the indicated time could be selectable by a single bit in the option's value. Or perhaps cleaner to use two different Option codes for that?

For Request-Timeout the proposed timeout is 2^T ms with T=0..255, seems not optimal for most use cases. For example I can select 128 ms or 256 ms, but nothing in between say 200ms. On the other hand the large values are not needed (T=68 gives the estimated age of the universe, already :)

One proposal could be to have a large variety of range and resolution achievable within a one byte value, from 8 ms up to 34 minutes:

 0
 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
| T         | TX|
+-+-+-+-+-+-+-+-+

T = Time
TX = Time Exponent

where the timeout is calculated as 
   timeout = 2^(TX * 4 + 3) * T

that appears a bit more complex than 2^T but in code it's quite efficient to implement:
	T <<= ( (TX << 2) + 3);

With an option value e.g. 25 I get my timeout=200 ms. Other schemes are of course very well possible, and we could add an extra byte here as well if we need additional variety or simplicity.

regards,
Esko

-----Original Message-----
From: Likepeng [mailto:[email-masked]] 
Sent: Monday 19 December 2011 2:36
To: Dijk, Esko; [email-masked]
Subject: RE: [core] Multicast response suppression, random back-off (ticket #177)

Hi Esko,

Thanks,
Kind Regards
Kepeng
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Dijk, Esko
Sent: Friday, December 16, 2011 10:39 PM
To: Guido Moritz; [email-masked]
Subject: Re: [core] Multicast response suppression, random back-off (ticket #177)

Hi Guido, all,

regards,
Esko

------------

Guido

Dear all,

 
 
regards
Esko

Esko Dijk

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 19 Dec 2011 14:46:50 +0100

Before we rathole on the duration data type for this, let me point
out:

-- we had a long discussion of such data types previously (around
   max-age), see

   http://tools.ietf.org/html/draft-bormann-coap-misc-10#appendix-B.2

   and following.  Some people didn't like the floating-point type at
   the time because it is not "closed under subtraction" (of course
   it's not, as it is non-negative, but there was some concern that
   people wouldn't be able to handle the accumulation of rounding
   errors during repeated reduction of the value).

-- if you need granularity below a second (why?), milliseconds is a
   bad base, in particular if you want to do binary floating point on
   top of it.  Better: mibiseconds (1/1024 s).

But before we nail down the data type representation, let's discuss
semantics first (and then maybe requirements on the data type).

Grüße, Carsten

---

From: Likepeng <[email-masked]>
Date: Tue, 20 Dec 2011 00:40:58 +0000

Hello Carsten and Esko,

Thanks for your suggestions and proposals.

We are trying to implement this feature in our implementation, and we will try different alternative ways to see which one is more efficient.

And also let's wait for more requirements for that from the industry.

Thanks,
Kind Regards
Kepeng
-----Original Message-----
From: Dijk, Esko [mailto:[email-masked]] 
Sent: Monday, December 19, 2011 9:24 PM
To: Likepeng; [email-masked]; Carsten Bormann
Subject: RE: [core] Multicast response suppression, random back-off (ticket #177)

Hello Kepeng,

 0
 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
| T         | TX|
+-+-+-+-+-+-+-+-+

T = Time
TX = Time Exponent

regards,
Esko

-----Original Message-----
From: Likepeng [mailto:[email-masked]] 
Sent: Monday 19 December 2011 2:36
To: Dijk, Esko; [email-masked]
Subject: RE: [core] Multicast response suppression, random back-off (ticket #177)

Hi Esko,

Thanks,
Kind Regards
Kepeng
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Dijk, Esko
Sent: Friday, December 16, 2011 10:39 PM
To: Guido Moritz; [email-masked]
Subject: Re: [core] Multicast response suppression, random back-off (ticket #177)

Hi Guido, all,

regards,
Esko

------------

Guido

Dear all,

 
 
regards
Esko

Esko Dijk

---

From: "Dijk, Esko" <[email-masked]>
Date: Tue, 20 Dec 2011 09:38:31 +0000

Hello Carsten,

thanks for the pointer; indeed my proposal is an (8,2) pseudo-FP type that denotes time in units of 8 ms (or alternatively, 8/1024 s)

About the semantics: There is a clearly different semantics between "response is most useful before time T" and "if possible please delay your response by a random time within [0,T]". The first is an application concern, while the second is more to 'engineer' the network traffic patterns to an acceptable behaviour i.e. related to the CoAP messaging 'sublayer'. Under the first semantics there's no reason to delay anything while under the second semantics this is the key point.

There was also an open question from Salvatore on the list (June 29) on Request-Timeout: what if the sensor is not able to handle a POST/PUT request in the requested interval of time, should it discard the request entirely or only cancel the sending of the response.

regards,
Esko

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]]
Sent: Monday 19 December 2011 14:47
To: Dijk, Esko
Cc: Likepeng; [email-masked]
Subject: Re: [core] Multicast response suppression, random back-off (ticket #177)

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 20 Dec 2011 13:08:03 +0100

> About the semantics: There is a clearly different semantics between "response is most useful before time T" and "if possible please delay your response by a random time within [0,T]". The first is…

While this is true, the difference is in the unicast (reply now) vs. multicast (reply with a dithered delay) semantics, not in the semantics of the option that is giving the desired time frame.
So I would still be happy to have a single option for response time frame for both cases.

> There was also an open question from Salvatore on the list (June 29) on Request-Timeout: what if the sensor is not able to handle a POST/PUT request in the requested interval of time, should it dis…

If you haven't ACKed yet, you want to quench the retransmissions.  The obvious way to do this would be to send some 4.xx/5.xx response.  (I'm not entirely sure which of those is right.)

If you did ACK and then find out you can't respond in time, you have promised to respond.  It sounds impolite not to heed this promise, so you should be sending a separate 4.xx/5.xx response as soon as you know that.  (The client can't rely on this behavior because the server may not find out in time and because the response time frame option is likely to remain Elective.)

Grüße, Carsten

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 21 Dec 2011 18:29:13 +0100

Hi there,

some consideration in line

> While this is true, the difference is in the unicast (reply now) vs. multicast (reply with a dithered delay) semantics, not in the semantics of the option that is giving the desired time frame.

just for the sake to try to understand better the scenario and the semantic:
could the "response is most useful before time T" semantic be somehow 
useful also for multicast request?

> So I would still be happy to have a single option for response time frame for both cases.

if we can define clear and distinct use case/semantic for unicast and 
multicast
I would also prefer to have a single option

>> There was also an open question from Salvatore on the list (June 29) on Request-Timeout: what if the sensor is not able to handle a POST/PUT request in the requested interval of time, should it di…

thanks for the quotation/reminder Esko :-)

> If you did ACK and then find out you can't respond in time, you have promised to respond.  It sounds impolite not to heed this promise, so you should be sending a separate 4.xx/5.xx response as soo…

which class of response to use 4.xx/5.xx is a good question

4.xx should be used in the scenario where is impossible in general for 
the server to answer within the requested interval;
e.g. the time to calculate the answer is >> Request-timeout

5.xx should be used when the impossibility to answer in time is due to 
the fact that the server is busy/overloaded with other stuff

having said that, my question was slightly different:
most likely the server can not figure it out in advance whether or not 
it will be able to honor the request-timeout
it will only discover that once it has already started to elaborate the 
POST/PUT request
however both POST and PUT request are not save, POST is not idempotent 
while PUT is idempotent.

so we have to think carefully how to apply the Request-Timeout (or 
whatever this option will finally named) for POST/PUT methods

cheers
Salvatore

-- 
Salvatore Loreto
www.sloreto.com

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 21 Dec 2011 19:58:56 +0100

> could the "response is most useful before time T" semantic be somehow useful also for multicast request?

I was trying to argue that the two time frames (client patience, server dithering) are exactly the same ones:
In the multicast case, the server would try to dither within the exactly the time frame that the requester is prepared to wait.
So there is never a good reason to indicate another time frame.
(Note that, in all cases, the time is extended by network latency.  There also should be some exhortation that a retransmitting sender *should* adjust the time if that is indeed what the sender wants.)

> most likely the server can not figure it out in advance whether or not it will be able to honor the request-timeout
> it will only discover that once it has already started to elaborate the POST/PUT request
> however both POST and PUT request are not save, POST is not idempotent while PUT is idempotent.
> 
> so we have to think carefully how to apply the Request-Timeout (or whatever this option will finally named) for POST/PUT methods

Right.  
But I'm pretty sure in the end what we write there will amount to "the server should do the right thing"…
The semantics are just too different in different applications and kinds of resources.

Grüße, Carsten

PS.: Maybe "Patience" is indeed the right name for that option...

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 21 Dec 2011 20:01:50 +0100

> PS.: Maybe "Patience" is indeed the right name for that option...
+1 for  "Patience"  option :-)