Subject: [core] #281: Safe-to-forward options

From: "core issue tracker" <[email-masked]>
Date: Wed, 06 Feb 2013 09:41:38 -0000

#281: Safe-to-forward options

 A GET request with an Observe Option to an intermediary can contain
 options that the intermediary doesn't recognise.

 Up to coap-11, this wasn't a problem, because these options were
 rejected/removed, so the request to the origin server didn't contain them.

 Since coap-12, options can be "safe to forward". An intermediary can
 include these in a request to the server even if it doesn't recognised
 them.

 This is a problem, because each of the options could affect the
 observation relationship between intermediary and server in a way that
 makes the notifications unsuitable for sharing between clients with
 different unrecognised options in their requests.

 Since the intermediary doesn't recognise the options, it can also not tell
 if the request will lead to a new observation relationship or override an
 existing one.

-- 
-------------------------+-------------------------------------------------
 Reporter:               |      Owner:  draft-ietf-core-
  [email-masked]         |  [email-masked]
     Type:  protocol     |     Status:  new
  defect                 |  Milestone:  post-WGLC-1
 Priority:  minor        |    Version:  observe-07
Component:  observe      |   Keywords:
 Severity:  In WG Last   |
  Call                   |
-------------------------+-------------------------------------------------

Ticket URL: <http://trac.tools.ietf.org/wg/core/trac/ticket/281>
core <http://tools.ietf.org/core/>

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 6 Feb 2013 14:23:08 +0200

I have no idea how to tackle this yet. Input is appreciated.

Klaus

---

From: Thomas Fossati <[email-masked]>
Date: Wed, 6 Feb 2013 12:38:17 +0000

>>  if the request will lead to a new observation relationship or override an

Would it be safe to treat the whole set of unknown but "safe to
forward" options as a key to identify same-observations ?

Suppose X, Y and Z are StF options, and the following events happen at
a proxy P:
(1) Client A requests an observation for resource r on server S,
appending X and Y and Z;
(2) Client B requests an observation for resource r on server S,
appending X and Y;
(3) Client C requests an observation for resource r on server S,
appending X and Y and Z.

At (1) P will create a new observation for resource r on server S.
At (2) P will create a brand new observation towards S.
At (3) P can reuse observation created at (1).

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 6 Feb 2013 14:56:47 +0200

> forward" options as a key to identify same-observations ?

How does the server know which options are unrecognised by the intermediary?

Klaus

---

From: Thomas Fossati <[email-masked]>
Date: Wed, 6 Feb 2013 14:09:26 +0000

> How does the server know which options are unrecognised by the intermediary?

I'm probably missing something, but: why is it important for the
server to know what the intermediary thinks about the StF options ?

If their semantics will trigger some special handling, the important
thing is that this is made visible to the client.

How can the proxy interfere with that if it clearly separates the
non-equivalent flows ?

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 6 Feb 2013 16:11:38 +0200

> server to know what the intermediary thinks about the StF options ?

Taking your example: Let's say the server recognises X, Y, and Z, and
only X is actually part of the "observe key". Then

At (1) P requests a new observation for resource r on server S. S
creates the observation with key (P, r, X) and sends notifications
affected by X, Y and Z.

At (2) P thinks it cannot reuse observation created at (1) and
requests a new observation towards S. S notes that an observation
already exists for key (P, r, X), replaces the entry and sends
notifications affected by X and Y but not by Z. Client A no longer
receives notifications.

At (3) P thinks it can reuse the observation created at (1). Client A
and C receive no notifications.

So things go wrong if the intermediary and the client do not agree an
how an observation is identified.

Or maybe it's me who's missing something?

Klaus

---

From: Thomas Fossati <[email-masked]>
Date: Wed, 6 Feb 2013 14:18:00 +0000

On Wed, Feb 6, 2013 at 1:21 PM, Klaus Hartke <[email-masked]> wrote:
> Maybe it's me who's missing something?

Ok, let's rewind a bit the discussion.

How many concurrent observations may exist between a proxy and a
server for a given resource ?

If only one, we're basically doomed.

If more than one, we need a mechanism to allow the server and the
proxy to agree on their number and accurate identification.

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 6 Feb 2013 16:27:14 +0200

> server for a given resource ?

Currently, it is one per (client endpoint, request URI).

Ticket #258 proposes to change this to one per (client endpoint,
request URI, list of accept options).

> proxy to agree on their number and accurate identification.

Yes, the client (or an intermediary in the client role) and the server
need to agree on the identification of an observation. If they don't
do this, it doesn't work.

There should be at most one observation per identifiable observation.
I.e., having more than one would only mean receiving duplicates of the
same notifications.

I've wondered for a while now if it wouldn't be better if this wasn't
enforced by the server, though, because the GET requests affecting
each other breaks REST. But the client (or intermediary in the client
role) should still (and must be able to) bundle multiple observations
into one wherever possible.

Klaus

---

From: Floris Van den Abeele <[email-masked]>
Date: Wed, 06 Feb 2013 15:31:27 +0100

I think the fundamental problem is that an observation request can not
be identified by the CoAP endpoint and the request URI alone. The presence
of some types of options will also lead to a new observation
relationship (e.g. an accept option or a block2 option).
The workaround now would be to establish new observation relationships
from different source ports for these cases.  Aggregation can then be 
performed
across all ports.

An alternative would be to include other options besides the request-URI
inside the identification of an observation relationship. One problem then
is how to distinguish between notifications from observe requests with
different options, one could simply echo all the options inside each
notification or provide an ETag-like mechanism.

Floris
Op 06-02-13 15:27, Klaus Hartke schreef:
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Klaus Hartke <[email-masked]>
Date: Wed, 6 Feb 2013 16:36:39 +0200

Floris Van den Abeele wrote:
> different options

That's what we have tokens for. So distinguishing between the
notifications is not a problem. The problem is the identification of
the observation relationship in the presence of unrecognised options.

Klaus

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 6 Feb 2013 16:15:48 +0100

> the observation relationship in the presence of unrecognised options.

I think the latter one is what Floris was really after.

We seem to have two places in the CoAP universe where we install state in the server that is keyed to the installing endpoint and to some (not yet very clearly defined set of) request options:
1) -observe: The observation relationship;
2) -block: The reassembly buffer in the stateful version of Block1.

Now, these two kinds of state item have quite different purposes lifetimes, so it may not be a good idea to handle them the same way.
For now, I'm assuming there will be some commonality.

Right now, an endpoint that wants to have multiple instances of the state has to
a) become multiple endpoints, e.g., by varying the port number.
b) vary some of those request options.

a is expensive with DTLS.
b is influenced by what request options actually go into the identifier of this state.

Robert Quattlebaum seems to suggest adding a "stoken" to enable b when there is no other request option to vary.

Inversely, an endpoint that has changed in some way (new IP address, new DTLS connection) has no way of picking up the state of its previous incarnation.
For 2, this might be about continuing to send blocks for POSTing during an address change.  Maybe not that likely.
For 1, this might be about redirecting an existing observation relationship to a new address.  Hmm.
There are obvious security implications to enabling this kind of tampering (but of course the endpoint information isn't securely transmitted anyway even now).

I haven't really made up my mind whether this is a problem worth solving in a general way.
Except that we do need to define what is included in the set of request options (b).

For 2, intermediaries don't actually combine requests, so there is little problem in including more request options in the differentiating set (except where these are manipulated in an unpredictable way by an intermediary, which would jeopardize putting the pieces together at the server).
For 1, we would like to be able to combine observation relationships.
As we have seen with Accept, that is tricky when the request options differ.
So the fallback, again, should probably be inclusive.

This needs more thinking.

Grüße, Carsten

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 7 Feb 2013 03:36:09 +0200

What about something like this:

* A client (or intermediary in the client role) must bundle multiple
observations into one wherever possible, but a server (or intermediary
in the server role) does not enforce this. Each GET request with an
Observe Option adds the client one time to the list of observers and
does not replace any existing entry. This solves the problem of
clients inadvertently causing servers to cancel an observation if they
don't agree on what makes two requests the same, and it liberates
-observe from the unRESTfulness.

* A client (or intermediary in the client role) can reuse an existing
observation if and only if the cache key matches. This may lead to
some redundant observations if the client does not recognise an
option, but if the option is part of the cache key then the likelihood
of exactly identical notifications should be low. This is consistent
with when a client reuses an existing response from its cache.

Klaus

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 7 Feb 2013 06:13:15 +0000

Hi Klaus, Carsten, Floris, Thomas, all,

Currently, the client can modify the observation relationship by doing another GET, with modified options.

However, if we apply the mechanism below, the client must at least include all cache-key options it used before. Am I correct that this means that the client cannot change the values of these options (or their existence) by just sending a new GET with "observe" option and modified other options? In other words, if the client wants to change a cache-key option, should it first de-register using RST and then re-register with the new cache-key option?

Best regards,
Bert

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Klaus Hartke
Sent: 07 February 2013 09:36
To: [email-masked] WG
Subject: Re: [core] #281: Safe-to-forward options

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 7 Feb 2013 08:31:57 +0200

> However, if we apply the mechanism below, the client must at least include all cache-key options it used before. Am I correct that this means that the client cannot change the values of these optio…

Yes, that's correct.

The only other solution I came up with so far: If a GET request
contains an Observe Option, treat any unrecognised options that are
safe-to-forward like options that are not safe, i.e., remove them or
reject the option with an error response depending on
elective/critical. I wouldn't be very happy with this solution.

Klaus

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 7 Feb 2013 06:39:15 +0000

Hi all,

Another option currently marked as "safe to forward" is eTag.

In case a reverse-proxy aggregates information from several servers, and delivers it to a client as a single resource, it should not forward the eTags from the servers. Also, if the client includes an eTag in a request and the proxy forwards it to the several servers, interesting errors will occur.

I guess the proxy not forwarding the eTag from any server will also resolve the second problem, as the client wouldn't know which eTag to include anyway. Also, a smart aggregating reverse-proxy should understand eTag and handle it appropriately, like described in section 5.7.3.

But more in general, what should aggregating reverse-proxies do with safe-to-forward options? They may make sense only for individual responses, so maybe aggregating reverse-proxies shouldn't forward any unknown options, even when they are "safe to forward".

Best regards,
Bert

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 7 Feb 2013 08:41:15 +0200

>> However, if we apply the mechanism below, the client must at least include all cache-key options it used before. Am I correct that this means that the client cannot change the values of these opti…

But is it a problem?

If it is, the client could include the token of a previously created
observation in its request. This would signal to the server to
update/replace the existing entry. If there is no token in the request
or the token doesn't match any existing list entry, a new entry is
created.

Klaus

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 7 Feb 2013 06:53:08 +0000

Hi Klaus,

> But is it a problem?

Maybe. It means more checking at registration of an observe relationship, as the server now needs to do bookkeeping of all options that were included at request time, and verify these options when it receives a new request.

Linking this somehow to the token seems to be a better solution. The server currently already keeps track of the token, as it needs to include it in the messages associated with an observation relationship anyway. So including the token in the (ip,port,resource) tupple to indentify the observation relationship seems to be easier to implement than adding bookkeeping for various cache-key options.

What do you think?

Best regards,
Bert

-----Original Message-----
From: Klaus Hartke [mailto:[email-masked]] 
Sent: 07 February 2013 14:41
To: Bert Greevenbosch
Cc: [email-masked] WG
Subject: Re: [core] #281: Safe-to-forward options

But is it a problem?

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 7 Feb 2013 09:00:08 +0200

Bert Greevenbosch wrote:
> It means more checking at registration of an observe relationship, as the server now needs to do bookkeeping of all options that were included at request time, and verify these options when it receives a new request.

The proposal was that the server doesn't perform any checking at all.
If a GET request with Observe Option comes in, it adds a new entry to
the list of observers. But if a proxy observes a resource on behalf of
a client, and then another client also wants to observe something
through the proxy, the proxy has to decide if it can reuse the
existing observation relationship or if it has to create a new one.
The proposal was that the server uses the cache-key for this. This
requires some bookkeeping at the proxy, but not at the server.

Klaus

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 7 Feb 2013 07:15:46 +0000

Hi Klaus,

I am still a bit confused about the cache-key. See also one of my comment 19 from the link below. I wrote:

http://www.ietf.org/mail-archive/web/core/current/msg03976.html
{
The cache key seems to be the collection of options that are needed to identify a particular representation of a resource. After much reading, I understand that when a request contains options that are in the cache key, and the proxy does not have a cached representation for exactly those values of the cache key options, then it needs to acquire the related representation, whereas if all options are proxy-safe and not in the cache key, the proxy can ignore them and provide the cached response.
}

The above text is my current understanding of the cache-key. Is it correct?

If so, I guess the server does not need to maintain bookkeeping of the cache-key, as the cache-key is designed for the proxy. The server simply returns whatever when it gets a request; it probably does not maintain a cache itself.

But maybe I misunderstand your e-mail below. Do you mean the proxy does the cache-key bookkeeping, and considers whether it needs to establish a new observation relationship or not, and not the server? So the server still maintains the (ip,port,resource) tuple and the proxy starts a new one e.g. through using a different port?

Best regards,
Bert

-----Original Message-----
From: Klaus Hartke [mailto:[email-masked]] 
Sent: 07 February 2013 15:00
To: Bert Greevenbosch
Cc: [email-masked] WG
Subject: Re: [core] #281: Safe-to-forward options

Klaus

---

From: Robert Quattlebaum <[email-masked]>
Date: Wed, 6 Feb 2013 23:18:30 -0800

> Linking this somehow to the token seems to be a better solution. The server currently already keeps track of the token, as it needs to include it in the messages associated with an observation rela…

For observing, I think this proposal makes the most sense. It's simple, self contained, no muss, no fuss. I think any confusion that could come from this perceived semantic entanglement would be minimal.

As Carsten mentioned, this problem stems from the same root problem as -block, but in this case simply using the token solves the problem without adding any complexity or losing any flexibility. I don't think both problems necessarily need the same solution.

-- Robert

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 7 Feb 2013 08:25:03 +0100

> But more in general, what should aggregating reverse-proxies do with safe-to-forward options? They may make sense only for individual responses, so maybe aggregating reverse-proxies shouldn't forwa…

I think that an "aggregating reverse-proxy" is necessarily even more coupled*) to the origin server(s) than reverse proxies already are.
First of all, we don't even have an application-independent way of aggregating responses; if we develop one, we should think about a way to aggregate response options as well.

One of the points of marking a request option safe-to-forward is being able to rely on that option making it unharmed to an origin server.
If you want a proxy to be allowed to swallow them anyway, then you need the consent of the origin servers.
(You may be able to get that for the more coupled case of reverse-proxies.)

Grüße, Carsten

*) http://en.wikipedia.org/wiki/Coupling_(computer_programming)

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 7 Feb 2013 09:52:03 +0200

> The above text is my current understanding of the cache-key. Is it correct?

Almost, if I understand your comment correctly. The algorithm for an
incoming request looks approximately like this:

1. Does the request contain an unrecognised, critical option that is
not safe-to-forward? If yes, return an error response; done.

2. Does the request contain an unrecognised, elective option that is
not safe-to-forward? If yes, remove it from the request.

3. Create the cache-key for the request. The cache-key includes
- the request method,
- the options that are recognised and specified in the draft/RFC to be
part of the cache-key,
- the options that are not recognised and have an option number which
indicates that the option is part of the cache-key.
Since the set of options recognised by a cache can vary between
different implementations, there can be different results for the
cache-key.

4. Does the cache contain a response with the same cache-key? If yes
and the response is fresh, return it to the client; done.

5. Make a request to the server.

Does that clear things up?

> But maybe I misunderstand your e-mail below. Do you mean the proxy does the cache-key bookkeeping, and considers whether it needs to establish a new observation relationship or not, and not the ser…

I'm not fully sure what I mean yet :-) I think what I mean is that the
proxy does the cache-key bookkeeping, and considers whether it needs
to establish a new observation relationship or not, and not the
server. The server maintains a (ip,port,resource,token) tuple and the
proxy starts a new one through using a different token. Does that make
sense?

Klaus

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 7 Feb 2013 08:02:06 +0000

Hi Carsten,

I agree that the behaviour of "reverse aggregating proxies" highly depends on the particular application, and that it is hard to set general guidelines. I guess most of these proxies will know exactly what they are doing, and thus not blindly forward unknown options - even if when these options are "safe to forward". 

I think the specialist behaviour of these proxy servers makes it infeasible to standardise how they should aggregate responses and response options in general. The consent from the origin server to the proxy is also highly dependent on the particular application.

Best regards,
Bert

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]] 
Sent: 07 February 2013 15:25
To: Bert Greevenbosch
Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [core] eTag safe to forward? (was: RE: #281: Safe-to-forward options)

Grüße, Carsten

---

From: Bert Greevenbosch <[email-masked]>
Date: Thu, 7 Feb 2013 08:19:20 +0000

Hi Klaus,

OK, that makes sense. So for the server, the token is going to be part of identification of the observation relationship, but apart from that the server doesn't need to do any more bookkeeping. I like that a lot, especially since it allows usage of the same client port for multiple observation relationships, independently of any options.

Thanks for your explanation on the cache key, indeed it clears things up.

I agree that it is up to the proxy to determine whether it can use an already existent observation relationship, or needs to establish a new observation relationship. This choice indeed depends on the cache-key options.

Best regards,
Bert

-----Original Message-----
From: Klaus Hartke [mailto:[email-masked]] 
Sent: 07 February 2013 15:52
To: Bert Greevenbosch
Cc: [email-masked] WG
Subject: Re: [core] #281: Safe-to-forward options

Klaus

---

From: Floris Van den Abeele <[email-masked]>
Date: Thu, 07 Feb 2013 10:18:47 +0100

Hi Klaus and Bert,

I like the idea of using tokens to enable more than 1 observation 
relationship from a CoAP endpoint. With this approach a client doesn't 
have to revert to using different sockets for new observe relationships 
with the same resource.

Fixing the mechanism for a client (or proxy) to determine whether or not 
two observation relationships are combinable to using Cache-Keys seems 
to be too restricting. Would this allow a proxy to exclude an unknown, 
elective, safe-to-forward option that is marked as being part of the 
Cache-Key from said determination ? For example, a proxy might know that 
the resource on the server doesn't support the unknown, safe-to-forward, 
elective option anyway (using 
draft-greevenbosch-core-profile-description-01 
<http://datatracker.ietf.org/doc/draft-greevenbosch-core-profile-description/> 
or other similar knowledge), and might choose to exclude this option for 
determining whether to combine relationships. For a server it doesn't 
matter anyway how the aggregation is actually performed. It just matters 
that the aggregation is maximal, which might not be the case when the 
mechanism should get fixed in the future.
To summarize: I feel that using the Cache-keys options for relationship 
aggregation can be a fair 'default' approach but that implementers 
should have to freedom to deviate from this approach when they deem it 
necessary.

Regards,
Floris

Op do 07 feb 2013 09:19:20 CET, Bert Greevenbosch schreef:
>
> Hi Klaus,
>
> OK, that makes sense. So for the server, the token is going to be part 
> of identification of the observation relationship, but apart from that 
> the server doesn't need to do any more bookkeeping. I like that a lot, 
> especially since it allows usage of the same client port for multiple 
> observation relationships, independently of any options.
>
> I agree that it is up to the proxy to determine whether it can use an 
> already existent observation relationship, or needs to establish a new 
> observation relationship. This choice indeed depends on the cache-key 
> options.
>
> Best regards,
> Bert
>

---

From: Jeroen Hoebeke <[email-masked]>
Date: Thu, 7 Feb 2013 10:35:01 +0100

Hi Klaus, Bert,

> Does that clear things up?

When applied to observe: 
- an incoming request with observe option 
- the intermediary does not have the resulting cache key 
- the intermediary establishes an observe relationship with the server on behalf of the client and, internally, stores that it has established this relationship on behalf of the client (+ link with cache-key)

What would be the desired behaviour when the same client makes a new request (using same port) with different options that result in a different cache key? If there was no intermediary, this would result in an update of the observe relationship. With an intermediary, you would like to have the same behaviour.

So, the intermediary should also check whether the observe request comes from a known IP,port(,token). In that case, it should remove the old relationship (related to the different cache key). Right?

> proxy starts a new one through using a different token.

Different port or also different token?
If token is used, a client can have multiple relationships from the same port using different tokens. Since the server has to store the token anyway, this does not seem to introduce much additional complexity and could make life of a client easier. In that case, there is also almost no restriction to the number of relations an intermediary can maintain on behalf of clients.
Assuming tokens are used: In that case: would an unrelated GET from the same port cancel them all? Or would you use a normal GET with the same token to cancel a specific relationship?

Kind regards,
Jeroen

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 7 Feb 2013 11:43:31 +0200

Floris Van den Abeele wrote:
> Would this allow a proxy to exclude an unknown, elective,
> safe-to-forward option that is marked as being part of the Cache-Key from
> said determination ? For example, a proxy might know that the resource on
> the server doesn't support the unknown, safe-to-forward, elective option

If the proxy has such a knowledge about the option, then the options
is not really unrecognised, isn't it? :-)

> aggregation can be a fair 'default' approach but that implementers should
> have to freedom to deviate from this approach when they deem it necessary.

A proxy must still meet the expectations of a client that observes a
resource through it for interoperability. So a proxy implementation
can not arbitrarily deviate from this. But if it has knowledge about
how the server will process an option, it can of course act
accordingly.

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 7 Feb 2013 12:23:40 +0200

> What would be the desired behaviour when the same client makes a new request (using same port) with different options that result in a different cache key? If there was no intermediary, this would…

It's the client(-half of an intermediary) that performs the
aggregation, not the server(-half of the intermediary).

1. Client A makes a GET request with Observe option and token X to proxy P.

2. Proxy P unconditionally creates the observation relationship
between A and P, using the token X for notifications.

3. Proxy P calculates the cache-key for the request and looks for an
existing observation relationship between P and server S. There is
none, so it makes a GET request with Observe option and token V to
server S.

4. Server S unconditionally creates the observation relationship
between P and S, using the token V for notifications.

5. Client B makes a GET request with Observe option and token Y to proxy P.

6. Proxy P unconditionally creates the observation relationship
between B and P, using the token Y for notifications.

7. Proxy P calculates the cache-key for the request and looks for an
existing observation relationship between P and server S. Assuming the
cache-key is equal to the cache-key in step 3, the proxy can reuse the
existing observation relationship between P and S, and does not need
to create a new one.

8. Client A makes a GET request with Observe option and token Z to proxy P.

9. Proxy P unconditionally creates the observation relationship
between A and P, using the token Z for notifications.

10. Proxy P calculates the cache-key for the request and looks for an
existing observation relationship between P and server S. Assuming the
cache-key is different from the cache-key in step 3, it makes a GET
request with Observe option and token W to server S.

11. Server S unconditionally creates the observation relationship
between P and S, using the token W for notifications.

> So, the intermediary should also check whether the observe request comes from a known IP,port(,token). In that case, it should remove the old relationship (related to the different cache key). Righ…

You mean something like this?

1. Client A makes a GET request with Observe option and token X to proxy P.

2. Proxy P unconditionally creates the observation relationship
between A and P, using the token X for notifications.

3. Proxy P calculates cache-key, etc.

4. Client A makes a GET request with Observe option and token X to proxy P.

The client shouldn't use token X for a second request while it's still in use.

> Assuming tokens are used: In that case: would an unrelated GET from the same port cancel them all? Or would you use a normal GET with the same token to cancel a specific relationship?

Since one of the goals is that a request doesn't interfere with any
other request at all (that's what REST calls "stateless
communication"), a GET request wouldn't cancel anything.

1. Client A makes a GET request with Observe option and token X to proxy P.

2. Proxy P unconditionally creates the observation relationship
between A and P, using the token X for notifications.

3. Proxy P calculates cache-key, etc.

4. Client A makes a GET request *without* Observe option and token Y to proxy P.

5. Proxy P calculates the cache-key for the request and looks for an
existing response in its cache. If there is none, it makes a GET
request without Observe option to server S.

Best regards,
Klaus

PS: By the way, thanks to all who have contributed to this so far. It
looks like we're really quickly converging on a working solution! :-D

---

From: Jeroen Hoebeke <[email-masked]>
Date: Thu, 7 Feb 2013 12:31:24 +0100

> between P and S, using the token W for notifications.

I agree.

> The client shouldn't use token X for a second request while it's still in use.

OK, in that case it should be made explicit that updating (not refreshing) an existing relationship means stopping the existing one and establishing a new one (must use new token). The use of tokens to identify relationships (in addition to the port) makes this easier.

Else, the above scenario could result into the following:
- If A does perform step 4, Proxy P detects the relationship exists and wants to create an answer. Since the cache key differs, P could create a second relationship for A (unless some check is being done), which is definitely not the intention.

So, the proxy should detect this behaviour or the client should be forbidden to exhibit such behaviour.

> request without Observe option to server S.

OK, but how do you stop the relationship established in step 2 (section 3.5 of observe draft) when using tokens to identify relationships?
- RST on confirmable notification?
- RST on non-confirmable notification (not guaranteed)
- Get without observe using same port + token
- ...

Kind regards,
jeroen

---

From: Klaus Hartke <[email-masked]>
Date: Fri, 8 Feb 2013 12:18:15 +0200

> So, the proxy should detect this behaviour or the client should be forbidden to exhibit such behaviour.

Section 5.3.1 of -coap already specifies that "the client SHOULD
generate tokens in such a way that tokens currently in use for a given
source/destination endpoint pair are unique." I'm not sure if much
more is needed; if a client wants to reuse a token that is currently
in use, it has to suffer the consequences of implementation defined
behaviour.

An interesting question is, of course: when stops a token being in
use? For example, if a client is no longer interested in a resource
and it just "forgets" the observation relationship (so the next
notification will yield a Reset), the token is still in use by the
server for a while.

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Fri, 8 Feb 2013 12:44:03 +0200

I'm not entirely happy with the proposed solution yet. One strong
reason why -observe was designed such that a GET request refreshes an
existing observation relationship, was that this makes the creation if
the observation relationship idempotent and observation relationships
self-healing: If there is a disagreement between the client and server
on whether an observation relationship exists or not, the next GET
request clears that up. With the proposed solution, if the client
believes that a observation relationship doesn't exist although it
does, it will create a new observation relationship and the existing
one will keep going.

Klaus

---

From: "Kovatsch Matthias" <[email-masked]>
Date: Mon, 11 Feb 2013 11:42:21 +0000

Dear list

I have a bad feeling about the complexity all this introduces. My main problem is that we are preparing for options that make a single resource behave in different ways depending on some yet unknown, maybe vendor-defined option (brace yourselves for RPC-like CoAP...).

If people want to have different behaviors, they should model this with different resources (or queries). That would definitely simplify the identification and merging of observe relationships.

I like the current proposal with the idempotence and self-healing properties Klaus mentioned. Another good simplification would be a non-repeatable Accept option. Then intermediaries do not have to care about various combinations that might end up with the same format in the notifications. If someone worries about efficiency because a client might need to probe the right Content-Format with several requests, just use the Link Format information.

Ciao
Matthias

---

From: Robert Quattlebaum <[email-masked]>
Date: Mon, 11 Feb 2013 09:51:45 -0800

Couldn't a client achieve the self-healing properties you describe by simply always using the same token?

---

From: Maciej Wasilak <[email-masked]>
Date: Tue, 12 Feb 2013 09:20:18 +0100

Hello All,

I think we should check real-life proxy usecases and take them into
consideration when talking about the preferred behavior. There might be
different conditions for forward proxy and reverse proxy. Example for
reverse proxy:

Server:
1. battery-powered room temperature sensor

Proxy:
1. mains-powered linux box

Observing clients:
1. heating controller (needs update only when T<20, content-type: SenML or
something similar)
2. air conditioning (needs update only when T>28, content-type: text )
3. control panel (needs all updates, content-type: JSON based proprietary
format)

In that case the intention for using proxy is preserving server's battery
power and increasing flexibility (server may not support all
content-types). I think the intended behavior is that proxy should receive
all updates from server, and it should pass only required updates to
clients (with correct content-type). Therefore it's not necessary to use
any options from the original requests in the proxy request.

Can someone come up with a decent case for forward-proxy observe
relationship?
Best Regards
Maciek

2013/2/11 Robert Quattlebaum <[email-masked]>

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 14 Feb 2013 10:32:32 +0200

Matthias Kovatsch wrote:
> I have a bad feeling about the complexity all this introduces.

CoAP is a complex protocol. This particular complexity was introduced
by safe-to-forward options. Unless we remove these from -coap again, I
don't think there's solution for -observe that is not slightly more
complex than it was before safe-to-forward options.

> My main problem is that we are preparing for options that make a single resource behave in different ways depending on some yet unknown, maybe vendor-defined option

But that's actually what most options do: they modify the request such
that the server makes the target resource behave slightly different
depending on the options. Accept, ETag, If-Match, If-None-Match,
Observe and Block* are instances of this. I don't think it would be
useful to preclude the addition of such options in the future.

> I like the current proposal with the idempotence and self-healing properties Klaus mentioned.

Yes, I'd like to keep these properties as well. But I'd also like GET
requests to not affect each other.

I've done a bit of experimentation in the last few days and think that
merging the observations at the client-side is a step in the right
direction. It aligns naturally with merging normal GET requests made
through a cache.

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 14 Feb 2013 10:28:50 +0200

> Couldn't a client achieve the self-healing properties you describe by simply always using the same token?

No. The token functions as a generation identifier for the sequence
numbers, so you always need to use a new token if you want to refresh
an existing observation.

Klaus

---

From: "Dijk, Esko" <[email-masked]>
Date: Thu, 14 Feb 2013 09:29:44 +0000

> No. The token functions as a generation identifier for the sequence numbers, so you always need
> to use a new token if you want to refresh an existing observation.

To me it's not yet clear why the sequence numbers need to be "reset" by the server upon refresh (i.e. new generation). Couldn't it just keep counting on? In any case, there's no requirement for the server to start at a specific number, or not? If so it could just keep counting.

I was thinking of the use case where the client reboots, loses state, so forgets about its ongoing observation relation and starts a new one. (At least it thinks its new and not a refresh.) Then it would typically use the same token again if the client is programmed to keep only one observation relation (e.g. default Token).

Esko

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Klaus Hartke
Sent: Thursday 14 February 2013 9:29
To: Robert Quattlebaum
Cc: [email-masked] WG
Subject: Re: [core] #281: Safe-to-forward options

> Couldn't a client achieve the self-healing properties you describe by simply always using the same token?

No. The token functions as a generation identifier for the sequence numbers, so you always need to use a new token if you want to refresh an existing observation.

Klaus

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 14 Feb 2013 11:40:56 +0200

> To me it's not yet clear why the sequence numbers need to be "reset" by the server upon refresh (i.e. new generation). Couldn't it just keep counting on? In any case, there's no requirement for the…

Yes, the server could keep counting on; then it wouldn't be a problem.
But currently the server is *not required* to keep counting on. The
client must use a token to give the server the freedom of choosing
between whether it counts on or not. Counting on may be difficult if
the server reboots and loses state.

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Thu, 14 Feb 2013 12:10:31 +0200

Maciej Wasilak wrote:
> Example for reverse proxy: [...]

Your scenario can be implemented in three ways:

1.
The server exposes three resources:
- one that changes only when T<20, content-type: SenML
- one that changes only when T>28, content-type: text
- one that changes whenever T changes, content-type: JSON based
proprietary format
The proxy creates one observation relationships for each of the three clients.

2.
The server exposes three resources:
- one that changes only when T<20, content-type: SenML
- one that changes only when T>28, content-type: SenML
- one that changes whenever T changes, content-type: SenML
The proxy creates one observation relationships for each of the three
clients, but translates between content types as needed.

3.
The server exposes one resource:
- one that changes whenever T changes, content-type: SenML
The proxy exposes three resources itself which. These provide
customized "views" of the original resource; the proxy filters and
projects the temperature values as needed.

These three ways provide different levels of coupling between the
proxy and the server. In the first case, the proxy is completely
application-agnostic, while in the third case it actually provides
part of the application functionality itself.

If the proxy is so strongly coupled to the server that it knows
whether the server recognises an option or not (so it can fill in the
support for it if not), then the proxy indeed does not have to forward
unrecognised options. It should still be possible to observe a
resource through a proxy that doesn't have such a knowledge though.

Klaus

---

From: Bert Greevenbosch <[email-masked]>
Date: Tue, 19 Feb 2013 02:11:35 +0000

Hi Esko, Klaus, Robert, all,

As I understand this, tokens and sequence numbers are independent. This means that the server could maintain a single counter for all resources, or maybe one counter per resource, or maybe one counter per resource per subscriber. It is up to the server implementation how to maintain the counters, as long as they are monotonically increasing except for the carry around 2^24.

The current proposal is to use the tokens as part of the identifier for the observation relationship, so the tuple (ip, port, token, resource) describes this. To change this observation relationship, the same token should be used. Another token would identify another observation relationship.

Indeed if the server reboots and looses state, it would restart counting and the ordering could be messed up. However, since the server lost its state, it is likely that the server will not send any more observation notifications. When the client realises this, it can re-establish the observation relationship, with a token as it prefers.

There indeed is a small chance that the client does not realise something happened to the server (e.g. reboot) that messed up the sequence numbers. I guess the only time that this is a problem is, when the client wants to modify the observation relationship without knowledge about the server's reboot. In this case, the client could send a GET with observe option and the previous token. Instead of modifying the non-existent observation relationship, the server establishes a new observation relationship. This may lead, depending on the server's implementation, to a restart of the counter and discontinuity in monotonically increase.

But does this have anything to do with adding token to the tuple? It seems that taking account of the token this the problem would be similar.

Corner case? If not, one could consider adding the last received sequence number to the observe request too, such that the server has knowledge about the last sequence number the client has received, and send some error when it is inconsistent. I am not sure if it is worth the complexity though.

What do you think?

Best regards,
Bert

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Klaus Hartke
Sent: 14 February 2013 17:41
To: Dijk, Esko
Cc: [email-masked] WG
Subject: Re: [core] #281: Safe-to-forward options

Klaus

---

From: Klaus Hartke <[email-masked]>
Date: Sat, 23 Feb 2013 16:11:10 +0200

Hi Bert, all,

> The current proposal is to use the tokens as part of the identifier for the observation relationship, so the tuple (ip, port, token, resource) describes this. To change this observation relationshi…

I think you have a point there. We're using tokens to match
notifications to a requests, so it feels right to use the token also
to check if the server is still aware of the client and to tell the
server to stop sending notifications. I'm not sure if labelling the
messages for these two operations as GET requests is the best solution
though.

And I'd prefer to keep that a token functions as generation identifier
for sequence numbers. Temporal serial arithmetic is complicated enough
already.

I'll try to implement the current ideas and see how the changes work
out. The IETF86 submission deadline is quite close though, so I'll
submit observe-08 soon without addressing ticket #281 for now.

Klaus