Subject: [core] draft-shelby-core-interfaces: call for adoption

From: Andrew Mcgregor <[email-masked]>
Date: Tue, 21 May 2013 10:28:40 +1000

Consensus of the room in Orlando was to adopt draft-shelby-core-interfaces as
a working group document.  This email is a call for confirmation of that
consensus.  Please respond if you support the adoption of this document.

Since REST is not yet in wide use in the microcontroller community,
documenting some examples of good CoAP design will be beneficial.  In
particular, it is not widely understood how to map certain concepts
such as batching or binding to REST.  We don't have an explicit
charter item for documenting this, but it is natural to have
explanatory material with the base specification.  (LWIG is not the
right group for this as this is not about protocol implementation, but
about design of REST interfaces.)

• Dec 2013 Submission to IESG of "CoRE Interfaces" for Informational

(Again, the objective is to provide examples of good usage for
implementers, not to standardize or even give strong recommendations.)

Andrew McGregor | SRE | [email-masked] | +61 4 8143 7128

---

From: "Rahman, Akbar" <[email-masked]>
Date: Mon, 20 May 2013 23:50:21 -0400

+1

Akbar

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Andrew Mcgregor
Sent: Monday, May 20, 2013 8:29 PM
To: [email-masked]
Subject: [core] draft-shelby-core-interfaces: call for adoption

Consensus of the room in Orlando was to adopt
draft-shelby-core-interfaces as a working group document.  This email is
a call for confirmation of that consensus.  Please respond if you
support the adoption of this document.

* Dec 2013 Submission to IESG of "CoRE Interfaces" for Informational

---

From: peter van der Stok <[email-masked]>
Date: Tue, 21 May 2013 08:38:14 +0200

+1

However, the scope and purpose of the document should be a bit clearer.
I see it as a base document that can be referred to by other documents 
that provide additional coap based services.

Having only ONE protocol with related security handling for many coap 
based services can be beneficial for the total amount of code in the 
coap based nodes.

Peter van der Stok

Rahman, Akbar schreef op 2013-05-21 05:50:
> +1
> 
> Akbar
> 
> FROM: [email-masked] [mailto:[email-masked]] ON BEHALF
> OF Andrew Mcgregor
> SENT: Monday, May 20, 2013 8:29 PM
> TO: [email-masked]
> SUBJECT: [core] draft-shelby-core-interfaces: call for adoption
> 
> draft-shelby-core-interfaces as a working group document. This email
> is a call for confirmation of that consensus. Please respond if you
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Lishitao <[email-masked]>
Date: Fri, 24 May 2013 09:27:46 +0000

First, I would like to say, this is a useful draft. It really gives some help and guidance when implementing M2M applications in the real world. 

But, I have concerns about section 5.9 "Resource Observation Attributes" in the interface draft.

I believe the best solution to express the observation condition is to use options, instead of using query parameters contained in the URI.

See the following draft for the detailed solution:
http://tools.ietf.org/id/draft-li-core-conditional-observe-03.txt

I remember that we discussed this before in PairsF2F meeting, and there was no conclusion yet.
http://www.ietf.org/proceedings/83/minutes/minutes-83-core.txt

And I think more discussion is needed for this feature. And I hope we can move this section out before accepting it as WG draft.

Note that the authors currently updating the draft and will upload it very soon.

Regards
Shitao

> -----Original Message-----
> peter van der Stok
> Sent: Tuesday, May 21, 2013 2:38 PM
> To: [email-masked]
> Subject: Re: [core] draft-shelby-core-interfaces: call for adoption
> 
> +1
>

---

From: Zach Shelby <[email-masked]>
Date: Fri, 24 May 2013 12:43:29 +0300

Hi Shitao,

Thanks for the feedback, I am aware of the work on using options for this purpose.

However, there are several reasons why I do believe the URI is the correct place for this, and that we should definitely include this feature in CoRE interfaces.

1. The Origin Server and the Resource, should be in control over any semantics regarding the resource representation. Working with parameters that affect the representation of the resource, MUST be handled as part of the REST interface. Confusing that into the CoAP options, which should be agnostic to what is inside the resource representation, would be the wrong thing to do. What happens if e.g. your resource representation is a more complex JSON object - how are you going to work on thresholds with CoAP options then? 

2. Other standards, e.g. OMA LWM2M are already using this same URI attribute model for controlling observations. It would be best that we try to align multiple standards using CoAP to do this in the same way where possible.

3. You are not going to have a one-size-fits-all with observations conditionals, doing this through REST allows us to provide at least one standard model. But it also allows implementations to evolve the REST interface as needed. Just because we have a REST design that *can* be used for controlling conditionals on resource types defined in CoRE Interfaces, it does not mean that is the only way to do it. Other resource types could inherit or extend this model, and others could do something different.

4. Finally, CoRE is about constrained RESTful environments, not just CoAP. Both the Resource Directory and CoRE Interfaces drafts are mostly usable over HTTP as well, as we want to have people in IoT doing things in the same way, also when HTTP is useful. By designing these generic interfaces using REST, that allows this re-use.

Now, that is not exclusive of someone using CoAP options to control observe conditionals, however a lot more thought would need to go into that. At least for me there are plenty of unanswered questions to that approach. 

Zach

-- 
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://www.sensinode.com @SensinodeIoT
Mobile: +358 40 7796297
Twitter: @zach_shelby
LinkedIn: http://fi.linkedin.com/in/zachshelby
6LoWPAN Book: http://6lowpan.net

---

From: Zach Shelby <[email-masked]>
Date: Fri, 24 May 2013 18:05:16 +0300

On May 21, 2013, at 9:38 AM, peter van der Stok <[email-masked]> wrote:

> I see it as a base document that can be referred to by other documents that provide additional coap based services.

Yes. It defines a REST design paradigm that can just simply be used by a developer, or referenced by other documents/standards building REST interfaces over CoAP.  We can surely (as a WG) explain the scope and purpose better. 

Zach

---

From: Floris Van den Abeele <[email-masked]>
Date: Fri, 24 May 2013 18:25:08 +0200

Hi Zach & all,

It seems to me that the approach in the interfaces draft limits the 
values that a REST resource will actually publish to the 'entire world'? 
Say that 2 clients want to observe the same resource with different 
conditions, with the current approach this isn't possible. A 
'REST-compatible way' to solve this (there might be other, better ideas) 
could be to create a new temporary (sub)resource (e.g. /temp/1, /temp/2) 
in response to the PUT requests for every client and let the client 
observe the 'tailor-made' sub-resource. Whether this is feasible for 
constrained devices, I am unsure. Are you using more than 1 client in 
your deployments? Another critique that I wanted to make is that there 
might be overlap with the URI-query parameters the resource itself 
actually wants to use (related to me reply to your point #3).

I was also wondering why the draft decided to move to a separate PUT 
request for specifying the conditions? Why didn't you keep it in the 
observe request as in previous versions? The above suggestion could 
justify the separate request, although a POST instead of a PUT request 
would be better suited here.

Cheers,
Floris

> 1. The Origin Server and the Resource, should be in control over any semantics regarding the resource representation. Working with parameters that affect the representation of the resource, MUST be…
The logic to filter a more complex JSON object according to a URI-query 
option or an other option (like the Condition option) would be exactly 
the same in both casing; I'm afraid I'm missing the latter point. In the 
conditional observe draft the representation of the resource is actually 
only altered on a per-client basis; so if you would observe the resource 
with a different source endpoint you would still see that the resource's 
representation isn't limited to what the conditions specify. Only the 
notifications that are sent to the client are filtered according to the 
conditions.
> 2. Other standards, e.g. OMA LWM2M are already using this same URI attribute model for controlling observations. It would be best that we try to align multiple standards using CoAP to do this in th…
I agree that it would be best to keep standards aligned where possible. 
Would you mind sending the specific OMA standard to me (off-list)? So 
far I have only found the 'LWM2M-4 Information Reporting' interface.
> 3. You are not going to have a one-size-fits-all with observations conditionals, doing this through REST allows us to provide at least one standard model. But it also allows implementations to evol…
One of the goals of the conditional observe draft is to avoid ending up 
with a plethora of incompatible/non-interoperable conditions. Although 
this discussion isn't related to using CoAP options vs URI queries for 
condition signalling; I think that we as a WG should try to define a set 
of 'common' conditions and refer readers of the interfaces draft to this 
documentation.

In light of these points, I would say that the section could still use 
some additional work/refining. It isn't substantial enough for me to 
block WG adoption though; so +1 for WG adoptation.
> 4. Finally, CoRE is about constrained RESTful environments, not just CoAP. Both the Resource Directory and CoRE Interfaces drafts are mostly usable over HTTP as well, as we want to have people in I…
I agree. Although some mechanisms like observe and block divert from 
this principle, so at a minimum I find it difficult where to draw the line.

---

From: Lishitao <[email-masked]>
Date: Sat, 25 May 2013 01:55:01 +0000

> From: Zach Shelby [mailto:[email-masked]]
> Sent: Friday, May 24, 2013 5:43 PM
> To: Lishitao
> Cc: [email-masked]; [email-masked]
> However, there are several reasons why I do believe the URI is the correct
> place for this, and that we should definitely include this feature in CoRE
> interfaces.
> 
> 1. The Origin Server and the Resource, should be in control over any semantics
> regarding the resource representation. Working with parameters that affect
> the representation of the resource, MUST be handled as part of the REST
> interface. Confusing that into the CoAP options, which should be agnostic to
> what is inside the resource representation, would be the wrong thing to do.
> What happens if e.g. your resource representation is a more complex JSON
> object - how are you going to work on thresholds with CoAP options then?
> 
 I am not quite understand about this, if the resource represents using JSON instead of link format, the client just needs to understand 
about the JSON format before sending any request. It is not a problem only for observe. 

And since the JSON object for resource representation is still under working, it needs a way to solve this problem if it has.

> 2. Other standards, e.g. OMA LWM2M are already using this same URI
> attribute model for controlling observations. It would be best that we try to
> align multiple standards using CoAP to do this in the same way where possible.
>
I think OMA LWM2M just defines the requirements and its structure. The detail about how to use coap is out of scope. 

> 3. You are not going to have a one-size-fits-all with observations conditionals,
> doing this through REST allows us to provide at least one standard model. But it
> also allows implementations to evolve the REST interface as needed. Just
> because we have a REST design that *can* be used for controlling conditionals
> on resource types defined in CoRE Interfaces, it does not mean that is the only
> way to do it. Other resource types could inherit or extend this model, and
> others could do something different.

Yes, using option is one of the possible solution, and can solve most of the problems, URI-query also has its limits.

> 4. Finally, CoRE is about constrained RESTful environments, not just CoAP. Both
> the Resource Directory and CoRE Interfaces drafts are mostly usable over
> HTTP as well, as we want to have people in IoT doing things in the same way,
> also when HTTP is useful. By designing these generic interfaces using REST,
> that allows this re-use.
>

HTTP does not has observe mechanism(something like HTTP streaming may provide the almost same function, but not using the same mechanism),
So I don't think that is a problem, anyway you need a way for interworking between coap and HTTP. Not all the HTTP staff can be used in coap, and vice versa.

> Now, that is not exclusive of someone using CoAP options to control observe
> conditionals, however a lot more thought would need to go into that. At least
> for me there are plenty of unanswered questions to that approach.
>

Yes, I also believe that conditional observe is not an easy issue, our draft has not cover all the problems. 
And it is also not that easy to describe clear just in one clause. So I strongly recommend that moves the conditional observe part from the interface draft, then we can study together to solve that problem.

shitao
> > First, I would like to say, this is a useful draft. It really gives some help and
> guidance when implementing M2M applications in the real world.
> >
> > I believe the best solution to express the observation condition is to use
> options, instead of using query parameters contained in the URI.
> >
> > And I think more discussion is needed for this feature. And I hope we can
> move this section out before accepting it as WG draft.
> >

---

From: Zach Shelby <[email-masked]>
Date: Mon, 27 May 2013 10:14:09 +0300

Floris,

On May 24, 2013, at 7:25 PM, Floris Van den Abeele <[email-masked]> wrote:

> Hi Zach & all,
> 
> It seems to me that the approach in the interfaces draft limits the values that a REST resource will actually publish to the 'entire world'? Say that 2 clients want to observe the same resource with different conditions, with the current approach this isn't possible. A 'REST-compatible way' to solve this (there might be other, better ideas) could be to create a new temporary (sub)resource (e.g. /temp/1, /temp/2) in response to the PUT requests for every client and let the client observe the 'tailor-made' sub-resource. Whether this is feasible for constrained devices, I am unsure. Are you using more than 1 client in your deployments?

Yes, I do agree this is an improvement we need to document for this design. For many contained devices, I would expect that the values from multiple clients are aggregated so that multiple sets of state are not needed. For more capable nodes, we could do what you propose above or just let the server hold multiple sets of resources, one for each endpoint. For example in the OMA Lightweight M2M standard we are using multiple subscribing clients and the approach is the second one.

> Another critique that I wanted to make is that there might be overlap with the URI-query parameters the resource itself actually wants to use (related to me reply to your point #3).

Sure, but CoRE Interfaces defines the REST interface already for sensors, parameters and actuators, and these query parameters are part of that interface. If you would want to apply these conditionals to a different REST interface you could, but then they become part of that REST interface. I don't really see an issue there as the inclusion of those query parameters is explicit.

> I was also wondering why the draft decided to move to a separate PUT request for specifying the conditions? Why didn't you keep it in the observe request as in previous versions? The above suggestion could justify the separate request, although a POST instead of a PUT request would be better suited here.

This was actually a decision made due to a problem pointed out by the working group. The old design where the query string was included with the Observe GET was incompatible with Caching, as the URI of the Observe GET defines the cache entry. This is a non-starter as Observe is very useful for maintaining Cache entries. 

The ability to handle separate sets of conditionals for clients is a nice bonus, and sure, we might evolve the interface to make use of POST for that purpose. 

This is exactly the point of making this a WG document, then we can evolve the document together as a WG. The whole point of CoRE Interfaces is to give good guidance and a starting place for REST interfaces over CoAP (sure, and HTTP) that can be used as-is or re-used and extended.

Thanks,
Zach

> The logic to filter a more complex JSON object according to a URI-query option or an other option (like the Condition option) would be exactly the same in both casing; I'm afraid I'm missing the latter point. In the conditional observe draft the representation of the resource is actually only altered on a per-client basis; so if you would observe the resource with a different source endpoint you would still see that the resource's representation isn't limited to what the conditions specify. Only the notifications that are sent to the client are filtered according to the conditions.
>> 
> I agree that it would be best to keep standards aligned where possible. Would you mind sending the specific OMA standard to me (off-list)? So far I have only found the 'LWM2M-4 Information Reporting' interface.
> One of the goals of the conditional observe draft is to avoid ending up with a plethora of incompatible/non-interoperable conditions. Although this discussion isn't related to using CoAP options vs URI queries for condition signalling; I think that we as a WG should try to define a set of 'common' conditions and refer readers of the interfaces draft to this documentation.
> 
> In light of these points, I would say that the section could still use some additional work/refining. It isn't substantial enough for me to block WG adoption though; so +1 for WG adoptation.
> I agree. Although some mechanisms like observe and block divert from this principle, so at a minimum I find it difficult where to draw the line.

---

From: Zach Shelby <[email-masked]>
Date: Mon, 27 May 2013 10:50:39 +0300

Hi Shitao,

We can just leave the WG to decide if they find a REST interface for conditionals useful as guidance, if enough people find it useful, then we should leave it in. The authors added it in the first place simply because we were asked to. I take it you don't find it useful, that is fine. 

I am seeing lots of people doing this already using REST. Giving developers advice as to a workable model for doing that would be useful for the community in general, and I think that is the purpose of CoRE Interfaces. For example the Cache issue is something easy to overlook without some advice.

> And since the JSON object for resource representation is still under working, it needs a way to solve this problem if it has.

I am referring to the Media Type of the resource representations returned by an Observable resource. The REST conditionals work because we know exactly what the Media Type is (text/plain), because it is defined as part of the REST interfaces in the draft. We are also able to make use of SenML as a batch format as the conditionals are per sub-resource, and not linked with the Observe GET.

This just won't work with CoAP Options. You have no control over what the Media Type is. The resource could return EXI, some specialised binary data, JSON or anything. The only thing you can practically do with a CoAP Option here is to control periodicity, which is not linked with resource representations. 

> I think OMA LWM2M just defines the requirements and its structure. The detail about how to use coap is out of scope. 

That standard defines the complete REST interface using CoAP between a device and a service. And it uses the CoRE Interfaces model for REST based observe conditionals. 

> So I don't think that is a problem, anyway you need a way for interworking between coap and HTTP. Not all the HTTP staff can be used in coap, and vice versa.

Think about an HTTP-CoAP proxy. We are using these all the time to access CoAP resources, and although HTTP does not have Observe, the proxy may often use Observe to update the proxy. The nice thing about the REST approach to conditionals is that it works transparently over HTTP through the proxy. Thus an HTTP based client can fully configure the observe behaviour of a CoAP resource. 

Regards,
Zach

---

From: Jeroen Hoebeke <[email-masked]>
Date: Mon, 27 May 2013 16:45:24 +0200

Hi Zach,

Thanks for the feedback.

>>> interface.

Conditional observe does not affect the representation of the resource, it affects the notifications you receive (when, which). Of course, the ability to conditionally observe a resource does depend on the underlying data type (e.g. xsd:integer). How this data type is represented does not matter.

For every data type, you can think of several interesting conditions. It would indeed be nice if this group can give guidance on how to provide this conditional observe functionality in a resource agnostic way (be it using a URI query or CoAP option) and with a scope that is broader than what is in the current core-interfaces draft. 

Based on our experience implementing and evaluation the conditional observe draft, we can contribute to this discussion. For example, one reason the conditional observed draft has chosen to use of a CoAP option is that it overwrites default observe behaviour:
- a client can select if notifications should be confirmable or not
- if the resource state does not change and max-age expires, no notification is sent. As a result, you only get the events of interest (and not the notifications that would be needed in order to have eventual consistency - especially not in case max-age is set to very low values or even zero to disable caching).

In that respect, the approach you have chosen is different.  

Kind regards,
Jeroen

---

From: Zach Shelby <[email-masked]>
Date: Mon, 27 May 2013 22:28:49 +0300

> Conditional observe does not affect the representation of the resource, it affects the notifications you receive (when, which). Of course, the ability to conditionally observe a resource does depen…

I actually think of a notification being sent each time a resource changes, and thus conditionals affect when a resource changes. But that is philosophical :-)

> For every data type, you can think of several interesting conditions. It would indeed be nice if this group can give guidance on how to provide this conditional observe functionality in a resource…

We could eventually do that. Although we probably need some more experience of what people actually need. This is an area where it is really easy to invent a lot of complex and fancy solutions, so we should be very careful.

CoRE Interfaces is separate from that work however. This gives *a* design pattern for a simple set of REST interfaces that can be used *right now*. That is valuable regardless of new mechanisms we might introduce in the future. If some day we introduce a more advanced mechanism, we can give advice on how to use it. Right now however, we only have CoAP and Observe.  

> In that respect, the approach you have chosen is different.  

I am sure there are several ways to solve the problem.

Thanks,
Zach

---

From: Likepeng <[email-masked]>
Date: Tue, 28 May 2013 03:03:24 +0000

> It would indeed be nice if this group can give guidance on how to provide this conditional observe functionality in a resource agnostic way (be it using a URI query or CoAP option) and with a scope that is broader than what is in the current core-interfaces draft.

I agree that the conditional observe functionality should be done in a resource agnostic way. 

Currently, observe is designed as an option, and I think it will be better to design the observe conditions as options, together with the observe option.

We have implemented conditional observe based on options, and it works quite well.

Kind Regards
Kepeng

-----邮件原件-----
发件人: [email-masked] [mailto:[email-masked]] 代表 Zach Shelby
发送时间: 2013年5月28日 3:29
收件人: Jeroen Hoebeke
抄送: [email-masked]
主题: Re: [core] draft-shelby-core-interfaces: call for adoption

Thanks,
Zach