Subject: [core] CoMI support for YANG notification and RPC

From: Michel Veillette <[email-masked]>
Date: Mon, 20 Apr 2015 20:03:40 +0000

The current revision of CoMI (draft-vanderstok-core-comi-06) specify YANG as data modeling language but do not support two important features of this language, notification and protocol operations (RPCs). Support of these core features of the YANG modeling language should be added to CoMI for the following reasons:

*         CoMI should not be limited to YANG modules specifically defined for it.
YANG modules already defined with notifications and protocol operations (RPCs) should be compatible with CoMI.

*         Notifications and protocol operations (RPCs) are optional and don't require extra resources when not needed.

*         The notification mechanism is complimentary to the already supported observe mechanism, both mechanism can be supported simultaneously and address two different use cases.

*         The notification mechanism:

o   Allow a client to subscribe to a notification stream using a single request (CoAP GET)

o   Allow a client to subscribe to a notification stream without the need to be aware of the specific data nodes implemented by this server

o   Allow a client to request a replay of past notifications (events recorded in the absence of this client)

o   Allow transfer of multiple information associated to an event, some can be specific to this event and as such should not be part of the management data (/mg)

*         RPCs is a controversy subject.
For sure, RPCs shall not be used to replace simple GET or PUT actions, however there is valid use cases for RPCs such as:

o   To implement complex actions (e.g. "toggle switch" can be implemented using a combination of GET switch state, ETag and PUT but can be implemented more efficiently using RPC)

o   A RPC can have both input and output parameters which is not possible with a GET or a PUT.

o   RPCs may carry information specific to this action and as such should not be part of the management data (/mg)

The following sections show possible text to be added to CoMI to support Notifications and RPCs.

________________________________

4.4.  Protocol operations

The CoMI protocol supports protocol operations defined using the YANG "rpc" statement. The solution implemented is based on [I-D.ietf-netconf-restconf] section 3.6.

4.4.1.  Server Support

A COMI server is not required to support COMI protocol operations. Clients may determine if a server supports protocol operations by sending a GET request to "/.well-known/core" including a resource type (RT) parameter with the value "core.rpc".

4.4.2.  Operation resource

The operation resource is a container that provides access to the data-model specific protocol operations supported by the server. Protocol operations are invoked by CoMI clients using a CoAP POST request on this resource.

4.4.3.  Encoding Operation Input Parameters

If the "rpc" statement has an "input" section, then the "input" node is provided in the message-body. The encoding of the "input" node follow CBOR mapping rules defined in section 5.

For example:

Assuming the following YANG defined rpc.

   rpc reboot {
      input {
         leaf delay {
            units seconds;
            type uint32;
            default 0;
         }
         leaf message { type string; }
         leaf language { type string; }
      }
   }

Assuming the module example-ops is module ID 43 or "r" in base64
Assuming the following YANG hash values:
"/example-ops:reboot" = 283286310 or "Q4psm" in base64
"/example-ops:reboot/example-ops:delay" = 980546150
"/example-ops:reboot/example-ops:message" = 882566796
"/example-ops:reboot/example-ops:language" = 284280286

The CoMI client might send the following POST request message:

   REQ: POST example.com/RPC/r/Q4psm
   {
      980546150 : 600,
      882566796 : "Going down for system maintenance",
      284280286 : "en-US"
   }

The CoMI server might respond:

   RES: 2.00 Ok

4.4.4.  Encoding Operation Output Parameters

If the "rpc" statement has an "output" section, then the "output" node is provided in the message-body of the CoAP response. The encoding of the "output" node follow CBOR mapping rules defined in section 5.

For example:

Assuming the following YANG defined rpc.

   rpc get-reboot-info {
      output {
         leaf reboot-time {
            units seconds;
            type uint32;
         }
         leaf message { type string; }
         leaf language { type string; }
      }
   }

Assuming the module example-ops is module ID 43 or "r" in base64
Assuming the following YANG hash values:
"/example-ops:get-reboot-info" = 683557230 or "ovkFu"
"/example-ops:get-reboot-info/example-ops:reboot-time " = 342555434
"/example-ops:get-reboot-info/example-ops:message" = 762496844
"/example-ops:get-reboot-info/example-ops:language" = 353194165

The client might send the following POST request message:

   REQ: POST example.com/RPC/r/ovkFu

The server might respond:

   RES: 2.05 Content (Content-Format: application/cbor)
   {
      342555434 : 30,
      762496844 : "Going down for system maintenance",
      353194165 : "en-US"
   }

________________________________

4.5.  Notifications

The CoMI protocol supports YANG-defined event notifications. The solution implemented is based on [I-D.ietf-netconf-restconf] section 6.

4.5.1.  Server Support

A COMI server is not required to support COMI notifications. Clients may determine if a server supports notifications by sending a GET request to "/.well-known/core" including a resource type (RT) parameter with the value "core.streams".

4.5.2.  Event Streams

A CoMI server that supports notifications MUST populate a stream resource for each notification delivery service access point. A CoMI client can retrieve the list of supported event streams from a CoMI server using the GET operation on the stream list.

The "restconf-state/streams" container definition in the "ietf-restconf-monitoring" module (defined in [I-D.ietf-netconf-restconf] section 9.3) is used to specify the structure and syntax of the child resources within the "streams" resource.

For example:

Assuming the module ietf-restconf-monitoring is module ID 2 or "C" in base64
Assuming the following YANG hash values:
"/rcmon:restconf-state/rcmon:streams" = 869048475 or "zzKCb" in base64
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream" = 305096898
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream/rcmon:name" = 931463846
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream/rcmon:description" = 425672327
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream/rcmon:replay-support" = 301154294
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream/rcmon:replay-log-creation-time" = 126010158
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream/rcmon:encoding" = 374857305
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream/rcmon:encoding/rcmon:type" = 836936555
"/rcmon:restconf-state/rcmon:streams/rcmon:Stream/rcmon:encoding/rcmon:events" = 201862865

The client might send the following request:

   REQ: GET example.com/mg/C/zzKCb

The server might send the following response:

   RES: 2.05 Content (Content-Type: application/cbor)
   {
     869048475 : {
       305096898 : {
         931463846 : "default",
         425672327 : "Default event stream",
         301154294 : true,
         126010158 : "2015-04-15T15:58:00Z",
         374857305 : {
           836936555 : "cbor",
           201862865 : "coaps://example.com/streams/default"
         }
       },
       305096898 : {
         931463846 : "syslog-critical",
         425672327 : "Critical and higher severity",
         301154294 : true,
         126010158  : "2015-04-14T00:00:00Z",
         374857305 : {
           836936555 : "cbor",
           201862865 : "coaps://example.com/streams/syslog-critical"
         }
       }
     }
   }

4.5.3.  Subscribing to Receive Notifications

CoMI clients can determine the URI for the subscription resource (to receive notifications) by sending a GET request for the "events" leaf within the stream list entry.  The value returned by the server can be used for the actual notification subscription.

The client will then send a GET request for the URI returned by the server with the "Accept" type "application/cbor".

4.5.3.1.  Query parameters

Query parameters are optional to implement, and only available if the server supports them.
            +------------+---------+-------------------------+
            | Name       | Section | Description             |
            +------------+---------+-------------------------+
            | start-time | 4.8.9   | replay event start time |
            | stop-time  | 4.8.10  | replay event stop time  |
            | filter     | 4.8.8   | content filter          |
            +------------+---------+-------------------------+

                      CoMI Streams Query Parameters

4.5.3.2.  The "start-time" Query Parameter

The "start-time" parameter is used to trigger the notification replay feature and indicate that the replay should start at the time specified.  If the stream does not support replay, per the "replay-support" attribute returned by the stream list entry for the stream resource, then the server MUST return the error code 4.00 Bad Request.

This parameter is only allowed for GET methods on a /streams data resource.  A 4.00 Bad Request error is returned if used for other methods or resource types.

If this parameter is not present, then a replay subscription is not being requested. It is not valid to specify start times that are later than the current time. If the value specified is earlier than the log can support, the replay will begin with the earliest available notification.

If the "replay-support" leaf is present in the "stream" entry, then the server MUST support the "start-time" and "stop-time" query parameters for that stream.

4.5.3.3.  The "stop-time" Query Parameter

The "stop-time" parameter is used with the replay feature to indicate the newest notifications of interest. This parameter MUST be used with and have a value later than the "start-time" parameter.

This parameter is only allowed for GET methods on a /streams data resource.  A 4.00 Bad Request error is returned if used for other methods or resource types.

If this parameter is not present, the notifications will continue until the subscription is terminated. Values in the future are also valid. In this case the subscription will automatically terminate at the specified "stop-time".

If the "replay-support" leaf is present in the "stream" entry, then the server MUST support the "start-time" and "stop-time" query parameters for that stream.

4.5.3.4.  The "filter" Query Parameter

The "filter" parameter is used to indicate which subset of all possible events are of interest. If not present, all notifications not precluded by other parameters will be sent.

This parameter is only allowed for GET methods on a /streams data resource.  A 4.00 Bad Request error is returned if used for other methods or resource types.

The format of this parameter is a comma separated list of module IDs formatted in base 64. To filter a subset of the notifications defined in a module, the module ID is followed by a comma separated list of the YANG hash values associated to each notification of interest. Each notification list is delimited by parentheses and hash values are formatted in base 64.

For example:

Assuming the module toaster is module ID 47129 or "LgZ" in base64
Assuming the module nc-notifications is module ID 18 or "S" in base64
Assuming the following YANG hash values:
"/rc:notification/toast:toastDone" = 128560783 or "Hqa6P" in base64
"/rc:notification/manageEvent:replayComplete" = 162935187 or "JtjGT"
"/rc:notification/manageEvent:notificationComplete" = 901415891 or "1uoPT"

To subscribe to all notifications of the toaster module, the CoMI client might send:

   REQ: GET /streams/default?filter=LgZ

To subscribe to all notifications of both modules, the CoMI client might send:

   REQ: GET /streams/default?filter=LgZ,S

To subscribe to the toastDone notification of the toaster module plus the replayComplete and the notificationComplete notifications of the nc-notifications module, the CoMI client might send:

   REQ: GET /streams/default?filter=LgZ(Hqa6P),S(JtjGT,1uoPT)

4.5.4.  Receiving Event Notifications

As defined in section 4.4.3., CoMI clients subscribe to a notification stream using a CoAP GET request. This request contains a Token which will be used for all subsequent CoAP messages associated with this notification stream. This CoAP response may contain one or multiple notifications, in the case the status code is set to 2.05 (Content). The CoAP response may also be empty, in the case the status code is set to 2.00 (Ok).

Subsequent notifications are sent by the CoMI server to each subscribed CoMI client in a non-confirmable CoAP message containing POST method. The Token included shall be set to the Token initially received in the subscription request. The status code of the POST message shall be set to 2.05 (Content), each POST message may contain one or multiple notifications.

The structure of each notification is based on the "notification" container defined below. This container is associated with the module name "ietf-restconf".

   container notification {
      description "Notification message wrapper.";
      leaf event-time {
         type yang:date-and-time;
         mandatory true;
         description "The time the event was generated by the event source.";
         reference "RFC 5277, section 4, <eventTime> element.";
      }
   }

Notifications are implemented as a single pair CBOR map with the key part set to the module ID and the value part containing CBOR map of the notification content.

For example:

Assuming the module toaster is module ID 47129
Assuming the following YANG hash values:
"/rc:notification/toast:toastDone" = 128560783
"/rc:notification/rc:event-time" = 1071794137
"/rc:notification/toast:toastDone" = 128560783
"/rc:notification/toast:toastDone/toast:toastStatus" = 30090935

The CoMI might sent the following CoAP request to subscribe to the notification stream.

   REQ: GET /streams/default&start-time=2015-04-01T00:00:00Z (Token 0x7A1D)

The CoMI might receive the following CoAP response.

   RES: 2.05 Content (Token 0x7A1D)
   {
     47129 : {                                       # module toaster
       1071794137 : "2015-04-01T12:16:51Z",          # leaf event-time
       128560783 : {                                 # notification toastDone
         30090935 : 0                                # leaf toastStatus
       }
     }
   }
   {
     47129 : {                                       # module toaster
       1071794137 : "2015-04-01T12:19:06Z",          # leaf event-time
       128560783 : {                                 # notification toastDone
         30090935 : 1                                # leaf toastStatus
       }
     }
   }

After some time, the CoMI might receive the following non-confirmable CoAP POST message.

   REQ: POST (Token 0x7A1D)
   {
     47129 : {                                       # module toaster
       1071794137 : "2015-04-01T18:05:23Z",          # leaf event-time
       128560783 : {                                 # notification toastDone
         30090935 : 0                                # leaf toastStatus
       }
     }
   }

Michel Veillette
System Architecture Director
Trilliant Inc.
Tel: 450-375-0556 ext. 237
[email-masked]<mailto:[email-masked]>
www.trilliantinc.com<http://www.trilliantinc.com/>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 20 Apr 2015 13:40:30 -0700

Hi,

This is an interesting proposal -- I support it.
There are details to work out of course.

There is no proposal for using observe + pub/sub to
implement CoMI notifications.

I think tradition event notifications are needed because they tend
to use less resources.  They tend to be aggregations and are sometimes
time throttled, often defined with a default periodic interval.

On Mon, Apr 20, 2015 at 1:03 PM, Michel Veillette <
[email-masked]> wrote:

>  The current revision of CoMI (draft-vanderstok-core-comi-06) specify
> YANG as data modeling language but do not support two important features of
> this language, notification and protocol operations (RPCs). Support of
> these core features of the YANG modeling language should be added to CoMI
> for the following reasons:
>
>
>
> ·         CoMI should not be limited to YANG modules specifically defined
> for it.
> ·         Notifications and protocol operations (RPCs) are optional and
> don’t require extra resources when not needed.
>

This would need to be optional to implement in the server.
In NETCONF/RESTCONF, they are also all-or-nothing,
meaning if notification delivery is supported, than all
YANG notification-stmts must be supported.  All rpc-stmts
must also be supported in every server (no consensus
on this point for RESTCONF yet).

IMO, CoMI compliance might be different than the original YANG
module intended for NETCONF or RESTCONF.  It might make
sense to let the server

 ·         The notification mechanism is complimentary to the already
> supported observe mechanism, both mechanism can be supported simultaneously
> and address two different use cases.
>
> ·         The notification mechanism:
>
> o   Allow a client to subscribe to a notification stream without the need
> to be aware of the specific data nodes implemented by this server
>

This is optional to implement on a NETCONF or RESTCONF server.
IMO not likely this would be supported by a CoMI server.

Andy

 o   Allow transfer of multiple information associated to an event, some
> can be specific to this event and as such should not be part of the
> management data (/mg)
>
> ·         RPCs is a controversy subject.
> For sure, RPCs shall not be used to replace simple GET or PUT actions,
> however there is valid use cases for RPCs such as:
>
> o   To implement complex actions (e.g. “toggle switch” can be implemented
> using a combination of GET switch state, ETag and PUT but can be
> implemented more efficiently using RPC)
>

>  ------------------------------
>
>
>
> 4.4.  Protocol operations
>
>
>
> The CoMI protocol supports protocol operations defined using the YANG
> "rpc" statement. The solution implemented is based on
> [I-D.ietf-netconf-restconf] section 3.6.
>
>
>
> 4.4.1.  Server Support
>
>
>
> A COMI server is not required to support COMI protocol operations. Clients
> may determine if a server supports protocol operations by sending a GET
> request to "/.well-known/core" including a resource type (RT) parameter
> with the value "core.rpc".
>
>
>
> The operation resource is a container that provides access to the
> data-model specific protocol operations supported by the server. Protocol
> operations are invoked by CoMI clients using a CoAP POST request on this
> resource.
>
>
>
> If the "rpc" statement has an "input" section, then the "input" node is
> provided in the message-body. The encoding of the "input" node follow CBOR
> mapping rules defined in section 5.
>
>
>
> For example:
>
>
>
> If the "rpc" statement has an "output" section, then the "output" node is
> provided in the message-body of the CoAP response. The encoding of the
> "output" node follow CBOR mapping rules defined in section 5.
>
>
>
>
>
> For example:
>
>
>
>  ------------------------------
>
>
>
> 4.5.  Notifications
>
>
>
> The CoMI protocol supports YANG-defined event notifications. The solution
> implemented is based on [I-D.ietf-netconf-restconf] section 6.
>
>
>
> 4.5.1.  Server Support
>
>
>
> A COMI server is not required to support COMI notifications. Clients may
> determine if a server supports notifications by sending a GET request to
> "/.well-known/core" including a resource type (RT) parameter with the value
> "core.streams".
>
>
>
> 4.5.2.  Event Streams
>
>
>
> A CoMI server that supports notifications MUST populate a stream resource
> for each notification delivery service access point. A CoMI client can
> retrieve the list of supported event streams from a CoMI server using the
> GET operation on the stream list.
>
>
>
> The "restconf-state/streams" container definition in the
> "ietf-restconf-monitoring" module (defined in [I-D.ietf-netconf-restconf]
> section 9.3) is used to specify the structure and syntax of the child
> resources within the "streams" resource.
>
>
>
> For example:
>
>
>
> CoMI clients can determine the URI for the subscription resource (to
> receive notifications) by sending a GET request for the "events" leaf
> within the stream list entry.  The value returned by the server can be used
> for the actual notification subscription.
>
>
>
> The client will then send a GET request for the URI returned by the server
> with the "Accept" type "application/cbor".
>
>
>
> The "start-time" parameter is used to trigger the notification replay
> feature and indicate that the replay should start at the time specified.
> If the stream does not support replay, per the "replay-support" attribute
> returned by the stream list entry for the stream resource, then the server
> MUST return the error code 4.00 Bad Request.
>
>
>
> This parameter is only allowed for GET methods on a /streams data
> resource.  A 4.00 Bad Request error is returned if used for other methods
> or resource types.
>
>
>
> If this parameter is not present, then a replay subscription is not being
> requested. It is not valid to specify start times that are later than the
> current time. If the value specified is earlier than the log can support,
> the replay will begin with the earliest available notification.
>
>
>
> If the "replay-support" leaf is present in the "stream" entry, then the
> server MUST support the "start-time" and "stop-time" query parameters for
> that stream.
>
>
>
> The "stop-time" parameter is used with the replay feature to indicate the
> newest notifications of interest. This parameter MUST be used with and have
> a value later than the "start-time" parameter.
>
>
>
> This parameter is only allowed for GET methods on a /streams data
> resource.  A 4.00 Bad Request error is returned if used for other methods
> or resource types.
>
>
>
> If this parameter is not present, the notifications will continue until
> the subscription is terminated. Values in the future are also valid. In
> this case the subscription will automatically terminate at the specified
> "stop-time".
>
>
>
> If the "replay-support" leaf is present in the "stream" entry, then the
> server MUST support the "start-time" and "stop-time" query parameters for
> that stream.
>
>
>
> The "filter" parameter is used to indicate which subset of all possible
> events are of interest. If not present, all notifications not precluded by
> other parameters will be sent.
>
>
>
> This parameter is only allowed for GET methods on a /streams data
> resource.  A 4.00 Bad Request error is returned if used for other methods
> or resource types.
>
>
>
> The format of this parameter is a comma separated list of module IDs
> formatted in base 64. To filter a subset of the notifications defined in a
> module, the module ID is followed by a comma separated list of the YANG
> hash values associated to each notification of interest. Each notification
> list is delimited by parentheses and hash values are formatted in base 64.
>
>
>
> For example:
>
>
>
> To subscribe to the toastDone notification of the toaster module plus the
> replayComplete and the notificationComplete notifications of the
> nc-notifications module, the CoMI client might send:
>
>
>
> As defined in section 4.4.3., CoMI clients subscribe to a notification
> stream using a CoAP GET request. This request contains a Token which will
> be used for all subsequent CoAP messages associated with this notification
> stream. This CoAP response may contain one or multiple notifications, in
> the case the status code is set to 2.05 (Content). The CoAP response may
> also be empty, in the case the status code is set to 2.00 (Ok).
>
>
>
> Subsequent notifications are sent by the CoMI server to each subscribed
> CoMI client in a non-confirmable CoAP message containing POST method. The
> Token included shall be set to the Token initially received in the
> subscription request. The status code of the POST message shall be set to
> 2.05 (Content), each POST message may contain one or multiple notifications.
>
>
>
> The structure of each notification is based on the "notification"
> container defined below. This container is associated with the module name
> "ietf-restconf".
>
>
>
>    container notification {
>
> Notifications are implemented as a single pair CBOR map with the key part
> set to the module ID and the value part containing CBOR map of the
> notification content.
>
>
>
> For example:
>
>
>
> [email-masked]
>
> www.trilliantinc.com
>
>
>
>
>
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core
>
>