Subject: [netmod] applied configuration and system-controlled entries

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 7 Jan 2016 11:20:14 +0100

Hi,

a good use of applied configuration could be to formalize the concept of system-controlled entries as defined in RFC 7223, routing-cfg, and probably elsewhere, too.

My idea is that system-controlled interfaces or other entries would appear in applied configuration, but not in intended configuration until something needs to be really configured. We could then permit leafrefs from intended configuration to refer to leafs in applied configuration. One case where this would be useful is the ACL module, where match conditions refering to interfaces currently have to use plain strings as references to interface names.

However, the above idea seems to be at odds with requirement 1D in opstate-reqs-02. I wonder: could that requirement be relaxed or removed so that the above use case can be supported?

Thanks, Lada
  
--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Robert Wilton <[email-masked]>
Date: Thu, 7 Jan 2016 16:43:32 +0000

Hi Lada,

I think that requirement 1D is fairly key to what is being asked for 
here to allow both the user and system to easily relate between what the 
operator desires and what configuration the system is actually using, so 
I wouldn't be particularly keen on loosening this requirement.

For the ACL example:
Would it be feasible to change the ACL module to use a leafref to the 
interface name, with the added constraint that you have to at least 
configure the existence of an interface before you can have any 
configuration referring to it?

Thanks,
Rob

On 07/01/2016 10:20, Ladislav Lhotka wrote:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
> .
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 08 Jan 2016 13:30:53 +0100

Robert Wilton <[email-masked]> writes:

> operator desires and what configuration the system is actually using,

In a way, system-controlled interfaces are default entries in the
interface list - and the system can certainly be using interfaces with
no configuration installed by NETCONF/RESTCONF clients.

> so I wouldn't be particularly keen on loosening this requirement.

OK, but then IMO this intended-applied dualism is of limited
utility. For many systems or services, asynchronicity is not an option,
or isn't important.

> configuration referring to it?

Well, yes, that's how it is supposed to be done now - also, for example,
for stacking interfaces as in Appendix B of RFC 7223.

It is not only extra work: the interface list can be locked, so it may
not be possible to immediately create a dummy interface entry and,
consequently, an ACL rule with that interface cannot be configured. In
this sense, using a string rather than a leafref looks like a reasonable
choice.

As Martin pointed out, with YANG 1.1 it would be possible to refer to an
interface entry in state data from configuration. On the other hand,
with "require-instance false" validation won't detect errors in ACL
configuration such as referring to a non-existent interface.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Fri, 8 Jan 2016 15:20:16 +0000

Hi Lada,

> utility. For many systems or services, asynchronicity is not an option,
I don't really agree.   I think that this is plausibly important to 
anyone who wants to manage network devices in an automated way.

Today, when a config request has been processed by NETCONF/YANG then it 
seems that the only thing the server actually promises is that it might 
apply the configuration at some point in the future.  It makes no 
guarantees as to whether that configuration has or ever will be 
successfully applied!  The expectation is that the operator can infer 
from the operational state of the system whether the configuration has 
been applied (which may not always be possible).

Personally, if I was an operator coding to NETCONF/YANG then I would 
prefer a device that supported the opstate semantics because compliant 
devices will tell you what configuration that are actually running.  
This makes it easier to control/manage.  I.e. I can easily find out 
if/when my configuration is applied, and I can easily retrieve any 
associated operational state.  In some cases this may mean that I don't 
need to check the derived state at all, in other cases it may simplify 
the checking of the derived state that I need to perform.

Again, regarding sync/async, I would also most likely choose async 
because I perceive that it is simpler to code in a robust fashion.

> this sense, using a string rather than a leafref looks like a reasonable
Locking is presumably specifically related to the config protocol being 
used.  As such I don't think that it should be used as a reason to 
compromise the structure of data model.

But having looked at the ACL model a bit more closely, and noticing how 
input_interface is used, then I agree with Martin, I think that it makes 
sense for the reference to be an interface-state-ref.

> configuration such as referring to a non-existent interface.
True, but for this particular scenario, it wouldn't be any different 
from matching on the wrong IP address (which also can't be validated by 
the device in question).

Presumably the ACL is still valid even if the input-interface for one 
ACE doesn't exist.  Wouldn't it just mean that one particular ACE entry 
wouldn't match any packets?

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 8 Jan 2016 16:42:33 +0100

> On 08 Jan 2016, at 16:20, Robert Wilton <[email-masked]> wrote:
> 
> Hi Lada,
> 
> I don't really agree.   I think that this is plausibly important to anyone who wants to manage network devices in an automated way.

I am currently working with my colleagues on two use cases:

1. RESTCONF interface to a DNS server that will cover the daemon configuration, policies for zone signing, and zone provisioning.

2. RESTCONF interface to an OpenWRT-based router.

For neither of them applied configuration seems to add any value.

> 
> Today, when a config request has been processed by NETCONF/YANG then it seems that the only thing the server actually promises is that it might apply the configuration at some point in the future.  It makes no guarantees as to whether that configuration has or ever

The server may simply send a reply only after the configuration has been completely applied. I understand that with complex devices it is not possible or practical to wait, but in my use cases it is probably just the right thing to do.

>  will be successfully applied!  The expectation is that the operator can infer from the operational state of the system whether the configuration has been applied (which may not always be possible).
> 
> Personally, if I was an operator coding to NETCONF/YANG then I would prefer a device that supported the opstate semantics because compliant devices will tell you what configuration that are actually running.  This makes it easier to control/manage.  I.e. I can easily find out if/when my configuration is applied, and I can easily retrieve any associated operational state.  In some cases this may mean that I don't need to check the derived state at all, in other cases it may simplify the checking of the derived state that I need to perform.
> 
> Again, regarding sync/async, I would also most likely choose async because I perceive that it is simpler to code in a robust fashion.
> 
>> 
>>> For the ACL example:
> Locking is presumably specifically related to the config protocol being used.  As such I don't think that it should be used as a reason to compromise the structure of data model.
> 
> But having looked at the ACL model a bit more closely, and noticing how input_interface is used, then I agree with Martin, I think that it makes sense for the reference to be an interface-state-ref.
> 
>> 
> True, but for this particular scenario, it wouldn't be any different from matching on the wrong IP address (which also can't be validated by the device in question).

Sure, but my point is that a leafref with require-instance=false isn't that much different from a plain string.

Lada

> 
> Presumably the ACL is still valid even if the input-interface for one ACE doesn't exist. Wouldn't it just mean that one particular ACE entry wouldn't match any packets?
> 
> Thanks,
> Rob
> 
> 
>> 
>> Lada
>> 
>>> Thanks,
>>> Rob
>>> 
>>>

---

From: Robert Wilton <[email-masked]>
Date: Fri, 8 Jan 2016 16:22:07 +0000

> For neither of them applied configuration seems to add any value.
OK.

> The server may simply send a reply only after the configuration has been completely applied.
Yes, the server may do this.  But the client cannot rely on that 
behaviour for a NETCONF server today.

When I first got involved in the opstate discussions, I mistakenly 
thought that when a NETCONF server replies to a request it mean that the 
configuration was applied (for some definition of applied). From my 
reading of draft-openconfig-netmod-opstate-01 section 4.2, I suspect 
that the authors were also under that impression.  However, several 
folks on the WG alias have made it clear that this behaviour is not 
specified by the NETCONF draft.

But the nice thing is, if the server complies with the opstate 
requirements draft, then when the client receives a successful reply to 
the synchronous configuration request it knows that configuration has 
actually been applied successfully.  It seems to me that these are the 
semantics that your client code probably desires.

Alternatively, perhaps the client code doesn't directly care whether or 
not the configuration has been successfully applied because it will 
infer it from the operational state.  In this case it can make async 
requests, with the knowledge that any server should reply to the config 
request reasonably expediently.

 From a client perspective, either of these two requests seem to be 
better than the status quo, where it can infer very little from the 
result of a request.  Two different server implementations may have very 
different behaviour and hence a robust client has to be coded to assume 
the least desirable behaviour.

>   I understand that with complex devices it is not possible or practical to wait, but in my use cases it is probably just the right thing to do.
For me, it is about what a client can reasonably assume about the 
servers behaviour, and I think that the tighter behaviour specified in 
the opstate requirements means that servers should behave how clients 
expect them to.

> Sure, but my point is that a leafref with require-instance=false isn't that much different from a plain string.
Yes, it may not tell you much extra over a plain string, but it still 
gives a clearer indication as to what the field is.

Thanks,
Rob

---

From: Gert Grammel <[email-masked]>
Date: Mon, 11 Jan 2016 13:25:33 +0000

Lada,

The requirement says:
       D.  When a configuration change for any intended configuration
           node has been successfully applied to the server (e.g. not
           failed, nor deferred due to absent hardware) then the
           existence and value of the corresponding applied
           configuration node must match the intended configuration
           node.

I don't see that this would limit the case you described below. In your case there is no intended config, hence there is no "corresponding applied configuration" either.

Besides that, the case you mentioned should be clearly in scope. 

--Gert

>-----Original Message-----
>From: netmod [mailto:[email-masked]] On Behalf Of Ladislav Lhotka
>Sent: 07 January 2016 11:20
>To: NETMOD WG
>Subject: [netmod] applied configuration and system-controlled entries
>
>Hi,
>
>a good use of applied configuration could be to formalize the concept of
>system-controlled entries as defined in RFC 7223, routing-cfg, and probably
>elsewhere, too.
>
>My idea is that system-controlled interfaces or other entries would appear in
>applied configuration, but not in intended configuration until something needs
>to be really configured. We could then permit leafrefs from intended
>configuration to refer to leafs in applied configuration. One case where this
>would be useful is the ACL module, where match conditions refering to
>interfaces currently have to use plain strings as references to interface names.
>
>However, the above idea seems to be at odds with requirement 1D in opstate-
>reqs-02. I wonder: could that requirement be relaxed or removed so that the
>above use case can be supported?
>
>Thanks, Lada
>
>--

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 Jan 2016 14:48:54 +0100

Hi Gert,

> On 11 Jan 2016, at 14:25, Gert Grammel <[email-masked]> wrote:
> 
> Lada,
> 
> The requirement says:

You are right, the requirement can be interpreted this way. I thought that applied configuration was supposed to be identical to intended after some synchronization period.

> Besides that, the case you mentioned should be clearly in scope.

Great, then I am open to discussing what this could mean for the existing modules (ietf-interfaces, ietf-routing, ACL etc.).

One useful change to YANG semantics could be that a leafref with require-instance=true would refer to applied configuration. Specifically, the ACL module could then simply use "if:interface-ref" (with require-instance=true) as the type for "input-interface". 

Thanks, Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 11 Jan 2016 14:54:36 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> > I don't see that this would limit the case you described below. In
> > your case there is no intended config, hence there is no
> > "corresponding applied configuration" either.
> 
> You are right, the requirement can be interpreted this way. I thought
> that applied configuration was supposed to be identical to intended
> after some synchronization period.

This is a very important point to clarify.  Can there ever be data in
"applied" that is not in "intended"?  I think Anees & Rob previously
said "no", but I might be wrong.

/martin

> Great, then I am open to discussing what this could mean for the
> existing modules (ietf-interfaces, ietf-routing, ACL etc.).
> 
> One useful change to YANG semantics could be that a leafref with
> require-instance=true would refer to applied
> configuration. Specifically, the ACL module could then simply use
> "if:interface-ref" (with require-instance=true) as the type for
> "input-interface".
> 
> Thanks, Lada
> 
> >  
> > 
> > --Gert
> >

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 Jan 2016 14:56:25 +0100

> On 11 Jan 2016, at 14:54, Martin Bjorklund <[email-masked]> wrote:
> 

Yup, I also had this impression.

Lada

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Mon, 11 Jan 2016 14:07:13 +0000

>said "no", but I might be wrong.

My opinion is that there is a 1-1 relationship between “applied” and
“intended” config.

Thanks,
Acee

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 11 Jan 2016 15:11:18 +0100

> said "no", but I might be wrong.

If there is time delay between editing intended and the applied config
matching the edits of intended, then I supose this can happen (I
delete a resource from intended but it is still around until intended
has been fully synced). I would find it interesting if some edits are
always assumed to be synchronous but others may be asynchronous.

And Lada, I think applied may happen to be never identical to intended
if, for example, hardware is absent or other missing resources prevent
certain parts of intended to become applied.

My understanding is that applied config in general forms an extended
subset of intended config. And to fully understand what a device is
doing, I may need to obtain its entire operational state since the
applied config may not include state obtained dynamically from other
sources.

But I might still all be wrong...

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 Jan 2016 15:11:39 +0100

> On 11 Jan 2016, at 15:07, Acee Lindem (acee) <[email-masked]> wrote:
> 
> 
> 

Do you mean their data model or instances? The point is: if the device has an interface, say "eth0" that is operational and configurable, but hasn't been configured so far via intended config, then does "eth0" entry appear in the "if:interface" list of applied config or not?

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 11 Jan 2016 15:13:06 +0100

> My opinion is that there is a 1-1 relationship between “applied” and

I do not understand. Please clarify what 1-1 means here.

/js

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Mon, 11 Jan 2016 14:14:08 +0000

On 1/11/16, 3:11 PM, "Ladislav Lhotka" <[email-masked]> wrote:

>Do you mean their data model or instances? The point is: if the device
>has an interface, say "eth0" that is operational and configurable, but
>hasn't been configured so far via intended config, then does "eth0" entry
>appear in the "if:interface" list of applied config or not?

Data model.

Acee

>>>>>> a good use of applied configuration could be to formalize the
>>>>>>concept
>>>>>> of

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Jan 2016 14:15:52 +0000

Hi Gert, Lada,

> You are right, the requirement can be interpreted this way. I thought that applied configuration was supposed to be identical to intended after some synchronization period.
Yes, when the system settles, and assuming that all configuration has 
been successfully applied, then the applied config nodes MUST exactly 
match the intended config nodes.

This point has been explicitly asked of Rob Shakir and Anees and they 
have confirmed that these are the semantics that are expected.

Thanks,
Rob

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Mon, 11 Jan 2016 14:20:05 +0000

>I do not understand. Please clarify what 1-1 means here.

In the context of these requirements, I do not agree that applied config
includes anything else. The data models for the intended and applied
config are the aligned and all other state falls in the category of
derived state.  

Thanks,
Acee

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Jan 2016 14:23:05 +0000

> But I might still all be wrong...
I think that your articulation is correct.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 Jan 2016 15:27:58 +0100

> On 11 Jan 2016, at 15:11, Juergen Schoenwaelder <[email-masked]> wrote:
> 

Using applied config for system-controlled entries would require that such an entry stays (forever) in applied config even after it has been deleted from intended. 

> certain parts of intended to become applied.

Yes, this is the use case of pre-provisioning, which is important, too, but in fact opposite: the question here is whether applied can contain stuff that's not (and never been) in intended.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 11 Jan 2016 15:49:53 +0100

> config are the aligned and all other state falls in the category of

Apparently, you talk about the data model while I think Martin's
question and the subsequent discussion was about instances.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 Jan 2016 15:51:59 +0100

> On 11 Jan 2016, at 15:15, Robert Wilton <[email-masked]> wrote:
> 
> Hi Gert, Lada,
> 
> Yes, when the system settles, and assuming that all configuration has been successfully applied, then the applied config nodes MUST exactly match the intended config nodes.
> 
> This point has been explicitly asked of Rob Shakir and Anees and they have confirmed that these are the semantics that are expected.

OK, back to square 1. :-) Then I think the requirements should make this point very clear.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Jan 2016 14:58:54 +0000

> Using applied config for system-controlled entries would require that such an entry stays (forever) in applied config even after it has been deleted from intended.
I think that this would make life harder for clients.

With the requirements as they are today, the client gives the intended 
config to the system, which it can monitor until the applied config 
matches the intended config.  At this point it knows everything is good 
and the config is fully applied.  Over time, if everything is behaving 
as expected, the client can reasonably expect that the applied 
configuration will always converge on the intended configuration.

Using applied config for system-controlled entries would break the 
simple logical relationship between intended and applied configuration, 
since there are now some special entries for which this rule does not 
always apply.

> Yes, this is the use case of pre-provisioning, which is important, too, but in fact opposite: the question here is whether applied can contain stuff that's not (and never been) in intended.

I think that the answer is basically no, unless it is an error condition 
and it is representing configuration that should be deleted.

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Jan 2016 15:30:18 +0000

> OK, back to square 1. :-) Then I think the requirements should make this point very clear.
They try to, but it is quite tricky to express given there are scenarios 
where the two can differ (e.g. failures (depending on config edit 
semantics), missing hardware, etc).  The intention of the word existence 
in the paragraph above is meant to indicate that a node must either 
exist in both intended and applied, or not exist in either.

Going back to your original problem, my understanding is that the only 
solution in YANG today is to have a config false hierarchy to represent 
system-controlled objects whose lifetime is independent from 
configuration, hence the existence of "interfaces-state" separate from 
"interfaces".  With this approach you lose the ability to retrieve all 
configuration and state together in a single sub-tree within a request 
but gain the flexibility of independent data lifetime.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 11 Jan 2016 16:36:27 +0100

> On 11 Jan 2016, at 15:58, Robert Wilton <[email-masked]> wrote:
> 
> 
> 

Hmm, I would say the opposite. For one, we could simplify the data models by reducing the duplicities in configuration and state trees.

> 
> With the requirements as they are today, the client gives the intended config to the system, which it can monitor until the applied config matches the intended config.  At this point it knows everything is good and the config is fully applied.  Over time, if everything is behaving as expected, the client can reasonably expect that the applied configuration will always converge on the intended configuration.

One could argue that leafs with default values are in applied configuration before they appear in intended. So my idea was to extend this to default entries of certain lists.

> 
> Using applied config for system-controlled entries would break the simple logical relationship between intended and applied configuration, since there are now some special entries for which this rule does not always apply.

The introduction of applied configuration would mean significant complications to all protocols, and perhaps even to YANG (although I'd hope not). Solving only the synchonicity issue with it is IMO insufficient payoff for all the troubles. 

Lada

> I think that the answer is basically no, unless it is an error condition and it is representing configuration that should be deleted.
> 
> Thanks,
> Rob
> 
> 
>> 
>> Lada
>>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 11 Jan 2016 17:13:48 +0100

> but gain the flexibility of independent data lifetime.

I think <get/> filters even today allow you to retrieve both subtrees
(intended and operational state) together if that is desired.

/js

---

From: Gert Grammel <[email-masked]>
Date: Mon, 11 Jan 2016 16:19:16 +0000

>Sent: 11 January 2016 16:36
>To: Robert Wilton
>Cc: [email-masked]
>Subject: Re: [netmod] applied configuration and system-controlled entries
>
>
>>>> On 11 Jan 2016, at 15:11, Juergen Schoenwaelder
>>>>>>> I don't see that this would limit the case you described below.
>>>>>>> In your case there is no intended config, hence there is no
>>>>>> You are right, the requirement can be interpreted this way. I
>>>>>> thought that applied configuration was supposed to be identical to
>>>>>> intended after some synchronization period.
>>>>> This is a very important point to clarify.  Can there ever be data
>>>>> in "applied" that is not in "intended"?  I think Anees & Rob
>>>>> previously said "no", but I might be wrong.
>>>>>
>>>> If there is time delay between editing intended and the applied
>>>> config matching the edits of intended, then I supose this can happen
>>>> (I delete a resource from intended but it is still around until
>>>> intended has been fully synced). I would find it interesting if some
>>>> edits are
>>> Using applied config for system-controlled entries would require that such
>an entry stays (forever) in applied config even after it has been deleted from
>intended.
>Hmm, I would say the opposite. For one, we could simplify the data models by
>reducing the duplicities in configuration and state trees.
>

Let's look at a practical example and see if we can converge:
1) A Server is happily running with intended == applied config and operational state is aligned with intended config.
2) A new HW is plugged into that server and that HW has some kind of identification number

This can practically fall into 3 cases:
a) it's a plug&play device and it is accepted to be used
b) it's an accepted device but shall be configured prior to become operational
c) it's considered a harmful (unplanned) event and the device shall be removed

In terms of a possible implementation, the event 
1) would update the applied config which -  among others - include the identification number and update the operational state of that HW in line with the applied config
2) Since the new applied config differs from the intended config, the client is notified about the change
3) Upon inspection of the config change, the client decides how to deal with the new item:
	a) accepting it the way it is: pushing down an intended config in line with the applied config information
	b) accepting it with modifications: pushing down a new intended config with additional leafs
	c) rejecting it: raising an alarm indicating the configuration mismatch, requiring manual intervention

>>
>> With the requirements as they are today, the client gives the intended config
>to the system, which it can monitor until the applied config matches the
>intended config.  At this point it knows everything is good and the config is
>fully applied.  Over time, if everything is behaving as expected, the client can
>reasonably expect that the applied configuration will always converge on the
>intended configuration.

Which is the case above. By updating the intended config the client accepts the config change.
>
>One could argue that leafs with default values are in applied configuration
>before they appear in intended. So my idea was to extend this to default
>entries of certain lists.
>
>>
>> Using applied config for system-controlled entries would break the simple
>logical relationship between intended and applied configuration, since there
>are now some special entries for which this rule does not always apply.
>
Don't get to the same conclusion, see above.

>The introduction of applied configuration would mean significant
>complications to all protocols, and perhaps even to YANG (although I'd hope
>not). Solving only the synchonicity issue with it is IMO insufficient payoff for all
>the troubles.
>
Don't see this either.

>>>> And Lada, I think applied may happen to be never identical to
>>>> intended if, for example, hardware is absent or other missing
>>>> resources prevent certain parts of intended to become applied.
>>> Yes, this is the use case of pre-provisioning, which is important, too, but in
>fact opposite: the question here is whether applied can contain stuff that's not
>(and never been) in intended.
>>
>> I think that the answer is basically no, unless it is an error condition and it is
>representing configuration that should be deleted.
>>
>> Thanks,
>> Rob
>>
>>
>>>
>>> Lada
>>>

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Jan 2016 16:57:30 +0000

Hi Gert,

Please see inline ...

> 2) A new HW is plugged into that server and that HW has some kind of identification number
I think that you can solve this more cleanly with interfaces-state or 
equivalent.  I don't think that a device should be creating 
configuration, the configuration should solely be controlled by the 
operator, and hence the ownership and flow of information is well defined.

> a) it's a plug&play device and it is accepted to be used
In this case, it could generate a YANG notification from something like 
the Entity YANG model (or equivalent), and instantiate the new resources 
in interfaces-state as appropriate (or equivalent).  The operator can 
then discover this and apply whatever configuration is appropriate (or 
perhaps they have put the configuration in previously waiting for the 
hardware to be inserted).

> b) it's an accepted device but shall be configured prior to become operational
Same as above.

> c) it's considered a harmful (unplanned) event and the device shall be removed
It generates a syslog error message, and perhaps Entity YANG model 
notification.  The system doesn't accept any configuration related to 
the unsupported HW.

> 	c) rejecting it: raising an alarm indicating the configuration mismatch, requiring manual intervention

I don't think that you need to use applied config to solve this. You can 
support this using operational state and notifications, this then allows 
the configuration to be controlled by a single entity (i.e. the operator).

> Which is the case above. By updating the intended config the client accepts the config change.
Personally I don't think that it should be up to the client to 
accept/reject a config change that the server is making.  The server 
should only notify what has changed in the system and leave it to the 
client to decide what configuration should be put on the system.

Besides, I'm not sure that there is a clean way for the client to reject 
such an applied config change.  I.e. I can't see how it can delete the 
configuration from the intended config because it already isn't there -  
it would need some special semantics and I don't think that they really 
help simplify this problem.

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Mon, 11 Jan 2016 17:11:49 +0000

> (intended and operational state) together if that is desired.
Yes, I think that this partly helps.

I think that it may be desirable (perhaps via a protocol extension) to 
have the two sub-trees merged into a single tree so that the config and 
state data is co-located.  However, to achieve this programmatically 
would probably require some more rules on consistent node naming between 
related config and state sub-trees.

Arguably it is similar to having separate datastores for intended and 
applied configuration, but wanting to retrieve with the data co-located 
in the same tree structure.  Which IIRC, you have previously suggested 
could be solved via an appropriate protocol extension as well.

Thanks,
Rob

---

From: Gert Grammel <[email-masked]>
Date: Mon, 11 Jan 2016 17:49:34 +0000

Rob,

I realize that the pre-conditions of the example made weren't clear. There is certainly the possibility to pre-configure a node in the intended config and wait until some HW is inserted. Indeed in a Telco environment this is the preferred case and will stay so for a while.

Lada mentioned a case which didn't have an intended config to begin with and asked how it fits in the model. I was using the HW insertion case to show how it can be dealt with: either by physical intervention (removing the HW) or by updating the intended config to accept the server state. Admittedly this is not the norm, but rather an exception. In any event, the difference between intended and applied is temporary.

--Gert

>From: Robert Wilton [mailto:[email-masked]]
>Sent: 11 January 2016 17:58
>To: Gert Grammel; Ladislav Lhotka
>Cc: [email-masked]
>>>>>>>> thought that applied configuration was supposed to be identical
>>>>>>>> to intended after some synchronization period.
>>>>>>> This is a very important point to clarify.  Can there ever be
>>>>>>> data in "applied" that is not in "intended"?  I think Anees & Rob
>>>>>> config matching the edits of intended, then I supose this can
>>>>>> happen (I delete a resource from intended but it is still around
>>>>>> until intended has been fully synced). I would find it interesting
>>>>>> if some edits are
>>> an entry stays (forever) in applied config even after it has been
>>> deleted from intended.
>>> Hmm, I would say the opposite. For one, we could simplify the data
>>> models by reducing the duplicities in configuration and state trees.
>>>
>> 1) A Server is happily running with intended == applied config and
>operational state is aligned with intended config.
>I think that you can solve this more cleanly with interfaces-state or equivalent.
>I don't think that a device should be creating configuration, the configuration
>should solely be controlled by the operator, and hence the ownership and flow
>of information is well defined.
>
I would rephrase your words a bit:
I don't think that a device should be creating *intended* configuration, the *intended* configuration
should solely be controlled by the operator, and hence the ownership and flow of information is well defined.
Instead the applied configuration should represent the server as-is and an inserted HW can't be changed by SW. 
However HW can be put in an operational state that inhibits it from being used.

>In this case, it could generate a YANG notification from something like the
>Entity YANG model (or equivalent), and instantiate the new resources in
>interfaces-state as appropriate (or equivalent).  The operator can then discover
>this and apply whatever configuration is appropriate (or perhaps they have put
>the configuration in previously waiting for the hardware to be inserted).

That would practically mean some pre-provisioning is already in the intended (and applied) config, and the operational state is aligned with the applied configuration upon insertion.

>notification.  The system doesn't accept any configuration related to the
>unsupported HW.

Which would mean there is an intended (and applied) config present in the server that would require HW insertion to generate a syslog message. There are certainly environments where unplanned insertion of HW shall trigger syslog error messages. However it wouldn't be a clean solution to require Error messages triggering a Client to update the server's intended config.

>> 1) would update the applied config which -  among others - include the
>> identification number and update the operational state of that HW in
>> line with the applied config
>I don't think that you need to use applied config to solve this. You can support
>this using operational state and notifications, this then allows the
>configuration to be controlled by a single entity (i.e. the operator).
>
>>
>config

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 11 Jan 2016 21:18:37 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> >> Using applied config for system-controlled entries would require that
> >> such an entry stays (forever) in applied config even after it has been
> >> deleted from intended.

This is the old idea of having the "operational state" datastore,
which would be all config true + all config false nodes.  One issue
with this is that the semantics of the node is different in the
different data stores, even if the syntax (by definition!) is the
same.  In order to handle this properly you need either two different
description statements, or two "sections" within the description
statement.

   list interface {
     description-config
       "The list of configured interfaces on the device.
        ...";
     description-oper
       "The list of interfaces on the device.
        
        System-controlled interfaces created by the system are
        always present in this list, whether they are configured or
        not.
        ...";
   }

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 12 Jan 2016 10:05:41 +0100

> description statements, or two "sections" within the description

I think this is not much different from default values. Leafs and
leaf-lists that have them defined in the data model may not be present
in intended config, yet one can consider them to be in applied config.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 12 Jan 2016 10:23:33 +0100

> 	a) accepting it the way it is: pushing down an intended config
> 	in line with the applied config information

This shouldn't be necessary. Default values are also accepted as
something the device operationally uses, but they needn't be copied to
intended config.

In fact, I think this intended-applied duality could be used for a sound
definition of default contents: defaults would be present in applied but
not in intended config. This would eliminate the need for with-defaults.

Otherwise, your scenario looks good to me.

Lada

---

From: Gert Grammel <[email-masked]>
Date: Tue, 12 Jan 2016 09:59:28 +0000

Lada,

Shortening the thread a bit, citing from your response:
In fact, I think this intended-applied duality could be used for a sound definition of default contents: defaults would be present in applied but not in intended config. This would eliminate the need for with-defaults.

That’s indeed what I am after. A lot of kludgy things could fall into their place by applying intended/applied config sensibly. Default values are one of them. Derived state of non-explicitly configured leafs (i.e. not part of intended config) is another. That said, we consciously need to keep existing models and implementation into account.

IMO the overall guideline using intended vs. applied config should be:

  1.  Intended config is owned by the client and can only be modified by the client. A Server may reject an intended config only if it is syntactically incorrect.
  2.  Applied config is owned by the server and can only be modified by the server to reflect configuration changes or to apply intended config
  3.  A client has rwx access to intended config and r access to applied config (x meaning to request the server to apply it in a certain way)
  4.  A server has r accesss to intended config and rw access to applied config
  5.  If after attempting to apply the intended config there are still differences between intended and applied, they shall be notified by the server to the client
  6.  The client shall resolve the differences by either modifying the intended config or by a maintenance action. In both cases, a difference is considered temporary.

With that guideline in mind, for a properly configured server, the intended config is a true subset of the applied config with a 1:1 relationship.

Gert

From: Ladislav Lhotka <[email-masked]<mailto:[email-masked]>>
Date: Tuesday 12 January 2016 10:23
To: Gert Grammel <[email-masked]<mailto:[email-masked]>>, Robert Wilton <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [netmod] applied configuration and system-controlled entries

Gert Grammel <[email-masked]<mailto:[email-masked]>> writes:

-----Original Message-----
From: netmod [mailto:[email-masked]] On Behalf Of Ladislav Lhotka
Sent: 11 January 2016 16:36
To: Robert Wilton
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [netmod] applied configuration and system-controlled entries

On 11 Jan 2016, at 15:58, Robert Wilton <[email-masked]<mailto:[email-masked]>> wrote:

On 11/01/2016 14:27, Ladislav Lhotka wrote:
On 11 Jan 2016, at 15:11, Juergen Schoenwaelder
<[email-masked]<mailto:[email-masked]>> wrote:

On Mon, Jan 11, 2016 at 02:54:36PM +0100, Martin Bjorklund wrote:
Ladislav Lhotka <[email-masked]<mailto:[email-masked]>> wrote:
Hi Gert,

On 11 Jan 2016, at 14:25, Gert Grammel <[email-masked]<mailto:[email-masked]>>
wrote:

Lada,

The requirement says:

Lada

b) accepting it with modifications: pushing down a new intended
config with additional leafs
c) rejecting it: raising an alarm indicating the configuration
mismatch, requiring manual intervention

Don't see this either.

Lada

Thanks,
Rob

Lada

/js

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod
.

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

---

From: Robert Wilton <[email-masked]>
Date: Tue, 12 Jan 2016 10:12:16 +0000

> in intended config, yet one can consider them to be in applied config.
I think that default values are logically just a way to make the 
configuration data more concise.  I.e. everywhere you have a default 
value then the equivalent configuration could be expressed with a 
explicit value set instead.

As such, I think that the default values apply equally to both the 
intended and applied configuration.

If after a config request it takes time for the system to apply a 
default value, then ideally the applied configuration should have an 
explicit leaf to show what value is actually in effect until the default 
value has actually been applied in the system.

This does open the question of how do you express the case that no value 
has been applied rather than a different value.  For the opstate 
encoding solution draft that I put forward (or using meta-data), then I 
think that it would probably be possible to extend the encoding to 
explicitly include this information if required.

Alternatively, and for the other proposed opstate solutions, then I 
expect that the most appropriate ideal semantics to handle this scenario 
would be to delay marking the parent object as being applied until every 
descendent child leaf with a default value set has a value applied in 
the system, either to the expected default value (in which case the 
child leaf wouldn't need to be present in the applied config), or 
transiently to another value (which would be in the applied config until 
the system updated and the correct default values have actually been 
applied).  In real systems, I would have thought that the default values 
are often implicitly programmed when the parent containing object is 
created anyway, so I suspect that this behaviour wouldn't actually be 
that onerous to implement.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 12 Jan 2016 11:41:29 +0100

> On 12 Jan 2016, at 11:12, Robert Wilton <[email-masked]> wrote:
> 
> 
> 
> I think that default values are logically just a way to make the configuration data more concise.  I.e. everywhere you have a default value then the equivalent configuration could be expressed with a explicit value set instead.
> 
> As such, I think that the default values apply equally to both the intended and applied configuration.

It is useful if the operator is able to retrieve the parameter values that the device is really using - which is the stated purpose of applied config. There may also be vendor-specific defaults that aren't documented in a data model, or defaults that are computed dynamically.

> 
> If after a config request it takes time for the system to apply a default value, then ideally the applied configuration should have an explicit leaf to show what value is actually in effect until the default value has actually been applied in the system.

My uderstanding is that default values come from the device so they needn't be applied. For example, if an interface card is activated, it will typically use some default values (e.g. MTU) without waiting for any configuration.

Lada 

> 
> This does open the question of how do you express the case that no value has been applied rather than a different value.  For the opstate encoding solution draft that I put forward (or using meta-data), then I think that it would probably be possible to extend the encoding to explicitly include this information if required.
> 
> Alternatively, and for the other proposed opstate solutions, then I expect that the most appropriate ideal semantics to handle this scenario would be to delay marking the parent object as being applied until every descendent child leaf with a default value set has a value applied in the system, either to the expected default value (in which case the child leaf wouldn't need to be present in the applied config), or transiently to another value (which would be in the applied config until the system updated and the correct default values have actually been applied).  In real systems, I would have thought that the default values are often implicitly programmed when the parent containing object is created anyway, so I suspect that this behaviour wouldn't actually be that onerous to implement.
> 
> Thanks,
> Rob
> 
> 
>> 
>> Lada
>> 
>>>    list interface {
>>>      description-config

---

From: Gert Grammel <[email-masked]>
Date: Tue, 12 Jan 2016 10:42:33 +0000

On 2016-12-01 11:12, "netmod on behalf of Robert Wilton"
<[email-masked] on behalf of [email-masked]> wrote:

>>>>>>>>>>In
>>>>>>>>>thought
>>>>>>>>data in

I think default values are just what they are. A set of values on the
server that have not been explicitly set by the client via
intended-config. It is certainly possible to set default values in
intended-config explicitly, but it is not strictly needed.

>intended and applied configuration.
While this is allowed (see above), I would consider default values apply
in practice to the applied config

>value has actually been applied in the system.
If the client has an indication whether the configuration application is
finished (with or without errors), it knows that the applied config is now
stable to be read. Reading applied config during an operation should be
avoided. Hence I would question the need for a per-leaf state and advocate
for a per-server state.

>explicitly include this information if required.
Perhaps I am missing something here. If the value supposed to be applied
is the same that already exists, just skip it in the server.
>that onerous to implement.
It appears to me that we run into those issues by requiring the intended
config to contain also the default values. That doesn¹t seem necessary. I
would also argue that marking the Œapplied=intended¹ state per leaf is
counter-productive. Assume the client sets an intended state of two values
(A,B) on two different leafs with rollback-on-error semantic. If applying
A succeeds and the client reads the applied state while applying B is
still in progress, a rollback may still happen. That would invalidate the
previous value of A. In other words it is unwise to read data from a
source while it is actually writing.

---

From: Robert Wilton <[email-masked]>
Date: Tue, 12 Jan 2016 13:29:46 +0000

> It is useful if the operator is able to retrieve the parameter values that the device is really using - which is the stated purpose of applied config. There may also be vendor-specific defaults tha…
OK, so in essence you are proposing that the intended and applied config 
nodes have different schemas, in that any default values in the YANG 
schema only apply to the leaves in the intended config schema and not 
the applied config schema.  As such, because the applied config schema 
doesn't haven't default values in the schema it must report all values 
it is using.

But if we were to do this, then I'm not entirely sure why you would keep 
default values in the intended configuration schema (other than 
backwards compatibility, and potentially for documentation purposes)?

Personally, I think that default values are a useful way to simply and 
shorten the configuration data set, but arguably this can apply equally 
to both intended and applied configuration.  For me, the main argument 
for not allowing default values in the applied config is because of the 
difficultly for the server to differentiate between a leaf that hasn't 
been applied at all (e.g. for an optional feature) and the same leaf 
that is applied with the default value.

I.e. I think that the applied configuration nodes already solve the 
problem of the device defaults not matching the configuration schema, 
since these differences should be reported in the applied configuration 
(or alternatively as deviations to the original schema that is being 
supported).

> My uderstanding is that default values come from the device so they needn't be applied. For example, if an interface card is activated, it will typically use some default values (e.g. MTU) without…

Yes.  But if the default values that are actually being used by the 
device don't match the implicitly expected value from leaf defaults then 
these should be reported in the applied configuration with the value set 
to the actual applied value.

Thanks,
Rob

---

From: Robert Wilton <[email-masked]>
Date: Tue, 12 Jan 2016 14:04:42 +0000

On 12/01/2016 10:42, Gert Grammel wrote:
> intended-config.

The default values are determined by the schema and the intended 
configuration.  You need to know both to know what default values the 
server is expected to use.

But in terms of what is actually applied, an intended configuration with 
defaults values can be mapped into an equivalent intended configuration 
with all implicit defaults replaced by the equivalent explicit values.  
What gets applied  in the device should be the same in both cases.

>   It is certainly possible to set default values in
Yes, of course.

> in practice to the applied config
I think that I disagree on this point.

The intended configuration is only complete when you consider its 
associated YANG schema and implicit default values.

> avoided. Hence I would question the need for a per-leaf state and advocate
I think that the whole purpose of the opstate requirements is that the 
operators effectively want to be able to track on a per leaf element 
rather than a per-server element.

I might be wrong, but I think that the design being consider by the 
opstate operators is along the lines of an eventual consistent model.  
E.g. something along the lines of:

At any point in time the operator knows what configuration state they 
would a device to be in (i.e. the intended configuration).
Whenever the client changes their intended configuration, they push down 
the updated intended configuration to the device.
The device starts applying the change (using the specified error 
handling semantics)
They register for notification for changes to all intended/applied 
config and derived state on the device so that they have a feedback 
mechanism to know whether the configuration is being successfully applied.

> is the same that already exists, just skip it in the server.
The case I was considering is where the server has no value applied 
(e.g. for an optional feature) rather than the default value.

> config to contain also the default values. That doesn¹t seem necessary. I
I can't see how you can have default values apply only to the applied 
configuration and not the intended configuration.  The intended 
configuration cannot be correctly interpreted if you don't also consider 
the default values specified in the schema.

> counter-productive. Assume the client sets an intended state of two values
Tracking intended vs applied on a per leaf basis is the essence of the 
opstate requirements, please lets not rehash these all yet again. :-)

> source while it is actually writing.
No, this is fine, and isn't a problem.  The server will eventually 
converge on a state and the client will be notified of that state. The 
fact that a client might temporarily see a valid state that is 
subsequently undone isn't a problem.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 12 Jan 2016 15:14:39 +0100

> On 12 Jan 2016, at 14:29, Robert Wilton <[email-masked]> wrote:
> 
> 
> 
> OK, so in essence you are proposing that the intended and applied config nodes have different schemas, in that any default values in the YANG schema only apply to the leaves in the intended config schema and not the applied config schema.  As such, because the applied config schema doesn't haven't default values in the schema it must report all values it is using.

Yes. Whether it makes the schema different depends on the definition of schema. In any case, this differentiation wouldn't be noticeable in YANG modules, it is just semantics of intended and applied configurations.

> 
> But if we were to do this, then I'm not entirely sure why you would keep default values in the intended configuration schema (other than backwards compatibility, and potentially for documentation purposes)?

Defaults have to be documented somewhere so that the client knows what values will be used if they are omitted in the intended config.

> 
> Personally, I think that default values are a useful way to simply and shorten the configuration data set, but arguably this can apply equally to both intended and applied configuration.  For me, the main argument for not allowing default values in the applied config is because of the difficultly for the server to differentiate between a leaf that hasn't been applied at all (e.g. for an optional feature) and the same leaf that is applied with the default value.

Defaults serve two purposes: (1) they save clients from writing all parameters explicitly, and (2) they make server's output more readable by suppressing some (presumably uninteresting) information. Requiring all defaults explicitly in applied config goes partly against #2, but on the other hand it is also important to be able learn (somehow) what values the device really uses.

> 
> I.e. I think that the applied configuration nodes already solve the problem of the device defaults not matching the configuration schema, since these differences should be reported in the applied configuration (or alternatively as deviations to the original schema that is being supported).

Only in an ideal world. Defaults may sometimes change, and what's in the data model may not be in sync what the device really uses. Not being able to learn the truth is frustrating.

Lada

> Yes.  But if the default values that are actually being used by the device don't match the implicitly expected value from leaf defaults then these should be reported in the applied configuration with the value set to the actual applied value.
> 
> Thanks,
> Rob
> 
> 
>> 
>> Lada
>>

---

From: Benoit Claise <[email-masked]>
Date: Tue, 12 Jan 2016 16:38:48 +0100

Lada,
> For neither of them applied configuration seems to add any value.
Fair enough. However, it doesn't entail that the opstate has no value 
for anybody.
Some operators have spoken loud and clear.

Regards, Benoit

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 12 Jan 2016 16:51:08 +0100

> On 12 Jan 2016, at 16:38, Benoit Claise <[email-masked]> wrote:
> 
> Lada,
> Fair enough. However, it doesn't entail that the opstate has no value for anybody.

Sure, what I wrote was: For many systems or services, asynchronicity is not an option, or isn't important.

Lada

---

From: Gert Grammel <[email-masked]>
Date: Tue, 12 Jan 2016 16:02:52 +0000

>server is expected to use.
No disagreement. The only point is whether default values *shall* or *may*
be provisioned via intended config
>I think that I disagree on this point.
OK, let’s keep this point for a further discussion.
>associated YANG schema and implicit default values.
True, but it doesn’t mean that the client which holds the complete
intended config has to explicitly push everything down to the server.
Again this is the distinction between *may* and *shall* and worth a
discussion. 
>rather than a per-server element.
The point has been that the client wants to know whether it’s intended
configuration has been applied. This information is missing from the old
asynch model and hence creates uncertainty. Any read attempt of the client
related to applied configuration may fall into a transition period and is
only a snapshot of a wobbly state.
Essentially we are discussing where to put a dirty-flag. In my (less
granular) view it is prudent to put the dirty-flag high up in the tree and
keep it true until the state change completed (irrespective if it was
successful or not). Another way is to keep a very granular view and stick
the dirty flag on all leafs that are supposed to be touched by the
intended state and cleaning them up after *all* intended config has been
attempted to apply. A bit of more hassle from an implementers point of
view, and I am not sure if it is worth.
 
>would a device to be in (i.e. the intended configuration).
OK
>the updated intended configuration to the device.
OK, from now on the client is blind about the applied state of the server
unless it can guess it from e.g. Derived state and so on.
>The device starts applying the change (using the specified error
OK
>mechanism to know whether the configuration is being successfully applied.
OK, so from now on the client knows that the new intended config is
active. It also means that the client can reliably read applied state.
>(e.g. for an optional feature) rather than the default value.
Whatever value the server applied (none or default) shouldn’t be an issue
unless the intended config imposes a certain value. I guess you had in
mind on how to deal with the opstate of a non-value. Pushing the opstate
higher up the tree would probably address the issue (see above).
>the default values specified in the schema.
Given that servers nowadays cover a huge amount of functionality where
often only a subset is required by a client, I would consider quite a bit
of default values irrelevant for the scope of the client. If e.g. You plan
to use a router that supports VPLS, but you do not care since it is not
needed, those defaults can just be left to the server.
>opstate requirements, please lets not rehash these all yet again. :-)
See above, you can do this but it’s quite a bit of work. The requirement
stems from a time where a client wasn’t able to figure out whether a leaf
has been changed already or was about to change in future. To me it
appears more of an implementation choice.
>subsequently undone isn't a problem.
I’d seriously question those two statements. When the server is updating
its applied state you have at best a 50% chance to read the correct value.
What meaningful conclusion is the client supposed to take from that? Out
of desperation, current implementations use that ‘trick’ by polling
various leaf values to figure out whether an intended state has already
been applied or not. However such intensive polling cycles are no more
needed once a server notifies the client upon completion of intended state
application. 
If the client chooses to read applied state during that period, it is at
least aware that the state is just a temporary snapshot.

---

From: Robert Wilton <[email-masked]>
Date: Tue, 12 Jan 2016 17:36:43 +0000

> be provisioned via intended config

I meant "May".

> Again this is the distinction between *may* and *shall* and worth a
Yes, I think that it is "may".

> only a snapshot of a wobbly state.
Yes, and that isn't a problem.

The intended config is updated, and then the applied config state 
converges on the intended config.

> view, and I am not sure if it is worth.

Having a single dirty flag doesn't really meet the requirements (as 
stated in the requirements draft).  It means that config reads from the 
server are unavailable (or stale) whilst a config change is in progress 
(which could take a long time).

Instead, the operators requirement is that the tracking is done on a per 
leaf basis (e.g. like the OpenConfig YANG models on 
https://github.com/openconfig/public), I believe with the expectation 
that the server should update the leaves as the configuration is being 
applied.  I.e. if the server had processed half the configuration from a 
1 million entry config then ideally it should be updating the applied 
configuration leaves as it goes so that the client knows what is 
currently programmed at that particular point in time.  In the ideal 
world the server should always be telling the client what configuration 
is applied at that specific point in time that the request is processed.

> unless it can guess it from e.g. Derived state and so on.
No, the client can read the current state from the server at any point 
in time, and ideally it should reflect the configuration at that point 
in time (even if it is half way through a config request).

> active. It also means that the client can reliably read applied state.
The changes to the individual leaves can be notified independently as 
they change state.  The client can reliably read the applied state at 
any time.

> unless the intended config imposes a certain value.
It is an issue because the client wants to know what configuration a 
device is actually running.  Ideally it wants a given device to exactly 
implement the configuration that they are requesting.  They don't really 
want lots of random device specific values that will require custom 
client code to handle.

>   I guess you had in
YANG already takes care of this, as per RFC6020bis section 7.6.1, only 
the defaults that are in scope for your configuration are relevant.

> appears more of an implementation choice.
I disagree.  The requirements are from the perceived deficiencies in 
NETCONF/YANG that need to be fixed for operators to use YANG effectively.

> least aware that the state is just a temporary snapshot.
The client doesn't poll.  Instead it registers for notification of 
changes and receives a continuous stream of updates.

At any point in time, the value read by the client is correct, it just 
happens that the system can be thought of being in the state of constant 
flux.

E.g. the global BGP routing table is constantly changing, but that 
doesn't mean that you can't take a useful snapshot of it, or monitor how 
it is changing.

Thanks,
Rob

---

From: Gert Grammel <[email-masked]>
Date: Wed, 13 Jan 2016 11:33:04 +0000

On 2016-12-01 18:36, "Robert Wilton" <[email-masked]> wrote:

>>>>>>>>>>>>>> On 11 Jan 2016, at 14:25, Gert Grammel
>>>>>>>>>>>>>><[email-masked]>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Lada,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The requirement says:
>>>>>>>>>>>>>>             failed, nor deferred due to absent hardware)
>>>>>>>>>>>>>>then
>>>>>>>>>>>>>> the
>>>>>>>>>>> has been fully synced). I would find it interesting if some
>>>>>>>>>>>edits
>>>>>>>>>>> are
>>>>>>>>>> such an entry stays (forever) in applied config even after it
>>>>>>>>>>has
>>>>>>>>>> been
>>>>>>>>>> deleted from intended.
>>>>>> I think this is not much different from default values. Leaves and
agreed: default values *may* be provisioned via intended config

>Yes, and that isn't a problem.
Whether it is a problem depends on what for the information is used by the
client. If the client periodically checks values to monitor the progress
of an intended config application, it wouldn’t be a problem. This is what
is actually done since there is a lack of information whether the intended
configuration is fully applied.
However if the client is collecting leaf information in order to derive an
action, it has to rely on the state. So here there is information needed
that can be relied upon.
We are discussing whether that information should be on a tree or a leaf
level. My preference goes for a higher level because it is more reliable
to implement, but as with all preferences there needs to be consensus.
If one  implements state on a leaf level there are 3 cases to consider:
A) rollback-on-error: All leafs touched in the config have to remain
‘dirty’ (see below for definition) until the last value has been applied
or the rollback is finished. Then in a second step the dirty flag is
removed.
B) stop-on-error: first all leafs supposed to be touched by the intended
config are marked, then config is applied and the ‘dirty’ flag can
immediately be removed from the leafs. However if an error occurs the
dirty flag of all subsequent leafs has to be cleared since there is no
config in progress anymore
C) continue on error: mark all affected leafs with the dirty flag and
clear it after leafs have been configured (successfully or non-sucessfully)

I fear that we’ll eventually end up with something like an orphan control
daemon in the server to clean up all the forgotten states, rendering the
fine level of state less useful than it appear.

>(which could take a long time).
Sorry, I was using a sloppy term without definition. I meant ‘dirty’ flag
is a marker indicating that a configuration application is still running.
Technically such a flag could be sticked at any point in the tree
including leafs. In other words, a client can read whatever and whenever
it wants. ‘dirty’ indicates that the attribute value cannot be relied
upon, since a configuration is ongoing. Hope this clarifies that there is
no unavailable or stale period for reading as stated in the requirements.

>in time (even if it is half way through a config request).
While we are in agreement that the client can poll the applied state at
any point in time (see above), it is not obliged to do so given that the
server sends an indication when the config is applied.
I also realized that I didn’t explain sufficiently what i meant with
‘blind’. Take the case of 2 different implementations of a server applying
configuration:
A) first set the new value in the internal database (aka. applied config)
and then program the HW.
B) first program the HW and then reflect the state in the internal database

Say the client’s intention is to change a leaf value from X to Y. After
configuring, the client is reading the leaf value between the first and
the second write:
A) will report Y, but still runs with X
B) will report X, but runs already with Y

After successful application (after step 2), both implementations will
consistently execute in HW and report the same value in the applied
config. However the client can’t rely on the information it reads in the
meantime - that’s what I meant with ‘blind’. If the only purpose is to
monitor the progress of applying the intended state, this level of
information is good enough, but I would argue that there are less intense
mechanisms to do so.
If the client truly needs to rely on the information read, it has to wait
for completion.
>they change state.
See discussion above about rollback-on-error etc. requiring a careful
definition on when to send notifications.
> The client can reliably read the applied state at
>any time.
No agreement here. As discussed above: "A client can read applied state at
any time, but it can’t reliably assume it being applied before completion
of an ongoing configuration”.
>want lots of random device specific values that will require custom
I think we are in agreement that the client *may* configure default values
to address such cases.
>>>> config to contain also the default values. That doesn¹t seem
>>>>necessary.
>>>> I
In other words, all defaults that are not in scope remain as set by the
supplier of the Server and will not be changed by the client. Looks we are
in agreement.
>NETCONF/YANG that need to be fixed for operators to use YANG effectively.
We agree that "Tracking intended vs applied on a per leaf basis” is a
requirement, but it appears to me that we have different ideas on the
interpretation of ‘applied' during the transition period. In an
implementation draft I would consider something along those lines:

If a server completed the application of intended config (with or without
errors), the applied config SHALL reflect the actual configuration in
force. All leaves where the value of intended == applied are in the state
"intended = applied”. All leaves with intended != applied are in error
(tbd)

If a server is in the process to apply configuration, the reported applied
value MAY NOT reflect the actual configuration in force. Then the fact
that the values of intended equal those of applied SHOULD NOT be taken as
reliable indication that the intended configuration is actually in force.
Likewise a value of intended != applied SHOULD NOT be taken as reliable
indication that the intended configuration is not in force.

NOTE: whether the applied=intended state needs to be coded on a
per-leaf-level or higher up in the tree is a separate issue.

---

From: Robert Wilton <[email-masked]>
Date: Wed, 13 Jan 2016 14:37:32 +0000

Hi Gert,

I'm wondering if all the discussion here would be more appropriate in 
the context of a specific solution draft and hence best deferred until a 
overall solution approach has been chosen?

Otherwise, if there is a specific proposal to change the requirements 
draft then could that be clarified please so that we can focus on that.

Thanks,
Rob

---

From: Gert Grammel <[email-masked]>
Date: Wed, 13 Jan 2016 15:51:00 +0000

Rob,

I think Ken already asked the question and we both responded that we see
this discussion in the scope of a solutions draft rather than related to
requirements. It is still the case I guess.

Gert

On 2016-13-01 15:37, "Robert Wilton" <[email-masked]> wrote:

>>>>>>>>>>>>>>>>              node has been successfully applied to the
>>>>>>>>>>>>>>>>server
>>>>>>>>>>>>>>>> (e.g. not
>>>>>>>>>>>> Using applied config for system-controlled entries would
>>>>>>>>>>>>require
>>>>>>>>>>>> that
>>>>>> If the client has an indication whether the configuration
>>>>>>application
>>>>>> is
>>>>>> finished (with or without errors), it knows that the applied config
>>>>>>is
>>>>>> now
>>>>>> still in progress, a rollback may still happen. That would
>>>>>>invalidate
>>>>>> the

---

From: Robert Wilton <[email-masked]>
Date: Mon, 18 Jan 2016 17:34:55 +0000

>          always present in this list, whether they are configured or
Having two description statements doesn't seem to be the end of the 
world.  Am I also right in presuming that the description-oper statement 
would only be need in a few areas (i.e. for system-controlled nodes)?

If we were able to align the interface config and state trees that would 
probably be a fairly significant step towards unifying the YANG models 
developed by OpenConfig and those being developed in IETF.

Rob