Subject: Re: [netmod] [Netconf] What should a server response be? - depending on NP-containers

From: Mahesh Jethanandani <[email-masked]>
Date: Fri, 19 Aug 2016 19:04:10 -0700

Moving the thread from netconf to netmod.

Will the authors pull 6020bis back into the WG to reach the rough consensus?

> On Aug 17, 2016, at 2:13 AM, Martin Bjorklund <[email-masked]> wrote:
> 
> Hi,
> 
> I have read this long ML thread twice now, and I agree with Andy that:
> 
> 1)  We should not / cannot make design changes in an errata or late in
>    AUTH48; in order to do this we need to pull the document back to
>    the WG and reach (rough) consensus on the behavior (note btw that
>    this thread is currently in NETCONF, it really should be NETMOD).
> 
> 2)  Since servers MAY delete NP-containers in some cases, clients can
>    easily handle NP-containers by using "merge" on them.
> 
> 
> I also agree with Jason that ideally the server should never fail on
> any kind of operation on an NP-container, regardless of current state
> and requested operation.  (But again, this is not a simple
> clarification of the current text.)
> 
> 
> And to answer the original question, I think the server that first got
> a request to create the empty NP-containers and then a request w/
> operation "none" is not correct when it fails with a "data-missing"
> error.  There is no text in 6241 or 6020 that supports this behavior.
> 
> 
> /martin
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

Mahesh Jethanandani
[email-masked]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 20 Aug 2016 10:29:22 +0200

Here is what I wrote on Thu, 16 Jun 2016 14:49:00 +0200:

: It is possible that people will find more bugs while this I-D sits in
: the RFC editor queue. My idea is to treat them pretty much in the way
: we treat errata of published RFCs (they need to be clearly written up,
: discussed on the list, there needs to be agreement on the bug and the
: proposed fix). If we get pre-publication errata with consensus, I hope
: we can address them during the editing/auth48 stage so we do not have
: to post an RFC with already known defects. Does this make sense to
: you?

As document shepherd, I believe there is no strong agreement on the
problem and there is no concrete proposal with strong consensus for a
modification of the document (which is in AUTH48). In fact, there has
been no defect description and proposed bug fix at all on the NETMOD
mailing list.

/js

On Fri, Aug 19, 2016 at 07:04:10PM -0700, Mahesh Jethanandani wrote:
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Sat, 20 Aug 2016 09:44:11 -0700

Hi,

This text in sec 7.5.8, para 2 is wrong:

   If a container does not have a "presence" statement and the last
   child node is deleted, the NETCONF server MAY delete the container.

This text in 6.4.1 is correct, which implies deleting an N container occurs
when its non-NP container
ancestor is deleted.

   If a node that exists in the accessible tree has a non-presence
   container as a child, then the non-presence container also exists in
   the tree.

This text in 7.5.7 covers the intended external behavior on an NP-container
better than the text in 7.5.8:

   If a non-presence container does not have any child nodes, the
   container may or may not be present in the XML encoding.

(BTW, why isn't this text using RFC 2119 terms like sec. 7.5.8?)

I propose that the cited text in sec. 7.5.8 be removed as a clarification.
It is a redundant (incorrect) restatement of the cited text in 7.5.7.

Andy

---

From: Vladimir Vassilev <[email-masked]>
Date: Sat, 20 Aug 2016 19:18:42 +0200

> been no defect description and proposed bug fix at all on the NETMOD
Hello,

I have strong objection to the text proposed as solution to issue #41:

6.4.1 Xpath Context:

The description of the issue itself is:

     Y41: clarification of "must" in NP-container <<Y41>>

I believe the 5 mails in the 
http://www.ietf.org/mail-archive/web/netmod/current/msg10015.html did 
not address all the negative consequences of such change in the rules 
for evaluation of validation statements regarding non-presence 
containers and the solution that was taken is not acceptable for the 
following reasons:

[1] the proposed text is not a clarification as indicated in Y41 but a 
backward incompatible change of the YANG validation statement evaluation 
rules.

[2] the clarification introduces further confusion for models like NACM 
where non-presence containers are used for access control and their 
explicit creation and deletion is the only sane way to enforce the 
configured rules. Always present non-presence containers that MAY be 
auto deleted by servers ... how will this work exactly?

[3] the proposed text leads to increased processing of large number of 
validation checks which is very unlikely to bring real value to YANG. 20 
non-presence containers with must statements per interface in 
96-interface switch is already heavy Xpath evaluation task. A task that 
in YANG 1.0 was not necessary if the containers were not explicitly created.

[4] the proposed text leads to less flexibility and excludes certain 
practical validation models e.g. 
https://www.ietf.org/mail-archive/web/netconf/current/msg11609.html

[5] the proposed text inflicts problems 1-4 and its clarification effect 
is not working for me.

I have a concrete proposal for a solution on the issue - remove the 
"non-presence containers MAY be deleted automatically" text from YANG 
1.1 instead of opening Pandora's box:

Instead of building further the pipe dream of non-presence containers 
that MAY be deleted automatically I propose that flexibility removed 
from YANG 1.1. All non-presence containers have to be created explicitly 
and the validation statements must be evaluated only for explicitly 
created containers (so long there is no change from YANG 1.0) and these 
containers MUST be deleted explicitly (replacing the "MAY be deleted 
automatically" YANG 1.0 optimization attempt which is the origin of the 
pipe dream) as one would logically expect. It is semantical meaning that 
is not present not the container which still has its usage giving 
structure to the data and especially in cases like NACM and validation 
statements where without certain explicit create/delete rules things get 
really confusing.

The concrete proposal in form of a patch to RFC6020 sent in this e-mail 
to the netconf mailing list: 
https://www.ietf.org/mail-archive/web/netconf/current/msg11637.html 
There will be even more obsoleted clarification text that will be 
irrelevant if the concept change is applied to YANG 1.1

Vladimir

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 22 Aug 2016 11:25:50 +0200

> I have strong objection to the text proposed as solution to issue #41:

Dear Vladimir Vassilev,

please note that we YANG 1.1 is in AUTH48 state, that is YANG 1.1 has
passed WG last call, IETF last call, and IESG approval. In other
words, we are way beyond the state in the IETF process where we
discuss the resolution of individual issues. As far as I recall from
the logs, issue #41 was closed about two years ago.

/js

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 22 Aug 2016 12:25:44 +0200

Dear Juergen Schoenwaelder,

> the logs, issue #41 was closed about two years ago.

I wrote my proposal for errata as if the RFC was published hoping it 
would  gain agreement on the bug and the proposed fix and hopefully get 
addressed during the editing/auth48 stage so we do not have to post an 
RFC with already known defects. I interpreted your mail as invitation 
for people with opinion on the issue to post their concrete proposals 
which is what I did.

> been no defect description and proposed bug fix at all on the NETMOD
Why is the "defect description and proposed bug fix" I propose not a 
candidate for consideration for a AUTH48 change?

Can someone with alternative opinion comment on the issues [1-4] I list. 
I really hope I am wrong and those are not as significant problems as it 
currently seems to me.

On 08/20/2016 07:18 PM, Vladimir Vassilev wrote:
> backward incompatible change of the YANG validation statement 
> evaluation rules.
>
> [2] the clarification introduces further confusion for models like 
> NACM where non-presence containers are used for access control and 
> their explicit creation and deletion is the only sane way to enforce 
> the configured rules. Always present non-presence containers that MAY 
> be auto deleted by servers ... how will this work exactly?
>
> validation checks which is very unlikely to bring real value to YANG. 
> 20 non-presence containers with must statements per interface in 
> 96-interface switch is already heavy Xpath evaluation task. A task 
> that in YANG 1.0 was not necessary if the containers were not 
> explicitly created.
>
> [5] the proposed text inflicts problems 1-4 and its clarification 
> effect is not working for me.
>
> from YANG 1.1. All non-presence containers have to be created 
> explicitly and the validation statements must be evaluated only for 
> explicitly created containers (so long there is no change from YANG 
> 1.0) and these containers MUST be deleted explicitly (replacing the 
> "MAY be deleted automatically" YANG 1.0 optimization attempt which is 
> the origin of the pipe dream) as one would logically expect. It is 
> semantical meaning that is not present not the container which still 
> has its usage giving structure to the data and especially in cases 
> like NACM and validation statements where without certain explicit 
> create/delete rules things get really confusing.
>
> The concrete proposal in form of a patch to RFC6020 sent in this 
> e-mail to the netconf mailing list: 

Vladimir

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 22 Aug 2016 13:56:28 +0200

> backward incompatible change of the YANG validation statement evaluation 

The charter doesn't limit YANG 1.1 to clarifications. The adopted
solution addresses ambiguous cases like this:

container top {
  must "foo > 42";
  leaf foo {
    type uint8;
  }
  leaf bar {
    type boolean;
    default true;
  }
}

If "top" doesn't exist, then according to sec. 7.6.1 of RFC 6020, the
default value of "bar" is in use, so somehow the NP-container "top"
should also be in use. The question then arises whether the "must"
constraint applies or not.

> auto deleted by servers ... how will this work exactly?

IMO the problem is in this auto-deletion option. I suspect my example
above may also be unclear from the NACM point of view - do the rules set
for "top" apply if "top" hasn't been explicitly configured?

> in YANG 1.0 was not necessary if the containers were not explicitly

Some implementations did it anyway and some didn't, which was considered
a problem.

> https://www.ietf.org/mail-archive/web/netconf/current/msg11609.html

Well... If the goal it to prevent np2_leaf from appearing when np1_leaf
is configured, then the most natural solution is to put a must statement on
np1_leaf:

leaf np1_leaf {
  type string;
  must "not(../../np2/np2_leaf)";
}

Or if you insisted on having it on the "np1" container, you can do this:

  must "not(np1_leaf) or not(../np2/np2_leaf)";

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 22 Aug 2016 14:56:18 +0200

> constraint applies or not.
This example is based on the bug I propose to be fixed. If you looked at 
the patch I propose in 
https://www.ietf.org/mail-archive/web/netconf/current/msg11637.html sec. 
7.1.6 of RFC 6020 is modified:

---
OLD:
7.6.1.  The leaf's default value

    The default value of a leaf is the value that the server uses if the
    leaf does not exist in the data tree.  The usage of the default value
    depends on the leaf's closest ancestor node in the schema tree that
    is not a non-presence container:

NEW:
7.6.1.  The leaf's default value

    The default value of a leaf is the value that the server uses if the
    leaf does not exist in the data tree.  The usage of the default value
    depends on the leaf's closest ancestor node in the schema tree:
---
The patch removes the "MAY be deleted" text and all artifacts resulting 
from poor attempts to cover the logical discontinuity it inflicts on 
YANG. It is the only justification of all special case clarifications 
regarding non-presence containers which are never going to be enough to 
cover all private cases.
> for "top" apply if "top" hasn't been explicitly configured?
+1 I do not know either. NACM is affected in a bad way. The "MAY be 
deleted" text was not that bad since one would assume sane servers would 
not auto delete data nodes which already have configured NACM rules.
> Some implementations did it anyway and some didn't, which was considered
The implementations that did it anyway were not conforming to YANG 1.0 
since non-presence containers were not part of the configuration tree by 
default unless they were explicitly created. By introducing the Y41 
"clarification" now they are conforming which is the only benefit at the 
cost of limited validation expression flexibility, higher validation 
workload, broken NACM and probably some other issues we are about to 
discover.
>    must "not(np1_leaf) or not(../np2/np2_leaf)";
Yes there are workarounds but the solution I propose is to remove the 
cause leading to the limitation and the need to use workarounds. If it 
is too late at least there will be a trace in the netmod mailing list 
indicating the issue was recognized before the publication of the RFC 
and something to consider in YANG 1.2

Vladimir

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 22 Aug 2016 15:36:51 +0200

> On 22 Aug 2016, at 14:56, Vladimir Vassilev <[email-masked]> wrote:
> 
> This example is based on the bug I propose to be fixed. If you looked at the patch I propose in https://www.ietf.org/mail-archive/web/netconf/current/msg11637.html sec. 7.1.6 of RFC 6020 is modified:
> 
> ---
> OLD:

This would effectively remove the distinction between presence and non-presence containers, and I am not in favour of doing so. In any case, this is not something to introduce in the AUTH48 stage.

> ---
> The patch removes the "MAY be deleted" text and all artifacts resulting from poor attempts to cover the logical discontinuity it inflicts on YANG. It is the only justification of all special case clarifications regarding non-presence containers which are never going to be enough to cover all private cases.
>> 
> +1 I do not know either. NACM is affected in a bad way. The "MAY be deleted" text was not that bad since one would assume sane servers would not auto delete data nodes which already have configured NACM rules.
>> 
> The implementations that did it anyway were not conforming to YANG 1.0 since non-presence containers were not part of the configuration tree by default unless they were explicitly created.

This wasn't clear in cases like my example.

> By introducing the Y41 "clarification" now they are conforming which is the only benefit at the cost of limited validation expression flexibility, higher validation workload, broken NACM and probably some other issues we are about to discover.

I don't agree with this conclusion. NACM probably needs to be updated anyway.

> Yes there are workarounds but the solution I propose is to remove the cause leading to the limitation and the need to use workarounds. If it is too late at least there will be a trace in the netmod mailing list indicating the issue was recognized before the publication of the RFC and something to consider in YANG 1.2

I don't see it as a workaround and I don't think there is ANY limitation whatsoever due to this. The rules are now clear, which is a good thing.

Lada

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 22 Aug 2016 17:59:37 +0200

> This would effectively remove the distinction between presence and non-presence containers, and I am not in favour of doing so. In any case, this is not something to introduce in the AUTH48 stage.
The distinction in the context of the primary reason for introduction of 
the "presence" statement indicating explicit semantic meaning of the 
presence of a container is still there. However I do not see the value 
of distinction in terms of creation and deletion. And it is that 
distinction introducing the bulk of clarification texts. The now 
mandatory evaluation of all must statements in non-presence containers 
is going to introduce very high and unnecessary processing load. I gave 
an example with 96 interfaces switch with 20 non-presence containers in 
/interfaces/interface. How about top level SDN manager?
> I don't agree with this conclusion. NACM probably needs to be updated anyway.
Which of the 3 issues pointed in the conclusion you don't agree with and 
why {1. limited validation expression flexibility, 2. higher validation 
workload, 3. broken NACM}? Difficult to not agree with 2. And 1 is 
predetermined from the fact of the reduced entropy attributed to a 
non-presence container - namely its existence now is determined by the 
existence of its parent (which reduces flexibility in a very certain way).
> I don't see it as a workaround and I don't think there is ANY limitation whatsoever due to this. The rules are now clear, which is a good thing.
I have expressed my concerns. I did not find anything addressing the 3 
issues above on the list and I thought it was worth posting something in 
the context of the ongoing discussion. There may be different opinions 
of the significance of the issues but they do exist.

Vladimir

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 22 Aug 2016 18:10:12 +0200

On Mon, Aug 22, 2016 at 05:59:37PM +0200, Vladimir Vassilev wrote:

> Which of the 3 issues pointed in the conclusion you don't agree with and why
> {1. limited validation expression flexibility, 2. higher validation

Can someone explain to me what exactly breaks NACM? An example would
help me.

/js (as contributor)

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 22 Aug 2016 18:15:50 +0200

> Can someone explain to me what exactly breaks NACM? An example would
"It is absolutely legal to configure "update" rights to /interfaces to a 
group of users reserving the "create" right to the superuser. How is 
this scenario handled by servers ignoring empty non-presence 
containers?" (this is excerpt from an earlier post on that thread)

If a non-presence container always exits in YANG 1.1 this usage example 
is not possible.

---

From: Andy Bierman <[email-masked]>
Date: Mon, 22 Aug 2016 09:32:14 -0700

On Mon, Aug 22, 2016 at 9:15 AM, Vladimir Vassilev <[email-masked]
> wrote:

>>> why
> group of users reserving the "create" right to the superuser. How is this
> scenario handled by servers ignoring empty non-presence containers?" (this
> is excerpt from an earlier post on that thread)
>
> If a non-presence container always exits in YANG 1.1 this usage example is
> not possible.
>
>
I have to agree that NACM does not have any special rules for NP containers.

I also find the concept of "has no semantics except to hold child nodes"
to be especially confusing.   The parent of all foo-things is in itself
semantic content,
especially when considering the NP-container used as the target of augments
or the target of a NACM rule.

IMO the only use-case for must-stmt in the container is to validate child
nodes,
so it is non-intuitive that must-stmt is always tested on NP-containers
without
any child nodes.  I think it can be made to work in the code, but it will
cause
customer astonishment.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 22 Aug 2016 18:37:18 +0200

> If a non-presence container always exits in YANG 1.1 this usage example is

Should I read 'ignoring empty non-presence containers' as 'removing empty
non-presence containers (form the XML encoding)'?

Isn't the idea that non-presence container always exits in YANG 1.1
for the purpose of validation, that is in the XPATH context.

Back to your example, what is the client going to update in
/interfaces if /interfaces is empty? Or is the scenario that the group
of users have create and update rights within /interfaces but no
create right on /interfaces?  I am trying to understand what exactly
the situation is that you think causes problems.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 22 Aug 2016 18:45:43 +0200 (CEST)

Vladimir Vassilev <[email-masked]> wrote:
> >> This example is based on the bug I propose to be fixed. If you looked
> >> at the patch I propose in
> >> sec. 7.1.6 of RFC 6020 is modified:
> >>
> >> ---
> >> OLD:
> > This would effectively remove the distinction between presence and
> > non-presence containers, and I am not in favour of doing so. In any
> > case, this is not something to introduce in the AUTH48 stage.
> The distinction in the context of the primary reason for introduction
> of the "presence" statement indicating explicit semantic meaning of
> the presence of a container is still there. However I do not see the
> value of distinction in terms of creation and deletion. And it is that
> is going to introduce very high and unnecessary processing load.

I disagree.  If the model needs to have some semantic validation
rules, the designer is going to put them in a place such that they are
evaluated when the need to be evaluated.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 22 Aug 2016 09:49:05 -0700

On Mon, Aug 22, 2016 at 9:37 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> is

This is an important point.
YANG 1.1 treats default nodes as if they always exist,
but this ifor conceptual XPath evaluation only.
The create and delete operations still fail on a node with a YANG default,
based on whether it actually exists in the implementation.
NP containers are no different than default nodes in this respect.

> the situation is that you think causes problems.

In our NACM implementation there will be a check on node /interfaces.
The operation better be "none" if the user is not allowed to write to
that node.

Consider a file system where /home is an NP-container.
You don't give user 'foo' access to write to /home, just /home/foo.

Andy

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 22 Aug 2016 19:21:06 +0200

> non-presence containers (form the XML encoding)'?
Yes the original post targeted the text: "If a container does not have a 
"presence" statement and the last
    child node is deleted, the NETCONF server MAY delete the container."

There is nothing indicating it is concerning only the XML encoding which 
I believe can be fixed with AUTH48 modification if we agree this is the 
intention.
> for the purpose of validation, that is in the XPATH context.
You have a point. The YANG 1.1 text addresses only the Xpath context:

     "If a node that exists in the accessible tree has a non-presence
     container as a child, then the non-presence container also exists in
     the tree. ".

However now we have introduced a separation between a data tree and 
Xpath context tree which only serves the case where data tree 
non-presence containers are removed from the data tree automatically 
(and not only from the XML encoding as assumed above). Or do you see any 
other justification? It is this logic that creates the confusion and the 
impression YANG 1.1 advocates the NACM unfriendly case of non-presence 
containers being auto deleted from the data tree.
> the situation is that you think causes problems.
Here I might experience revelation I always assumed update on a 
container means you can create children in it but you can not create it 
if it does not exist already if you only have update rights but no 
create rights. There is no text in NACM RFC 6536 detailing what updating 
container means. Are you saying one can only replace values of leafs 
that already exist?

Vladimir

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 22 Aug 2016 19:35:56 +0200

> evaluated when the need to be evaluated.
So designers augmenting /interfaces/interface with non-presence 
container with child leaves should just pick one of the leaves in YANG 
1.1 and put the must statements there instead of the parent container. 
Yes this might work. There is no problem adding extra "../" in the Xpath 
expressions. But what is the justification of all that except the "If a 
container does not have a "presence" statement and the last child node 
is deleted, the NETCONF server MAY delete the container." text?

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 22 Aug 2016 20:12:27 +0200

On Mon, Aug 22, 2016 at 07:21:06PM +0200, Vladimir Vassilev wrote:
> There is nothing indicating it is concerning only the XML encoding which I
> believe can be fixed with AUTH48 modification if we agree this is the
> intention.
> > 
> However now we have introduced a separation between a data tree and Xpath
> context tree which only serves the case where data tree non-presence
> containers are removed from the data tree automatically (and not only from
> the XML encoding as assumed above). Or do you see any other justification?
> It is this logic that creates the confusion and the impression YANG 1.1
> advocates the NACM unfriendly case of non-presence containers being auto
> deleted from the data tree.

What we need is a clear problem statement people can agree on and a
clear proposal how to resolve the problem that people can also agree
on. I think we are not even close to a clear problem statement since
logic often appears circular.

> Here I might experience revelation I always assumed update on a container
> means you can create children in it but you can not create it if it does not
> exist already if you only have update rights but no create rights. There is
> no text in NACM RFC 6536 detailing what updating container means. Are you
> saying one can only replace values of leafs that already exist?

RFC 6536 says:

   o  Create: allows the client to add a new data node instance to a
      datastore.

   o  Update: allows the client to update an existing data node instance
      in a datastore.

For me, this means that create and update are different operations and
in the context of a container, update does not really make any sense
since a container node does not have a value that can be updated.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 22 Aug 2016 20:27:15 +0200 (CEST)

> 1.1 and put the must statements there instead of the parent

Sorry I can't parse this.  What is "leaves in YANG 1.1"?

My point is that must expressions don't just exist w/o any reason for
the sake of theoretical arguments.  If there is a YANG 1.1 model with
a must expression in an NP-container, then there is probably a reason
for it - the designer wants it to be evaluated if its parent exist.
If we change the rules for when must expressions are evaluated, the
module designer will have to figure out some other way to add these
must expressions, and the performance will be the same.  So your
argument that many must expressions can lead to performance issues is
not really a problem in itself.

/martin

> Yes this might work. There is no problem adding extra "../"
> in the Xpath expressions. But what is the justification of all that
> except the "If a container does not have a "presence" statement and
> the last child node is deleted, the NETCONF server MAY delete the
> container." text?
>

---

From: Vladimir Vassilev <[email-masked]>
Date: Mon, 22 Aug 2016 21:41:08 +0200

> Sorry I can't parse this.  What is "leaves in YANG 1.1"?
Well for example the designer wants to add container with multiple leaf 
children (leaves) to /interfaces/interface. Assume there is only 1 of 
the 96 interfaces on a switch that can have inet address for the purpose 
of managing the device:

augment "/if:interfaces/if:interface" {
     container inet {
         must "../name = 'me0'" {
             description
                  "The inet container is only valid for the management 
('me0') interface.";
         }
         leaf address {
             type inet:ip-prefix;
         }
     }
}

So in this example the must expression will be evaluated not only for 
interfaces the user attempts to create 
/interfaces/interface/inet/address but for all those he does not attempt 
to create it. This is not the case in YANG 1.0 and this is example how 
processing of validation expressions increases in YANG 1.1

In this case the designer can copy the must expression to the address 
leaf and it will be evaluated only when the user attempts to create that 
leaf ... however when it is not a single leaf in the container the 
workaround can get trickier.

> for it - the designer wants it to be evaluated if its parent exist.
Well this would be the case in YANG 1.1. In YANG 1.0 the designer wants 
it to be evaluated if the non-presence container itself exists not its 
parent. Now this changes because of the "clarification" from Y41.
> not really a problem in itself.
We are changing the rules from YANG 1.0. Auto evaluation of all 
non-presence containers must statements is now mandatory when their 
parent container exists. I still don't see how there is no problem with 
the increased must statement processing and believe there is 
misunderstanding somewhere.

Vladimir

---

From: Alex Campbell <[email-masked]>
Date: Mon, 22 Aug 2016 22:08:19 +0000

The intention in this case is obviously to evaluate the 'must' statement if
the container contains any values; what would break if we said that

   A non-presence container exists in the data tree if and only if it has
   any children which exist in the data tree.

thus disallowing the existence of empty NP-containers in the data tree?

________________________________________
From: netmod <[email-masked]> on behalf of Vladimir Vassilev <[email-masked]>
Sent: Tuesday, 23 August 2016 7:41 a.m.
To: Martin Bjorklund; [email-masked]
Subject: Re: [netmod] [Netconf] What should a server response be? - depending on NP-containers

Vladimir

---

From: Vladimir Vassilev <[email-masked]>
Date: Tue, 23 Aug 2016 08:42:34 +0200

> thus disallowing the existence of empty NP-containers in the data tree?

The question is where is the misunderstanding.

    the tree."

What does this mean? I believe there is confusion based on "the tree" 
refering not to the data tree but the Xpath context. At least I hoped 
until I realized the text was introduced as a solution to Y41 
'clarification of "must" in NP-container'. That definitely means it 
addresses the must statements in the non-presence containers and it 
means "the tree" as in the data tree.

1. If "then the non-presence container also exists in the tree." is 
refering to the Xpath context and not the data tree (which is not 
obvious since nowhere is the Xpath context introduced as "the tree") 
then the Xpath validation statements in the non-presence container do 
not have to be evaluated. The only effect is that other Xpath 
expressions testing for the existence of the non-presence container will 
return true.

2. If the meaning is that the non-presence container exists virtually in 
the data tree and its pertaining Xpath expressions have to be evaluated 
then there is a problem. Then it is not only that many unnecessary Xpath 
evaluations will be processed but also designers have to make sure their 
Xpath expressions do not fail when the non-presence container is not 
even created (it is not a problem if the user attempts to create it 
empty then at least there is some bad initiative from him asking for 
error message). Like in the example there will be 95 errors for all 
containers not part of the interface named "me0". There can never be a 
valid configuration for YANG 1.1 for this example. If the must statement 
is modified from must "../name = 'me0'" to "../name='me0' or not 
(./address)" then it will work as intended in YANG 1.0 with the overhead 
of 95 unnecessary evaluations.

If the text you propose is added without the current text removed it 
will be even more difficult to understand the clarification.

Vladimir

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 23 Aug 2016 09:33:48 +0200

On Tue, Aug 23, 2016 at 08:42:34AM +0200, Vladimir Vassilev wrote:
> refering not to the data tree but the Xpath context. At least I hoped until
> I realized the text was introduced as a solution to Y41 'clarification of
> "must" in NP-container'. That definitely means it addresses the must
> statements in the non-presence containers and it means "the tree" as in the
> data tree.

My reading is that 'tree' refers to the 'accessible tree' used earlier
in the sentence. The accessible tree itself is defined just above the
quoted sentence. If my reading of the text is correct, then the
obvious clarification would be:

OLD

NEW

   the accessible tree.

/js

---

From: Vladimir Vassilev <[email-masked]>
Date: Tue, 23 Aug 2016 10:10:12 +0200

>     container as a child, then the non-presence container also exists in
So should the must statements defined in the non-presence container 
which is now part of the accessible tree be evaluated or not?

Vladimir

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 23 Aug 2016 10:15:17 +0200 (CEST)

Vladimir Vassilev <[email-masked]> wrote:
> >> until

They are evaluated.  See Section 7.5.3:

   When a datastore is validated, all "must" constraints are
   conceptually evaluated once for each node in the accessible tree (see
   Section 6.4.1).

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 23 Aug 2016 10:17:03 +0200 (CEST)

>    container as a child, then the non-presence container also exists in

I think this is a useful and correct clarification.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 23 Aug 2016 10:25:59 +0200

On Tue, Aug 23, 2016 at 10:10:12AM +0200, Vladimir Vassilev wrote:
> So should the must statements defined in the non-presence container which is
> now part of the accessible tree be evaluated or not?
>

I think we need to carefully dissect things in order to make
progress. The above text is about XPATH contexts and accessible
trees. It is not about what needs to be checked during
validation. This is defined in section 8 and in particular section
8.1.

   The following properties are true in a valid data tree:

   o  All "must" constraints MUST evaluate to "true".

This clearly talks about the data tree. The resolution of issue 42
says:

   2014-07-21 meeting proposal: Clarify that for validation purposes,
   NP containers always exist.

Putting these pieces together, it seems to me that for the purpose of
validation, an NP container is assumed to always exist in the data
tree and its MUST statement is evaluated.

/js

---

From: Vladimir Vassilev <[email-masked]>
Date: Tue, 23 Aug 2016 10:33:52 +0200

>     conceptually evaluated once for each node in the accessible tree (see
Then we have the case I objected to and the example:

YANG 1.0:

YANG 1.1 (replace "../name = 'me0'" with "../name='me0' or not 
(./address)" and process 95 unnecessary Xpath evaluations).

I think this proves the argument that there will be more unnecessary 
Xpath processing. In addition it illustrates how a simple task requires 
ugly patch (the ".. or not (./address)" added to the must expression) 
just to ensure the expression does not fail in the default case where 
the interface is not named "me0" and the user has not even attempted to 
create empty /interfaces/interface/inet container in YANG 1.1.

Vladimir

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 23 Aug 2016 11:09:24 +0200 (CEST)

> augment "/if:interfaces/if:interface" {

This should have been a 'when' expression, not 'must'.  Alternatively,
it should have been a P-container, since obviously the container has
some semantics.  Or alternatively, the must expression should have
been on the address leaf, since this is what you really checked.

/martin

>             description
>                  "The inet container is only valid for the management ('me0')
>                  interface.";
>         }
>         leaf address {
>             type inet:ip-prefix;
>         }
>     }
> }
> 
> Xpath processing.  In addition it illustrates how a simple task
> requires ugly patch (the ".. or not (./address)" added to the must
> expression) just to ensure the expression does not fail in the default
> case where the interface is not named "me0" and the user has not even
> attempted to create empty /interfaces/interface/inet container in YANG
> 1.1.
> 
> Vladimir
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 23 Aug 2016 12:28:23 +0200

> On 23 Aug 2016, at 10:25, Juergen Schoenwaelder <[email-masked]> wrote:
> 

But only if the closest ancestor that is not an NP-container exists, or there is no such ancestor.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 23 Aug 2016 12:45:58 +0200

> On 23 Aug 2016, at 11:09, Martin Bjorklund <[email-masked]> wrote:
> 

An interesting case would be if there is the "when" statement as you suggest, and then also one or more "must" statements. IMO, in this case the "when" statement needs to be evaluated first using the procedure of sec. 7.21.5, and if (and only if) the result is true, then the "must" statements are evaluated. Again, an NP-container is no different from a leaf with a default value.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 23 Aug 2016 12:46:40 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> An interesting case would be if there is the "when" statement as you
> suggest, and then also one or more "must" statements. IMO, in this
> case the "when" statement needs to be evaluated first using the
> procedure of sec. 7.21.5, and if (and only if) the result is true,
> then the "must" statements are evaluated.

Exactly.  This is no different from any other node w/ both 'when' and
'must' (or 'mandatory', or 'min-elements')

/martin

> Again, an NP-container is no
> different from a leaf with a default value.

> >>                 ('me0')

---

From: Vladimir Vassilev <[email-masked]>
Date: Tue, 23 Aug 2016 13:36:17 +0200

> been on the address leaf, since this is what you really checked.
I claim that any example of a must statement in non-presence container 
is affected in a similar way (Has anyone example that proves the 
contrary?). 'when' statements are alternative when the 'must' statement 
depends on data nodes that are not child nodes (which I admit is the 
case in this example). The thing with 'must' expressions in non-presence 
container is they are used in complicated models where 'mandatory' and 
'choice' solutions are not an option and for whatever reason the design 
got to that point the fact is that by introducing the auto-evaluation of 
non-presence containers it is even more complicated to use them. Here an 
example that stops working in YANG 1.1:

augment "/if:interfaces/if:interface" {
     container 2-company-3-crowd {
         must "(a and b and not(c)) or (not(a) and b and c) or (a and 
not(b) and c)" {
         }
         leaf a {
             type string;
         }
         leaf b {
             type string;
         }
         leaf c {
             type string;
         }
     }
}

... to make it work one has to accept to not deny the creation of 
/interfaces/interface/2-company-3-crowd as empty container and modify 
the expression adding ...or (not(a) and not(b) and not(c)).

That said I admit it is not often one needs non-presence container must 
statements. But in the cases one needs them the YANG 1.1 text is not 
helping. It causes all the statements to be processed when no one 
attempts to create them or child nodes residing in them. I find no 
justification for that.