Subject: [netmod] include

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 09:55:37 +0100

Hi,

I am wondering about the following situation:

Module M includes submodule A, and A in turn includes submodule B, but M does NOT include B.

I think this is not supposed to be legal (and so does pyang), but I cannot find any text in RFC 6020 saying that this is not allowed.

So how is it actually?

Thanks, Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Jernej Tuljak <[email-masked]>
Date: Thu, 21 Feb 2013 10:00:25 +0100

It's right there in "7.1.6. The Include Statement":

The "include" statement is used to make content from a submodule
    available to that submodule's parent module, or to another submodule
    of that parent module.

Hence, a submodule may only include parent module's submodules.

Jernej

Dne 21.2.2013 9:55, piše Ladislav Lhotka:
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 21 Feb 2013 10:01:23 +0100

> I think this is not supposed to be legal (and so does pyang), but I cannot find any text in RFC 6020 saying that this is not allowed.

Does M reference anything defined in B?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 10:03:27 +0100

> Does M reference anything defined in B?

No.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 10:10:09 +0100

>   available to that submodule's parent module, or to another submodule

Right, but suppose one doesn't want/need to make the content of B available in M (and in modules importing M).

> Hence, a submodule may only include parent module's submodules.

My reading is that a submodule is assigned to a main module via 'belongs-to'.

Lada

---

From: Jernej Tuljak <[email-masked]>
Date: Thu, 21 Feb 2013 10:20:00 +0100

> Right, but suppose one doesn't want/need to make the content of B available in M (and in modules importing M).

You cannot achieve that. The relationship between a module and a 
submodule is clearly composition, not an association. A submodule is 
just a way to split a large file into multiple smaller ones, nothing 
else. You eventually end up with a single logical file no matter what 
you do.

What you are trying to do would require submodules being treated as 
imports, IMHO.

Jernej

---

From: Jonathan Hansford <[email-masked]>
Date: Thu, 21 Feb 2013 09:24:20 +0000

On 21.02.2013 09:10, Ladislav Lhotka wrote: 

> On Feb 21, 2013, at
10:00 AM, Jernej Tuljak <[email-masked]> wrote:
>> It's
right there in "7.1.6. The Include Statement": The "include" statement
is used to make content from a submodule available to that submodule's
parent module, or to another submodule of that parent module.
Right, but suppose one doesn't want/need to make the content of B
available in M (and in modules importing M).
>> Hence, a submodule
may only include parent module's submodules.
> My reading is that a
submodule is assigned to a main module via 'belongs-to'.

I agree, it is
'belongs-to' that defines the module/submodule relationship, not
'include'. And surely, if a module includes submodule A which itself
includes submodule B, then the module has indirectly included submodule
B.

Jonathan

> Lada
> 
>> Jernej Dne 21.2.2013 9:55, piše Ladislav
Lhotka: 
>> 
>>> Hi, I am wondering about the following situation:
Module M includes submodule A, and A in turn includes submodule B, but M
does NOT include B. I think this is not supposed to be legal (and so
does pyang), but I cannot find any text in RFC 6020 saying that this is
not allowed. So how is it actually? Thanks, Lada -- Ladislav Lhotka,
CZ.NIC Labs PGP Key ID: E74E8C0C
_______________________________________________ netmod mailing list
[email-masked] [1]https://www.ietf.org/mailman/listinfo/netmod [2]
--
> netmod mailing list
[email-masked]://www.ietf.org/mailman/listinfo/netmod

Links:
------
[1] mailto:[email-masked]
[2]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 10:31:02 +0100

On Feb 21, 2013, at 10:20 AM, Jernej Tuljak <[email-masked]> wrote:

> You cannot achieve that. The relationship between a module and a submodule is clearly composition, not an association. A submodule is just a way to split a large file into multiple smaller ones, nothing else. You eventually end up with a single logical file no matter what you do.

Not really. For example, a submodule cannot reference stuff defined in the main module.

> 
> What you are trying to do would require submodules being treated as imports, IMHO.

I don't think so. In fact, the situation I described might be useful as an additional way for structuring the name space.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 10:42:17 +0100

On Feb 21, 2013, at 10:24 AM, Jonathan Hansford <[email-masked]> wrote:

>>> It's right there in "7.1.6. The Include Statement": The "include" statement is used to make content from a submodule available to that submodule's parent module, or to another submodule of that parent module.
> I agree, it is 'belongs-to' that defines the module/submodule relationship, not 'include'. And surely, if a module includes submodule A which itself includes submodule B, then the module has indirectly included submodule B.

This conclusion can hardly be drawn from 6020 text, and I actually think submodule inclusion was not designed to be transitive. For instance, if submodule A includes submodule B, and B includes C, then A cannot refer to C's definitions without including C.

Lada   

>  
> Jonathan
>> Lada
>>> Jernej Dne 21.2.2013 9:55, piše Ladislav Lhotka:
>>>> Hi, I am wondering about the following situation: Module M includes submodule A, and A in turn includes submodule B, but M does NOT include B. I think this is not supposed to be legal (and so does pyang), but I cannot find any text in RFC 6020 saying that this is not allowed. So how is it actually? Thanks, Lada -- Ladislav Lhotka, CZ.NIC Labs PGP Key ID: E74E8C0C _______________________________________________ netmod mailing list [email-masked]://www.ietf.org/mailman/listinfo/netmod
>> --

---

From: t.petch <[email-masked]>
Date: Thu, 21 Feb 2013 10:21:24 +0000

----- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
To: "Jonathan Hansford" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 21, 2013 9:42 AM
Subject: Re: [netmod] include

On Feb 21, 2013, at 10:24 AM, Jonathan Hansford <[email-masked]>
wrote:

>>> It's right there in "7.1.6. The Include Statement": The "include"
statement is used to make content from a submodule available to that
submodule's parent module, or to another submodule of that parent
module.
> I agree, it is 'belongs-to' that defines the module/submodule
relationship, not 'include'. And surely, if a module includes submodule
A which itself includes submodule B, then the module has indirectly
included submodule B.

This conclusion can hardly be drawn from 6020 text, and I actually think
submodule inclusion was not designed to be transitive. For instance, if
submodule A includes submodule B, and B includes C, then A cannot refer
to C's definitions without including C.

<tp>
s5.1
"There MUST NOT be any circular chains of imports or includes.  For
   example, if submodule "a" includes submodule "b", "b" cannot include
   "a". "
so chains are clearly permitted

do
"
   o  For a module to reference definitions in one of its submodules,
      the module MUST include the submodule.

   o  For a submodule to reference definitions in a second submodule of
      the same module, the first submodule MUST include the second
      submodule."

seems clear that A must include C even if B includes C.

Tom Petch

Lada
>>>> Hi, I am wondering about the following situation: Module M includes
submodule A, and A in turn includes submodule B, but M does NOT include
B. I think this is not supposed to be legal (and so does pyang), but I
cannot find any text in RFC 6020 saying that this is not allowed. So how
is it actually? Thanks, Lada -- Ladislav Lhotka, CZ.NIC Labs PGP Key ID:
E74E8C0C _______________________________________________

---

From: Jernej Tuljak <[email-masked]>
Date: Thu, 21 Feb 2013 11:34:23 +0100

> This conclusion can hardly be drawn from 6020 text, and I actually think submodule inclusion was not designed to be transitive. For instance, if submodule A includes submodule B, and B includes C,…

5.1 says that "submodules are partial modules that contribute 
definitions to a  module" (not to a submodule). A submodule that 
contributes nothing is therefore not a submodule. And a submodule may 
only be allowed to contribute to a module via an include statement. I 
can't find text to support a contribution of another kind.

Both "belongs-to" and "include" are required counterparts (a belongs-to 
must always be matched to an include statement in the main module) and 
define a composition relationship. This also eliminates transitivity as 
an option.

A "belongs-to" is not enough to define the relationship between a module 
and a submodule, because a module is never even aware of a submodule's 
existence if include statement is not used. A base unit of definition in 
YANG is a module. There's no such thing as a standalone submodule 
without contributions.

Jernej

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 11:36:51 +0100

> seems clear that A must include C even if B includes C.

Yes, that was exactly my point - and the first bullet above indicates that there is no indirect inclusion for the main module either.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 11:54:42 +0100

On Feb 21, 2013, at 11:34 AM, Jernej Tuljak <[email-masked]> wrote:

> 5.1 says that "submodules are partial modules that contribute definitions to a  module" (not to a submodule). A submodule that contributes nothing is therefore not a submodule. And a submodule may only be allowed to contribute to a module via an include statement. I can't find text to support a contribution of another kind.

It depends on what you mean by "contribute". A submodule can provide type and grouping definitions that are used by other submodules that are included by the main module and contribute data nodes to the model schema. Assuming the same static scoping as for imports, this could work just fine.

> 
> Both "belongs-to" and "include" are required counterparts (a belongs-to must always be matched to an include statement in the main module) and define a composition relationship. This also eliminates transitivity as an option.

This might indeed be the right thing to do but my point is that it doesn't follow from RFC 6020, and the opposite interpretation could IMO make sense, too.

> 
> A "belongs-to" is not enough to define the relationship between a module and a submodule, because a module is never even aware of a submodule's existence if include statement is not used. A base unit of definition in YANG is a module. There's no such thing as a standalone submodule without contributions.

It is more about visibility of definitions.

Lada

---

From: Jonathan Hansford <[email-masked]>
Date: Thu, 21 Feb 2013 11:03:23 +0000

On 21.02.2013 10:34, Jernej Tuljak wrote: 

> Dne 21.2.2013 10:42,
piše Ladislav Lhotka:
> 
>> On Feb 21, 2013, at 10:24 AM, Jonathan
Hansford <[email-masked] [2]> wrote: 
>> 
>>> On 21.02.2013
09:10, Ladislav Lhotka wrote: 
>>> 
>>>> On Feb 21, 2013, at 10:00 AM,
>>>>> Hence, a
submodule may only include parent module's submodules.
>>>> My reading
is that a submodule is assigned to a main module via 'belongs-to'.
>>> I
agree, it is 'belongs-to' that defines the module/submodule
relationship, not 'include'. And surely, if a module includes submodule
A which itself includes submodule B, then the module has indirectly
included submodule B.
>> This conclusion can hardly be drawn from 6020
text, and I actually think submodule inclusion was not designed to be
transitive. For instance, if submodule A includes submodule B, and B
includes C, then A cannot refer to C's definitions without including
C.
> Both
"belongs-to" and "include" are required counterparts (a belongs-to 

> YANG is a module. There's no
such thing as a standalone submodule 

5.1
states "For a module to reference definitions in one of its submodules,
the module MUST include the submodule." That would seem to infer that a
module only needs to include a submodule if it needs to reference
definitions from that submodule. Some submodules may only be referenced
by other submodules rather than by the module itself. Does the module
need to be "aware" of a submodule's existence if it doesn't reference
it?

Jernej

Jernej Dne 21.2.2013 9:55, piše Ladislav Lhotka: 
>>>>>> Hi, I am
wondering about the following situation: Module M includes submodule A,
and A in turn includes submodule B, but M does NOT include B. I think
this is not supposed to be legal (and so does pyang), but I cannot find
any text in RFC 6020 saying that this is not allowed. So how is it
actually? Thanks, Lada -- Ladislav Lhotka, CZ.NIC Labs PGP Key ID:
E74E8C0C _______________________________________________ netmod mailing
list [email-masked]:
[3]//www.ietf.org/mailman/listinfo/netmod
>>>> -- Ladislav Lhotka,
CZ.NIC Labs PGP Key ID: E74E8C0C
_______________________________________________ netmod mailing list
[email-masked]: [4]//www.ietf.org/mailman/listinfo/netmod
_______________________________________________ netmod mailing list
[email-masked] [5]https://www.ietf.org/mailman/listinfo/netmod [6]
-- Ladislav Lhotka, CZ.NIC Labs PGP Key ID: E74E8C0C
_______________________________________________ netmod mailing list
[email-masked] [7]https://www.ietf.org/mailman/listinfo/netmod [8]

Links:
------
[1] mailto:[email-masked]
[2]
mailto:[email-masked]
[3] mailto:[email-masked]:
[4]
mailto:[email-masked]:
[5] mailto:[email-masked]
[6]
https://www.ietf.org/mailman/listinfo/netmod
[7]
mailto:[email-masked]
[8] https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Thu, 21 Feb 2013 05:28:47 -0800

On Thu, Feb 21, 2013 at 3:03 AM, Jonathan Hansford
<[email-masked]>wrote:

> **
> submodule's parent module, or to another submodule of that parent module.
>
> relationship, not 'include'. And surely, if a module includes submodule A
> which itself includes submodule B, then the module has indirectly included
> submodule B.
>
> submodule A includes submodule B, and B includes C, then A cannot refer to
> C's definitions without including C.
>
>  5.1 states "For a module to reference definitions in one of its submodules, the module MUST include the submodule." That would seem to infer that a module only needs to include a submodule if it needs to reference definitions from that submodule. Some submodules may only be referenced by other submodules rather than by the module itself. Does the module need to be "aware" of a submodule's existence if it doesn't reference it?
>
>
Sub-modules are not that intuitive.
They appear to create a hierarchy but that is not true.
If mod-A includes submod-B includes submod-C and C defines top
level definitions (including augment) then a client that imports
mod-A can access the definitions from submod-C.

The hierarchy is only for using definitions.
mod-A cannot directly reference definitions from submod-C without including
it.
Submodules cannot reference any definitions from the main module ever,
which is really not intuitive.

Include-stmt only affects definition resolution within the module.
To the client, submodules are semantically invisible.

Jernej
> Lada

Andy

> submodule A, and A in turn includes submodule B, but M does NOT include B.
> I think this is not supposed to be legal (and so does pyang), but I cannot
> find any text in RFC 6020 saying that this is not allowed. So how is it
> actually? Thanks, Lada -- Ladislav Lhotka, CZ.NIC Labs PGP Key ID: E74E8C0C

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 14:53:08 +0100

> mod-A can access the definitions from submod-C.

Even if mod-A doesn't include submod-C? This could lead to nasty ambiguities, for example if both mod-A and submod-C define a top-level grouping with the same name. From the viewpoint of mod-A it is OK because mod-A doesn't see the C's definition, but an importing module would see both.

Lada

> mod-A cannot directly reference definitions from submod-C without including it.

---

From: Andy Bierman <[email-masked]>
Date: Thu, 21 Feb 2013 06:09:07 -0800

On Thu, Feb 21, 2013 at 5:53 AM, Ladislav Lhotka <[email-masked]> wrote:

> Even if mod-A doesn't include submod-C? This could lead to nasty
> ambiguities, for example if both mod-A and submod-C define a top-level
> grouping with the same name. From the viewpoint of mod-A it is OK because
> mod-A doesn't see the C's definition, but an importing module would see
> both.
>
>
The YANG compiler has to detect this name conflict.
Breaking up a module into sub-modules does not change the rule
that sibling identifiers cannot have the same extended-name.

Example:

module A {
   ...
   prefix a;
   include ipv4;
   include ipv6;

  container ipv4 {
     uses ipv4-group;
  }
  container ipv6 {
     uses ipv6-group;
  }
}

submodule ipv4 {
   belongs-to A { prefix a; }
   include ip;

   grouping ipv4-group {
      uses ip-group;
      ....
    }
}

submodule ipv6 {
   belongs-to A { prefix a; }
   include ip;

   grouping ipv6-group {
      uses ip-group;
      ....
    }
}

submodule ip {
   belongs-to A { prefix a; }
   grouping ip-group {
      ....
    }
}

Lada
Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 15:32:29 +0100

> that sibling identifiers cannot have the same extended-name.

Hmm, so does it matter at all whether mod-A includes submod-C or not? I would say top-level defs from submod-C are not siblings of top-level defs in mod-A *unless* mod-A includes submod-C.

I'd like to hear Martin's opinion, but he is most likely skiing or something. Given how he implemented the parser in pyang, I guess he will say that the main module must include all submodules belonging to it.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 21 Feb 2013 06:56:17 -0800

On Thu, Feb 21, 2013 at 6:32 AM, Ladislav Lhotka <[email-masked]> wrote:

> Hmm, so does it matter at all whether mod-A includes submod-C or not? I
> would say top-level defs from submod-C are not siblings of top-level defs
> in mod-A *unless* mod-A includes submod-C.
>
> I'd like to hear Martin's opinion, but he is most likely skiing or
> something. Given how he implemented the parser in pyang, I guess he will
> say that the main module must include all submodules belonging to it.
>
>
>From my understanding of groupings, nothing in the above example breaks
any rules in RFC 6020.

I'd like to hear from Phil, because he was the one who has the most
experience
with submodules (IMO).  There is supposed to be the same layered information
hiding that groupings provide if they are in modules or submodules.

The YANG compiler only needs to use submod-A definitions within mod-A
if there any statements in modA itself that reference those definitions.
It really doesn't matter if it's groupings, identities, data nodes,
typedefs, etc.

In my interpretation, the belongs-to-stmt says that definitions in this
submodule are added to the namespace for that module.

The include-stmt does not affect the definitions that are exported,
it affected the definitions that are imported.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 21 Feb 2013 15:56:24 +0100 (CET)

Hi,

Ladislav Lhotka <[email-masked]> wrote:
> something.

Not yet, but next week!

> Given how he implemented the parser in pyang, I guess he
> will say that the main module must include all submodules belonging to
> it.

Yes.  So the original question was:

  Module M includes submodule A, and A in turn includes submodule B, but
  M does NOT include B.

  I think this is not supposed to be legal (and so does pyang)

This is correct, it is not legal.  A module must include all its
submodules.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 21 Feb 2013 07:00:43 -0800

> This is correct, it is not legal.  A module must include all its
Where does it say that, because that sure is not how Phil explained
submodules
to the design team way back when it was written.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 16:11:12 +0100

> This is correct, it is not legal.  A module must include all its

Yup. I originally thought the problem was only in the (lack of) specification in 6020 but apparently Andy's view of how submodules work is different. So I am afraid a simple erratum of 6020 won't do.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 21 Feb 2013 07:17:57 -0800

On Thu, Feb 21, 2013 at 7:11 AM, Ladislav Lhotka <[email-masked]> wrote:

> Yup. I originally thought the problem was only in the (lack of)
> specification in 6020 but apparently Andy's view of how submodules work is
> different. So I am afraid a simple erratum of 6020 won't do.
>
>
The include-stmt just affects how prefixed definitions are resolved within
the current sub-module.

There is only 1 module namespace and the belongs-to-stmt indicates
that all top-level definitions in the sub-module are top-level definitions
in the module namespace.

When mod-B imports mod-A it is importing the entire mod-A namespace,
not some hierarchy of sub-module namespaces.

Andy

---

From: Jonathan Hansford <[email-masked]>
Date: Thu, 21 Feb 2013 15:19:55 +0000

On 21.02.2013 14:56, Martin Bjorklund wrote: 

> Hi,
> 
> Ladislav
Lhotka <[email-masked]> wrote:
> 
>> I'd like to hear Martin's opinion,
but he is most likely skiing or something.
> Not yet, but next
week!
> Module M includes
submodule A, and A in turn includes submodule B, but
> M does NOT
include B.
> This is correct, it is not legal. A module must include all
its

If it is not legal, why does 5.1 just state "For a
module to reference definitions in one of its submodules, the module
MUST include the submodule" rather than the explicit "A module MUST
include all its
submodules"?

Jonathan

/martin

netmod
mailing
list

---

From: Phil Shafer <[email-masked]>
Date: Thu, 21 Feb 2013 10:31:30 -0500

Andy Bierman writes:
>I'd like to hear from Phil....

My take is that there's nothing wrong with SM-A including SM-B which
isn't explicitly included by the module.  The "belongs-to" should
suffice to tell the compiler that this is acceptable.  Forcing the
module to grok the division of submodules is not necessary or
convenient.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 16:52:00 +0100

> that all top-level definitions in the sub-module are top-level definitions

This is not what 6020 says.

7.2.2.  The belongs-to Statement

   The "belongs-to" statement specifies the module to which the
   submodule belongs.  The argument is an identifier that is the name of
   the module.

   A submodule MUST only be included by the module to which it belongs,
   or by another submodule that belongs to that module.

   ...

It is the 'include' statement that pulls in the submodule contents. Sec. 7.1.6:

   When a module includes a submodule, it incorporates the contents of
   the submodule into the node hierarchy of the module.  When a
   submodule includes another submodule, the target submodule's
   definitions are made available to the current submodule.

IMO, the interpretation that is closest to the current text is:

The main module needn't include all submodules that are declared as belonging to it. However, only the contents of explicitly included submodules are available in the main module and, by extension, in any modules that import it.

> not some hierarchy of sub-module namespaces.

It imports the contents of the main module, which may be affected by include statements that are present in the main module.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 16:57:30 +0100

> module to grok the division of submodules is not necessary or

But what about the other aspect: If the main module doesn't include SM-B, is the SM-B content part of the module or not?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 21 Feb 2013 08:58:36 -0800

On Thu, Feb 21, 2013 at 7:57 AM, Ladislav Lhotka <[email-masked]> wrote:

> But what about the other aspect: If the main module doesn't include SM-B,
> is the SM-B content part of the module or not?
>
>

I agree with Phil and also the text that Jonathan pointed out.
Here is some more:

   The mandatory "prefix" substatement assigns a prefix for the module
   to which the submodule belongs.  All definitions in the local
   submodule and any included submodules can be accessed by using the
   prefix.

 - para 1 identifies how the module namespace is derived.
 - para 2 says that a sub-module can only be part of 1 module namespace.
 - para 3 seems to be ambiguous because you think it refers to external
   modules that reference definitions from the module with submods.
   Looks to me like the last sentence refers only to referencing definitions
   within the current module.

The intent of sub-modules was that they are implementation mechanisms
used by the server developer -- like code.  If mod-A includes submod-B
and the developers decide submod-B is getting too big and would
like to partition it, this should not affect the upstream sub-modules or
main module.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 21 Feb 2013 18:43:52 +0100 (CET)

Hi,

So I checked the archives and found out that also I can change my
mind: http://www.ietf.org/mail-archive/web/netmod/current/msg01599.html

But I can't find an definitive answer anywhere.  My guess is that we
found the problems that Lada points out below, and changed this so
that the module must list all its submodules.

Note this text from 7.1.6

So when a submodule includes another submodule it works in the same
way as when a module imports another modules - the submodule can refer
to the definitions in the included submodule, but its contents are not
"copied".

More below:

> > is the SM-B content part of the module or not?

Exactly, this is a very relevant question.

I don't think the current text supports the use case that Phil
mentioned, since when SM-A includes SM-B, SM-B's definitions are NOT
incorporated into SM-A.

But then if the above would be allowed anyway, what would this mean:

  module M {
    ....
    include SM-A;
  }

  submodule SM-A {
    ...
    include SM-B;

    augment /b {
      leaf a { ... }
    }
  }

  submodule SM-B {
    ...
    container b;
  }

M doesn't include SM-B, so container /b isn't exported. Whatabout
/b/a?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 21 Feb 2013 18:52:19 +0100

> M doesn't include SM-B, so container /b isn't exported. Whatabout

In my view, neither /b nor /b/a can be part of the module tree. In fact, we can get a very similar issue with imports.

Lada

---

From: Phil Shafer <[email-masked]>
Date: Thu, 21 Feb 2013 13:31:47 -0500

>ntent part of the module or not?

No, since it was never included.  SM-B may be a set of types/grouping/etc
that are not needed for the module, only the submodules.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 21 Feb 2013 19:45:06 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> >But what about the other aspect: If the main module doesn't include SM-B, is
> >the SM-B co
> >ntent part of the module or not?
> 

This would work only for typedefs and groupings.  If the submodule
defines rpcs, they won't be exported through the module.  Same for
notifications, containers, lists, ..., identities, features.

There is nothing in RFC 6020 that explains this behavior, so I don't
think this works.

When I checked the archives (both yang & netmod @ ietf and private
design team emails), the motivation for allowing this was that a
submodule owner was free to split his submodule into more submodules,
w/o having to update the main module.  The interpretation above does
not support this use case.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 21 Feb 2013 10:58:29 -0800

> that are not needed for the module, only the submodules.

Another example:

module AA {
   include B;
   leaf x { type string; }
}

submodule B {
   include C;
   leaf y { type string; }

   uses aa:C-GRP;

}

submodule C {
   leaf z { type string; }

   grouping C-GRP {
      leaf zz {
          when "/aa:z = 'test-one'";
          type string;
        }
    }
}

You are saying that leafs x, y, zz will be visible to an external
module, but not leaf z?

I don't think that is what we implemented.
All 4 leafs will be exported.

What about the when expression in leaf zz that references leaf z?
Is that an error or does that auto-magically export leaf z?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 11:30:01 +0100

> that are not needed for the module, only the submodules.

Yes, this makes good sense, and is in fact in accord with the spec. I think it is absolutely necessary that all submodules that contribute nodes to the schema (+ features etc.) be listed in the main module (or otherwise they would have to be in the server's hello).

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 11:39:58 +0100

> notifications, containers, lists, ..., identities, features.

Right, if they are not included in the main module, then they are not present.

> There is nothing in RFC 6020 that explains this behavior, so I don't

I think this sentence is clear enough:

   When a module includes a submodule, it incorporates the contents of
   the submodule into the node hierarchy of the module.

IMO this implies that if a submodule is not included by the main module, its contents are not incorporated.

> not support this use case.

Yes, but writing a new submodule must not mean that its contents are automatically incorporated in the schema.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 11:56:16 +0100

> Is that an error or does that auto-magically export leaf z?

This is just another race condition wrt 'when', and I would only have to repeat my previous objections against making 'when' constraints a part of schema validation.

In my view, a sound procedure for your example would be something like this:

1. First construct the schema: it contains x, y and zz, but not z because C is not included by AA.

2. Then, an instance document containing zz would be classified as invalid because the 'when' *semantic* condition can never be true.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 12:06:07 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> This is just another race condition wrt 'when', and I would only have to
> repeat my previous objections against making 'when' constraints a part of
> schema validation.

In my view, this is simple.  When the 'when' expression is evaluated,
there is no node /a:z, so it will always fail.

But this is also a reason for requiring modules to list all
submodules.  Otherwise, people will forget to include them, and get
confused by the result.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 12:54:55 +0100

> submodules.  Otherwise, people will forget to include them, and get

Yes, but if a submodule only defines groupings/typedefs that are used by other submodules, then the main module needn't be involved, and there is no risk of confusion or interoperability problems.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 05:37:11 -0800

> submodules.  Otherwise, people will forget to include them, and get

I disagree.
Perhaps the spec is ambiguous, but I don't agree adding busy work for
the YANG module writer is needed.  The YANG compiler can easily
figure out what to do based on the belongs-to-stmt.  IMO, the text
for that clause says definitions in a submodule are part of the module.

I haven't read anything in this thread that would cause me to change any
code.
The spec supports both implementation plans IMO, and it will take a lot
more than an errata to change it.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 14:50:03 +0100

> figure out what to do based on the belongs-to-stmt.  IMO, the text

Do you mean this?

module M {
  /* no include at all */
}

submodule A {
  /* some data nodes */
}

What if the server has the submodule but the client hasn't? Then the contract breaks because each side has a different data model. I also don't like the idea that someone smuggles a submodule into a repository and its contents automatically become part of a module whose author needn't be aware of it. This looks like a security hole.

Lada

> I haven't read anything in this thread that would cause me to change any code.

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 14:51:15 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> figure out what to do based on the belongs-to-stmt.

No one has suggested this.

> more than an errata to change it.

There are three, not two, alternatives:

  1.  Require the module to list all submodules (current pyang
      behavior)

  2.  Recursively incorporate the contents from all submodules into
      the module (current yuma behavior)

  3.  Allow submodules that are not listed in the main module, but do
      not incorporate contents from non-listed submodules

The spec is not crystal clear, but I think that only interpretation
(3) above is supported by the spec.

Specifically, this text makes (1) unsupported:

   Modules are only allowed to
   include submodules that belong to that module, as defined by the
   "belongs-to" statement (see Section 7.2.2).  Submodules are only
   allowed to include other submodules belonging to the same module.

And this text makes (2) unsupported:

This is unfortunate.  I think both (1) and (2) are useful, but (3) is
mostly just confusing.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 22 Feb 2013 15:03:16 +0100

>    allowed to include other submodules belonging to the same module.

I do not see why this text makes (1) unsupported. This text only puts
up a restriction that you can't include submodules that belong to
other modules, no?
 
>    definitions are made available to the current submodule.

I think the text is ambiguous since it remains unclear what "made
available to the current submodule" means.

> This is unfortunate.  I think both (1) and (2) are useful, but (3) is

My reading is that the current text is ambiguous (and apparently
implementors came to two different interpretations). So we need to
decide what was/is desired and then fix things to avoid the ambiguity.

/js

---

From: Jonathan Hansford <[email-masked]>
Date: Fri, 22 Feb 2013 14:09:16 +0000

Can someone clarify why transitivity is considered so unacceptable?
I'm still learning about YANG and a clarification of this point would
really help. 

On 22.02.2013 14:03, Juergen Schoenwaelder wrote: 

> On
Fri, Feb 22, 2013 at 02:51:15PM +0100, Martin Bjorklund wrote:
Andy Bierman <[email-masked] [1]> wrote: 
>>> Perhaps the spec
is ambiguous, but I don't agree adding busy work for the YANG module
writer is needed. The YANG compiler can easily figure out what to do
based on the belongs-to-stmt.
>> No one has suggested this. 
haven't read anything in this thread that would cause me to change any
code. The spec supports both implementation plans IMO, and it will take
a lot more than an errata to change it.
>> There are three, not two,
alternatives: 1. Require the module to list all submodules (current
pyang behavior) 2. Recursively incorporate the contents from all
submodules into the module (current yuma behavior) 3. Allow submodules
that are not listed in the main module, but do not incorporate contents
from non-listed submodules The spec is not crystal clear, but I think
that only interpretation (3) above is supported by the spec.
Specifically, this text makes (1) unsupported: Modules are only allowed
to include submodules that belong to that module, as defined by the
"belongs-to" statement (see Section 7.2.2). Submodules are only allowed
to include other submodules belonging to the same module.
> I do not
see why this text makes (1) unsupported. This text only puts
> up a
restriction that you can't include submodules that belong to
> other
modules, no?
> 
>> And this text makes (2) unsupported: When a module
includes a submodule, it incorporates the contents of the submodule into
the node hierarchy of the module. When a submodule includes another
submodule, the target submodule's definitions are made available to the
current submodule.
> This
is unfortunate. I think both (1) and (2) are useful, but (3) is mostly
just confusing. 
> implementors came to two different interpretatio>

Links:
------
[1] mailto:[email-masked]

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 15:10:27 +0100 (CET)

> up a restriction that you can't include submodules that belong to

You can interpret this text to allow a submodule A to include a
submodule B that has a "belongs-to M" statement.  But no other text
says that M MUST also include B.

> available to the current submodule" means.

Exactly the same words are used to describe "import".  When a
submodule uses "include" on a submodule it works like "import" on a
module.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 15:12:36 +0100

>      the module (current yuma behavior)

I wonder what "all" means here. Are the submodules looked up in an implementation-dependent list of search directories, or in the entire universe?
 
> This is unfortunate.  I think both (1) and (2) are useful, but (3) is

I am fine with (1), which however needs an update to RFC 6020.

I have strong objections against (2).

And I don't agree (3) is confusing. In C, if you don't include an existing header file, its content is missing. How this could be confusing or surprising? I actually think (3) can be occassionally useful, and it is essentially compatible with the current spec.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 15:17:24 +0100 (CET)

> Can someone clarify why transitivity is considered so unacceptable?

As I wrote earlier, I think it could be useful.  But the text doesn't
support it, imo.

Pro: Lets a submodule designer be free to split the submodule into more
     submodules w/o touching the main module.

Con: When you look at the module, it is less obvious what is actually
     included; you need to follow all the pointers recursively to find
     out.

(and the current text needs some tweakings to allow this)

/martin

---

From: Jonathan Hansford <[email-masked]>
Date: Fri, 22 Feb 2013 14:17:59 +0000

On 22.02.2013 14:12, Ladislav Lhotka wrote: 

> On Feb 22, 2013, at
2:51 PM, Martin Bjorklund <[email-masked]> wrote:
> 
>> Andy Bierman
<[email-masked] [1]> wrote: 
>> 
>>> Perhaps the spec is ambiguous,
but I don't agree adding busy work for the YANG module writer is needed.
The YANG compiler can easily figure out what to do based on the
belongs-to-stmt.
>>> I haven't read
anything in this thread that would cause me to change any code. The spec
supports both implementation plans IMO, and it will take a lot more than
an errata to change it.
>> There are three, not two, alternatives: 1.
Require the module to list all submodules (current pyang behavior) 2.
Recursively incorporate the contents from all submodules into the module
(current yuma behavior)
> 
> I wonder what "all" means here. Are the
submodules looked up in an implementation-dependent list of search
directories, or in the entire universe?

Surely "all" means all directly
or indirectly included submodules. Just writing a YANG submodule with a
"belongs-to" shouldn't make it part of the module unless the module or
an included submodule includes it.

Jonathan

>> 3. Allow submodules
that are not listed in the main module, but do not incorporate contents
from non-listed submodules The spec is not crystal clear, but I think
that only interpretation (3) above is supported by the spec.
Specifically, this text makes (1) unsupported: Modules are only allowed
to include submodules that belong to that module, as defined by the
"belongs-to" statement (see Section 7.2.2). Submodules are only allowed
to include other submodules belonging to the same module. And this text
makes (2) unsupported: When a module includes a submodule, it
incorporates the contents of the submodule into the node hierarchy of
the module. When a submodule includes another submodule, the target
submodule's definitions are made available to the current submodule.
This is unfortunate. I think both (1) and (2) are useful, but (3) is
mostly just confusing.
> And
I don't agree (3) is confusing. In C, if you don't include an existing
header file, its content is missing. How this could be confusing or
surprising? I actually think (3) can be occassionally useful, and it is
essentially compatible with the current spec.

Links:
------

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 06:21:55 -0800

On Fri, Feb 22, 2013 at 6:12 AM, Ladislav Lhotka <[email-masked]> wrote:

> I wonder what "all" means here. Are the submodules looked up in an
> implementation-dependent list of search directories, or in the entire
> universe?
>

What does this have to do with the problem?
Finding submod-C is no different whether the include-stmt is
in the main module or a submodule.

I like the C language definition of #include and this it works fine
for YANG -- if A includes B and B includes C then the definitions
from C are available to A.  You could never refactor code if you
had to change all the upstream code to add new #include statements.
I don't understand your strong objections.  Why is this a bug and
not a feature?

Andy

> And I don't agree (3) is confusing. In C, if you don't include an existing

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 15:24:36 +0100

On Feb 22, 2013, at 3:17 PM, Jonathan Hansford <[email-masked]> wrote:

>>>> Perhaps the spec is ambiguous, but I don't agree adding busy work for the YANG module writer is needed. The YANG compiler can easily figure out what to do based on the belongs-to-stmt.
>>>> I haven't read anything in this thread that would cause me to change any code. The spec supports both implementation plans IMO, and it will take a lot more than an errata to change it.
>>> There are three, not two, alternatives: 1. Require the module to list all submodules (current pyang behavior) 2. Recursively incorporate the contents from all submodules into the module (current yuma behavior)
> Surely "all" means all directly or indirectly included submodules. Just writing a YANG submodule with a "belongs-to" shouldn't make it part of the module unless the module or an included submodule includes it.

I may have misunderstood Andy, but he wrote that it is the "belongs-to" statement that incorporates the contents of a submodule into the main module.

Lada
 
>  
> Jonathan
>>> 3. Allow submodules that are not listed in the main module, but do not incorporate contents from non-listed submodules The spec is not crystal clear, but I think that only interpretation (3) above is supported by the spec. Specifically, this text makes (1) unsupported: Modules are only allowed to include submodules that belong to that module, as defined by the "belongs-to" statement (see Section 7.2.2). Submodules are only allowed to include other submodules belonging to the same module. And this text makes (2) unsupported: When a module includes a submodule, it incorporates the contents of the submodule into the node hierarchy of the module. When a submodule includes another submodule, the target submodule's definitions are made available to the current submodule. This is unfortunate. I think both (1) and (2) are useful, but (3) is mostly just confusing.

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 06:35:16 -0800

On Fri, Feb 22, 2013 at 6:24 AM, Ladislav Lhotka <[email-masked]> wrote:

> the YANG module writer is needed. The YANG compiler can easily figure out
> what to do based on the belongs-to-stmt.
> >>>> I haven't read anything in this thread that would cause me to change
> any code. The spec supports both implementation plans IMO, and it will take
> >>> There are three, not two, alternatives: 1. Require the module to list
> all submodules (current pyang behavior) 2. Recursively incorporate the
> contents from all submodules into the module (current yuma behavior)
> > Surely "all" means all directly or indirectly included submodules. Just
> writing a YANG submodule with a "belongs-to" shouldn't make it part of the
> module unless the module o an included submodule includes it.
>
> I may have misunderstood Andy, but he wrote that it is the "belongs-to"
> statement that incorporates the contents of a submodule into the main
> module.

I am not suggesting that submodules that are not included in any way are
part of
the module. They are just bugs.  Obviously a YANG compiler cannot guess that
unmentioned submodules exist somewhere.  But anytime the compiler encounters
an include or import, it has to process it.

It is up to the developer to decide if the submodule organization is an
unmanageable mess
or a work or art.  Since a module does not need to list all the nested
imports that will
occur (mod A imports B which imports C -- mod A doesn't need to say
anything about C),
I don't agree this is needed for submodules either.

Andy

>
> > Jonathan
> >>> 3. Allow submodules that are not listed in the main module, but do not
> incorporate contents from non-listed submodules The spec is not crystal
> clear, but I think that only interpretation (3) above is supported by the
> spec. Specifically, this text makes (1) unsupported: Modules are only
> allowed to include submodules that belong to that module, as defined by the
> "belongs-to" statement (see Section 7.2.2). Submodules are only allowed to
> include other submodules belonging to the same module. And this text makes
> (2) unsupported: When a module includes a submodule, it incorporates the
> contents of the submodule into the node hierarchy of the module. When a
> submodule includes another submodule, the target submodule's definitions
> are made available to the current submodule. This is unfortunate. I think
> both (1) and (2) are useful, but (3) is mostly just confusing.
> >> And I don't agree (3) is confusing. In C, if you don't include an
> existing header file, its content is missing. How this could be confusing
> or surprising? I actually think (3) can be occassionally useful, and it is

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 15:43:00 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> Since a module does not need to list all the nested
> imports that will

Huh?

  module A {
    ...
    import B {
      prefix b;
    }
   
    leaf foo {
      type c:bar;  // do you think this is legal?
    }
  }

  module B {
    ...
    import C {
      prefix c;
    }
  }

  module C {
    ...
    typedef bar { ... }
  }

I would say that the example above is illegal, and why would include
work differently?

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 15:50:17 +0100

On Feb 22, 2013, at 3:35 PM, Andy Bierman <[email-masked]> wrote:

> > Surely "all" means all directly or indirectly included submodules. Just writing a YANG submodule with a "belongs-to" shouldn't make it part of the module unless the module o an included submodule includes it.
> 
> I am not suggesting that submodules that are not included in any way are part of

OK, so it was indeed a misunderstanding.

> 
> It is up to the developer to decide if the submodule organization is an unmanageable mess
> or a work or art.  Since a module does not need to list all the nested imports that will
> occur (mod A imports B which imports C -- mod A doesn't need to say anything about C),

I now have an impression that the concept of submodules is unnecessarily complicated, perhaps it could work simply as #include in cpp, i. e. the text of a module would be constructed by applying recursively all includes. Then submodules could also directly refer to definitions in the main module. 

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 06:55:27 -0800

> I would say that the example above is illegal, and why would include
This isn't what I said -- module B uses module C, not A.
Obviously if module A uses something in C it needs to import it
directly, but it will pick up any definitions from B that are using
definitions from C.

I think the text in question is ambiguous and open to misinterpretation,
but let's stop with the strawman stuff.  The issue is whether
definitions from nested submodules are visible to external modules or not.

Andy

---

From: Jonathan Hansford <[email-masked]>
Date: Fri, 22 Feb 2013 14:55:26 +0000

3:35 PM, Andy Bierman <[email-masked]> wrote:
>> On Fri, Feb 22,
2013 at 6:24 AM, Ladislav Lhotka <[email-masked] [3]> wrote: On Feb 22,
2013, at 3:17 PM, Jonathan Hansford <[email-masked] [4]> wrote:

>>>> On Feb
22, 2013, at 2:51 PM, Martin Bjorklund <[email-masked] [2]> wrote: 

Perhaps the spec is ambiguous, but I don't agree adding busy work for
the YANG module writer is needed. The YANG compiler can easily figure
out what to do based on the belongs-to-stmt.
>>>>> No one has suggested
this. 
>>>>> 
>>>>>> I haven't read anything in this thread that would
cause me to change any code. The spec supports both implementation plans
IMO, and it will take a lot more than an errata to change it.
There are three, not two, alternatives: 1. Require the module to list
all submodules (current pyang behavior) 2. Recursively incorporate the
contents from all submodules into the module (current yuma
behavior)
>>>> I wonder what "all" means here. Are the submodules looked
up in an implementation-dependent list of search directories, or in the
entire universe?
>>> Surely "all" means all directly or indirectly
included submodules. Just writing a YANG submodule with a "belongs-to"
shouldn't make it part of the module unless the module o an included
submodule includes it.
>> I may have misunderstood Andy, but he wrote
that it is the "belongs-to" statement that incorporates the contents of
a submodule into the main module. I am not suggesting that submodules
that are not included in any way are part of the module. They are just
bugs. Obviously a YANG compiler cannot guess that unmentioned submodules
exist somewhere. But anytime the compiler encounters an include or
import, it has to process it.
> 
> OK, so it was indeed a
misunderstanding.
> 
>> It is up to the developer to decide if the
submodule organization is an unmanageable mess or a work or art. Since a
module does not need to list all the nested imports that will occur (mod
A imports B which imports C -- mod A doesn't need to say anything about
C), I don't agree this is needed for submodules either.
> I now have
an impression that the concept of submodules is unnecessarily
complicated, perhaps it could work simply as #include in cpp, i. e. the
text of a module would be constructed by applying recursively all
includes. Then submodules could also directly refer to definitions in
the main module.

Wouldn't that require the submodule to include the
module, which is not allowed?

Jonathan

Lada

>> Lada Andy 
Jonathan 
>>> 
>>>>> 3. Allow submodules that are not listed in the main
module, but do not incorporate contents from non-listed submodules The
spec is not crystal clear, but I think that only interpretation (3)
above is supported by the spec. Specifically, this text makes (1)
unsupported: Modules are only allowed to include submodules that belong
to that module, as defined by the "belongs-to" statement (see Section
7.2.2). Submodules are only allowed to include other submodules
belonging to the same module. And this text makes (2) unsupported: When
a module includes a submodule, it incorporates the contents of the
submodule into the node hierarchy of the module. When a submodule
includes another submodule, the target submodule's definitions are made
available to the current submodule. This is unfortunate. I think both
(1) and (2) are useful, but (3) is mostly just confusing.
>>>> I am fine
with (1), which however needs an update to RFC 6020. I have strong
objections against (2). And I don't agree (3) is confusing. In C, if you
don't include an existing header file, its content is missing. How this
could be confusing or surprising? I actually think (3) can be
occassionally useful, and it is essentially compatible with the current
spec. Lada 
>>>> 
>>>>> /martin
>>>> -- Ladislav Lhotka, CZ.NIC Labs PGP
Key ID: E74E8C0C _______________________________________________ netmod
mailing list [email-masked]:
[5]//www.ietf.org/mailman/listinfo/netmod
_______________________________________________ netmod mailing list
[email-masked] [6]https://www.ietf.org/mailman/listinfo/netmod [7]
-- Ladislav Lhotka, CZ.NIC Labs PGP Key ID: E74E8C0C
_______________________________________________ netmod mailing list
[email-masked] [8]https://www.ietf.org/mailman/listinfo/netmod [9]
--

Links:
------
[1] mailto:[email-masked]
[2]
mailto:[email-masked]
[3] mailto:[email-masked]
[4]
mailto:[email-masked]
[5] mailto:[email-masked]:
[6]
mailto:[email-masked]
[7]
https://www.ietf.org/mailman/listinfo/netmod
[8]
mailto:[email-masked]
[9] https://www.ietf.org/mailman/listinfo/netmod

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 15:57:13 +0100

> I would say that the example above is illegal, and why would include

It is clearly illegal because there is no definition of the "c" prefix in the context of module A.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 16:13:31 +0100

On Feb 22, 2013, at 3:55 PM, Jonathan Hansford <[email-masked]> wrote:

>>> On Fri, Feb 22, 2013 at 6:24 AM, Ladislav Lhotka <[email-masked]> wrote: On Feb 22, 2013, at 3:17 PM, Jonathan Hansford <[email-masked]> wrote:
>>> I may have misunderstood Andy, but he wrote that it is the "belongs-to" statement that incorporates the contents of a submodule into the main module. I am not suggesting that submodules that are not included in any way are part of the module. They are just bugs. Obviously a YANG compiler cannot guess that unmentioned submodules exist somewhere. But anytime the compiler encounters an include or import, it has to process it.
>>> It is up to the developer to decide if the submodule organization is an unmanageable mess or a work or art. Since a module does not need to list all the nested imports that will occur (mod A imports B which imports C -- mod A doesn't need to say anything about C), I don't agree this is needed for submodules either.
> Wouldn't that require the submodule to include the module, which is not allowed?

No. I mean that each include would be simply replaced by the contents of the corresponding submodule (data nodes, rpcs, identities, features etc., but obviously not metadata). If recursive includes were allowed, I see no benefit of requiring the main module to include a submodule in order to be able to use definitions from that submodule.

Whenever namespace modularity is needed, modules and imports should be used, and not submodules. 

Lada

>>>>> I am fine with (1), which however needs an update to RFC 6020. I have strong objections against (2). And I don't agree (3) is confusing. In C, if you don't include an existing header file, its content is missing. How this could be confusing or surprising? I actually think (3) can be occassionally useful, and it is essentially compatible with the current spec. Lada
>>>>>> /martin
>>>>> -- Ladislav Lhotka, CZ.NIC Labs PGP Key ID: E74E8C0C _______________________________________________ netmod mailing list [email-masked]://www.ietf.org/mailman/listinfo/netmod
>>>> _______________________________________________ netmod mailing list [email-masked]://www.ietf.org/mailman/listinfo/netmod
>>> -- Ladislav Lhotka, CZ.NIC Labs PGP Key ID: E74E8C0C _______________________________________________ netmod mailing list [email-masked]://www.ietf.org/mailman/listinfo/netmod
>> --

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 07:19:17 -0800

On Fri, Feb 22, 2013 at 6:09 AM, Jonathan Hansford
<[email-masked]>wrote:

> **
>
> Can someone clarify why transitivity is considered so unacceptable? I'm
> still learning about YANG and a clarification of this point would really
> help.
>

I think we are all still learning YANG ;-)

Even if a submodule was considered its own namespace, the transitivity
seems to work fine.

1) Does the including mod or submod inherit everything from the included
namespace?
 A: Yes: All top-level definitions are exported; there is no way to declare
such a definition
     to be private.

2) What are the contents of the submodule namespace? A or B?
    A) all of the top-level definitions in just that submodule
    B) all of the top-level definitions in the submodule and any submodules
         included into that submodule (or nested)

We are saying the answer is B and Martin and Lada are saying A.
The text is not entirely clear, and you can over-emphisise some text and
ignore other text to make a point, but that doesn't fix the RFC.

Andy

On 22.02.2013 14:03, Juergen Schoenwaelder wrote:
> Perhaps the spec is ambiguous, but I don't agree adding busy work for the
> YANG module writer is needed. The YANG compiler can easily figure out what
> to do based on the belongs-to-stmt.
>
> code. The spec supports both implementation plans IMO, and it will take a
> lot more than an errata to change it.
>
> There are three, not two, alternatives: 1. Require the module to list all
> submodules (current pyang behavior) 2. Recursively incorporate the contents
> from all submodules into the module (current yuma behavior) 3. Allow
> submodules that are not listed in the main module, but do not incorporate
> contents from non-listed submodules The spec is not crystal clear, but I
> think that only interpretation (3) above is supported by the spec.
> Specifically, this text makes (1) unsupported: Modules are only allowed to
> include other submodules belonging to the same module.
>
> And this text makes (2) unsupported: When a module includes a submodule,
> it incorporates the contents of the submodule into the node hierarchy of

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 16:19:41 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> >>> On Fri, Feb 22, 2013 at 6:24 AM, Ladislav Lhotka <[email-masked]> wrote: On
> >>> Feb 22, 2013, at 3:17 PM, Jonathan Hansford <[email-masked]> wrote:
> >>>>>>> any code. The spec supports both implementation plans IMO, and it will
> >>>>>>> take a lot more than an errata to change it.
> >>> module. I am not suggesting that submodules that are not included in any
> >>> way are part of the module. They are just bugs. Obviously a YANG compiler
> >>> cannot guess that unmentioned submodules exist somewhere. But anytime the
> >>> compiler encounters an include or import, it has to process it.
> >>> unmanageable mess or a work or art. Since a module does not need to list
> >>> all the nested imports that will occur (mod A imports B which imports C --
> >>> mod A doesn't need to say anything about C), I don't agree this is needed
> >>> for submodules either.
> >> I now have an impression that the concept of submodules is unnecessarily
> >> complicated, perhaps it could work simply as #include in cpp, i. e. the text
> >> of a module would be constructed by applying recursively all includes. Then
> >> submodules could also directly refer to definitions in the main module.
> >>  
> No. I mean that each include would be simply replaced by the contents of the
> corresponding submodule (data nodes, rpcs, identities, features etc., but
> obviously not metadata).

But what about collisions?  Compare w/ #ifndef.

Anyway, we're not going to completely redesign how submodules work at
this point in time.

/martin

---

From: Jonathan Hansford <[email-masked]>
Date: Fri, 22 Feb 2013 15:30:08 +0000

On 22.02.2013 15:19, Martin Bjorklund wrote: 

> Ladislav Lhotka
<[email-masked]> wrote:
> 
>> On Feb 22, 2013, at 3:55 PM, Jonathan
Hansford <[email-masked] [6]> wrote: 
>> 
>>> On 22.02.2013
14:50, Ladislav Lhotka wrote: 
>>> 
>>>> On Feb 22, 2013, at 3:35 PM,
Andy Bierman <[email-masked] [5]> wrote: 
>>>>> On Fri, Feb
22, 2013 at 6:24 AM, Ladislav Lhotka <[email-masked] [3]> wrote: On Feb
22, 2013, at 3:17 PM, Jonathan Hansford <[email-masked] [4]>

>>>>>> 
>>>>>>> On Feb 22, 2013, at 2:51 PM, Martin Bjorklund
<[email-masked] [2]> wrote: 
>>>>>>>>> Perhaps the spec is
ambiguous, but I don't agree adding busy work for the YANG module writer
is needed. The YANG compiler can easily figure out what to do based on
the belongs-to-stmt.

>>>>>>>>> I haven't read anything in this thread that would cause me to
change any code. The spec supports both implementation plans IMO, and it
will take a lot more than an errata to change it.
>>>>>>>> There are
three, not two, alternatives: 1. Require the module to list all
submodules (current pyang behavior) 2. Recursively incorporate the
contents from all submodules into the module (current yuma
behavior)
>>>>>>> I wonder what "all" means here. Are the submodules
looked up in an implementation-dependent list of search directories, or
in the entire universe?
>>>>>> Surely "all" means all directly or
indirectly included submodules. Just writing a YANG submodule with a
"belongs-to" shouldn't make it part of the module unless the module o an
included submodule includes it.
>>>>> I may have misunderstood Andy, but
he wrote that it is the "belongs-to" statement that incorporates the
contents of a submodule into the main module. I am not suggesting that
submodules that are not included in any way are part of the module. They
are just bugs. Obviously a YANG compiler cannot guess that unmentioned
submodules exist somewhere. But anytime the compiler encounters an
include or import, it has to process it.
>>>> OK, so it was indeed a
misunderstanding. 
>>>> 
>>>>> It is up to the developer to decide if
the submodule organization is an unmanageable mess or a work or art.
Since a module does not need to list all the nested imports that will
occur (mod A imports B which imports C -- mod A doesn't need to say
anything about C), I don't agree this is needed for submodules
either.
>>>> I now have an impression that the concept of submodules is
unnecessarily complicated, perhaps it could work simply as #include in
cpp, i. e. the text of a module would be constructed by applying
recursively all includes. Then submodules could also directly refer to
definitions in the main module.
>>> Wouldn't that require the submodule
to include the module, which is not allowed?
>> No. I mean that each
include would be simply replaced by the contents of the corresponding
submodule (data nodes, rpcs, identities, features etc., but obviously
not metadata).
this point in time.

But we need to agree how to remove ambiguities. Can
we afford for two different tools to have different understandings of
how "include" should be implemented? Doesn't the current situation
undermine YANG?

/martin

Links:
------
[1]
mailto:[email-masked]
[2] mailto:[email-masked]
[3]
mailto:[email-masked]
[4] mailto:[email-masked]
[5]
mailto:[email-masked]
[6] mailto:[email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 07:31:41 -0800

On Fri, Feb 22, 2013 at 7:19 AM, Martin Bjorklund <[email-masked]> wrote:

> figure
> change
> unnecessarily
No, but we should try to clarify the RFC when it is unclear.

Yuma treats submodules as a partitioned namespace, not a hierarchy
of submodule namespaces.  There is a big difference:

module A {
   include B;
   include C;
}

submodule B {
  include C;
  leaf x { type a:mystring; }
}

submodule C {
  typedef mystring {
    type string { length 1..max; }
  }
  leaf y { type mystring; }
}

So, is this an error because leaf y is included into the main module twice?
Obviously the intent is to just use the typedef from C but all definitions
are
pulled into the submodule namespace.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 16:36:14 +0100 (CET)

> No, but we should try to clarify the RFC when it is unclear.

Yes.

> So, is this an error because leaf y is included into the main module

With the current text, the leaf y is not included twice.  When
submodule B includes C, it works like an import ("makes the
definitions available for reference").  When module A includes C, it
"copies" the definitions from C into itself.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 17:04:34 +0100

> But what about collisions?  Compare w/ #ifndef.

With transitive inclusions, collisions have to be avoided in any case. Consider this:

module M {
  include A;
  …
}

module A {
  belongs-to A;
  include B;
  …
}

module B {
  belongs-to A;
  grouping ggg {
    …
  }
  …
}

Even if module M doesn't include B, it cannot define its own "ggg" because an importing module would see both.
So from the point of view of collision avoidance, it is irrelevant whether M includes B or not. 

> Anyway, we're not going to completely redesign how submodules work at

Well, it seems we will have to do something, so why not get it right?

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 17:22:57 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> With transitive inclusions, collisions have to be avoided in any
> case.

Exactly.  But with the current text, collisions are avoided, since it
is only the module that "incorporates" the contents of the
submodules.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 08:33:08 -0800

> is only the module that "incorporates" the contents of the

IMO, for export purposes, submodules are conceptually flat -- they all
contribute
to the module namespace.  The compiler has to detect any naming collisions
for the
entire module namespace.

With your interpretation, how would the following corner-case be handled?

module A {
  include B;
  typedef X { type string; }
}

submodule B {
  include C;
  leaf a { type a:X; }
}

submodule C {
  typedef X { type int32; }
}

In Yuma this would be a duplicate name error for 'X'.
Is that correct or is leaf a an int32 and no problem?
(Looks like a problem to me.)

Andy

---

From: Jonathan Hansford <[email-masked]>
Date: Fri, 22 Feb 2013 16:39:39 +0000

On 22.02.2013 16:33, Andy Bierman wrote: 

> On Fri, Feb 22, 2013 at
8:22 AM, Martin Bjorklund <[email-masked] [3]> wrote:
>> Ladislav
Lhotka <[email-masked] [1]> wrote:
>> >
>> > On Feb 22, 2013, at 4:19 PM,
Martin Bjorklund <[email-masked] [2]> wrote:
>> Exactly. But with
the current text, collisions are avoided, since it
> IMO, for
export purposes, submodules are conceptually flat -- they all contribute

> to the module namespace. The compiler has to detect any naming
collisions for the 
> With your
interpretation, how would the following corner-case be handled? 
module A { 
> submodule
B { 
> In Yuma this would be a duplicate name
error for 'X'.

Does Yuma effectively surround each include with an
implicit #ifndef (i.e. the only the first include of a submodule is
parsed)? 

> Andy

Links:
------
[1] mailto:[email-masked]

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 17:40:27 +0100

> is only the module that "incorporates" the contents of the

Yes, but that means non-transitive inclusion. It would be worthwhile to consider pros and cons of all three options.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 17:46:28 +0100 (CET)

>   typedef X { type int32; }

This would be an error b/c A does not include C.

If A included C, it would be a duplicate definition error.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 22 Feb 2013 17:47:42 +0100 (CET)

> If A included C, it would be a duplicate definition error.

Sorry, this was for alternative (1).

In alternative (3), this would be legal and type X is exported from A
as a string.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 17:50:20 +0100

On Feb 22, 2013, at 5:33 PM, Andy Bierman <[email-masked]> wrote:

> IMO, for export purposes, submodules are conceptually flat -- they all contribute
> to the module namespace.  The compiler has to detect any naming collisions for the
> entire module namespace.
> 

With scenario (3), leaf "a" would be int32, and for a module importing A, the X type would be an alias for string. With scenario (1), it is an error because B includes C but A doesn't.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 08:51:02 -0800

On Fri, Feb 22, 2013 at 8:39 AM, Jonathan Hansford
<[email-masked]>wrote:

>  Does Yuma effectively surround each include with an implicit #ifndef
> (i.e. the only the first include of a submodule is parsed)?
>

yes.
Since a submodule is only parsed once, it doesn't matter how many includes
appear for it (main module or submodules).

There is no clear text that says either implementation strategy is right or
wrong.
IMO, the Yuma approach is more user-friendly but Martin thinks listing all
the
submodules in the main module is more user-friendly.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 17:58:06 +0100

On Feb 22, 2013, at 5:51 PM, Andy Bierman <[email-masked]> wrote:

> 
> 
> On Fri, Feb 22, 2013 at 8:39 AM, Jonathan Hansford <[email-masked]> wrote:
>  
> Does Yuma effectively surround each include with an implicit #ifndef (i.e. the only the first include of a submodule is parsed)?
> 
> yes.

I have one more question. If we have

>  typedef X { type int32; }

does Yuma allow to use type X within module A?

Lada

> 
> There is no clear text that says either implementation strategy is right or wrong.
> IMO, the Yuma approach is more user-friendly but Martin thinks listing all the

---

From: Jonathan Hansford <[email-masked]>
Date: Fri, 22 Feb 2013 17:01:28 +0000

Lhotka <[email-masked] [1]> wrote:
>>>> > On Feb 22, 2013, at 4:19
PM, Martin Bjorklund <[email-masked] [2]> wrote:
>>>> > With transitive
inclusions, collisions have to be avoided in any
>>>> > case.
Exactly. But with the current text, collisions are avoided, since
it
submodules.
>>> to the module namespace.
The compiler has to detect any naming collisions for the 
>>> entire
module namespace. 
>>> leaf a {
type a:X; } 

> yes. 
Since a submodule is only parsed once, it doesn't matter how many
includes 
> There is no
clear text that says either implementation strategy is right or wrong.

> submodules in the main module is more
user-friendly.

To my mind, if something from a submodule is directly
used in the module then it is more "user-friendly" (i.e. intuitive) for
that submodule to be included by the module. If a submodule's content is
only directly used in another submodule it is more "user-friendly" for
that submodule to be included in the submodule. 
Jonathan 

>>> Is that
correct or is leaf a an int32 and no problem? 

Links:
------
[1]
mailto:[email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 09:03:16 -0800

On Fri, Feb 22, 2013 at 8:50 AM, Ladislav Lhotka <[email-masked]> wrote:

> With scenario (3), leaf "a" would be int32, and for a module importing A,
> the X type would be an alias for string. With scenario (1), it is an error
> because B includes C but A doesn't.
>
>
IMO, any compiler should report an error because it has to parse module A
in order to get to submodule C.  It must include B and then C and then
must parse the duplicate typedef.

But my example seems to be legal YANG if submodules are treated as separate
namespaces.
I don't think there is any text that says A MUST include C.
If it does there will be a conflict, but if C is only visible to B,
and the only typedef X visible to B is int32, the compiler must accept it,
and the typedef "X" has a hidden different version within the module.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 09:06:18 -0800

> does Yuma allow to use type X within module A?
No -- it would be an unresolved identifier because there is no "include C",
which tells the compiler where it is allowed to look for type X.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 18:12:14 +0100

On Feb 22, 2013, at 6:03 PM, Andy Bierman <[email-masked]> wrote:

> But my example seems to be legal YANG if submodules are treated as separate namespaces.

Yes, and that is, I think, the essence of (3). It can be useful, for instance, if one wants to share some definitions among multiple submodules but doesn't want to expose them to external modules. This is actually how I stumbled upon this whole issue.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 18:16:09 +0100

> which tells the compiler where it is allowed to look for type X.

But what's the point of requiring this include if there cannot be any other X anyway? In A, you can just use the same set of top-level definitions that any importing modules sees, even without that include.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 22 Feb 2013 09:24:00 -0800

On Fri, Feb 22, 2013 at 9:12 AM, Ladislav Lhotka <[email-masked]> wrote:

> > With scenario (3), leaf "a" would be int32, and for a module importing
> A, the X type would be an alias for string. With scenario (1), it is an
> error because B includes C but A doesn't.
> >
> >
> Yes, and that is, I think, the essence of (3). It can be useful, for
> instance, if one wants to share some definitions among multiple submodules
> but doesn't want to expose them to external modules. This is actually how I
> stumbled upon this whole issue.
>
>
IMO, (C) is quite dangerous and complicated. 1 namespace per module is
plenty.
I never liked submodules in the first place and this would make YANG even
more complicated.
At this point there is absolutely no difference between submodules and
modules, and
we might as well get rid of them.

I much prefer Kent's solution he made last year (I am adapting it):

 -- get rid of submodules completely and allow modules to contribute
to an existing namespace -- remove the restriction of 1 module == 1 XML
namespace.
That makes YANG way simpler, and there is absolutely no need for submodules
whatsoever.

But that is kind of radical and we are not working on YANG 2.0.
But someday if we do, the issue will come up again.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 22 Feb 2013 18:39:41 +0100

On Feb 22, 2013, at 6:24 PM, Andy Bierman <[email-masked]> wrote:

> IMO, (C) is quite dangerous and complicated. 1 namespace per module is plenty.
> I never liked submodules in the first place and this would make YANG even more complicated.
> At this point there is absolutely no difference between submodules and modules, and
> to an existing namespace -- remove the restriction of 1 module == 1 XML namespace.

And how about a simple textual include mechanism along the lines of #include in cpp or rather <include> in XSLT, see

http://www.w3.org/TR/1999/REC-xslt-19991116#include

Lada