Subject: Re: [netmod] include

From: Martin Bjorklund <[email-masked]>
Date: Mon, 01 Apr 2013 21:13:51 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> Hi,
> 
> for the time being, an easy fix seems to be to require that all submodules be
> explicitly included from the main module. This may even qualify as an erratum.

+1

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 1 Apr 2013 18:40:12 -0700

>> explicitly included from the main module. This may even qualify as an erratum.

Section 6.2.1 already requires that identifiers with a specific
namespace MUST be unique, so no new text is needed at all.

6.2.1.  Identifiers and Their Namespaces

   Each identifier is valid in a namespace that depends on the type of
   the YANG item being defined.  All identifiers defined in a namespace
   MUST be unique.

   o  All module and submodule names share the same global module
      identifier namespace.

   o  All extension names defined in a module and its submodules share
      the same extension identifier namespace.

   o  All feature names defined in a module and its submodules share the
      same feature identifier namespace.

   o  All identity names defined in a module and its submodules share
      the same identity identifier namespace.

   o  All derived type names defined within a parent node or at the top
      level of the module or its submodules share the same type
      identifier namespace.  This namespace is scoped to all descendant
      nodes of the parent node or module.  This means that any
      descendent node may use that typedef, and it MUST NOT define a
      typedef with the same name.

   o  All grouping names defined within a parent node or at the top
      level of the module or its submodules share the same grouping
      identifier namespace.  This namespace is scoped to all descendant
      nodes of the parent node or module.  This means that any
      descendent node may use that grouping, and it MUST NOT define a
      grouping with the same name.

   o  All leafs, leaf-lists, lists, containers, choices, rpcs,
      notifications, and anyxmls defined (directly or through a uses
      statement) within a parent node or at the top level of the module
      or its submodules share the same identifier namespace.  This
      namespace is scoped to the parent node or module, unless the
      parent node is a case node.  In that case, the namespace is scoped
      to the closest ancestor node that is not a case or choice node.

   o  All cases within a choice share the same case identifier
      namespace.  This namespace is scoped to the parent choice node.

> /martin

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 02 Apr 2013 11:49:41 +0200

Andy Bierman <[email-masked]> writes:

>
> The fact is, you cannot rely the information in the YANG modules to
> determine the set of files that were used to construct the schema tree
> in use by the server.
>
> 1)  module namspaces are assumed to be globally unique, not module names.
>      Import "foo" does not reliably identify anything. The <capability> URI
>     for the YANG module is needed to map namespace to module-name.

I think the message that module names should be kept unique is quite clear. After all, even the uniqueness of URIs is not enforced and works only as long as people follow the rules.

>
>  2) Since submodules are not advertised as YANG module capabilities,
>      the <hello> cannot be used to map sub-module names to the
>      namespace-stmt value.

Yes, that's why it is important for the client to be able to find a complete list of submodules in the main module.
 
>
>  3) Since import-by-revision and include-by-revision are optional to use,
>      the import or include statements cannot be used to reliably determine
>      which revision of the specified module or submodule to use.

This is a problem of identifying a version of a data model, but it is IMO orthogonal to the present subject, which is how to determine the components of a single version.

>      The <capability> URI can usually be used to resolve import, but
>      nothing except the 'schema' list contains this information for submodules.
>      (Making them mandatory to use would illuminate how fragile they are.)

A YANG data model is supposed to be a contract between the server and client, so it is up to the client to make an autonomous decision whether the stuff received from the server is valid.

Lada
  
>
>> Lada
>>
>
> Andy
>
>>
>>>
>>> The include-stmt just says how to resolve prefixes within the module
>>> that is being parsed. The import-stmt says how to resolve prefixes
>>> from other modules.
>>>
>>>
>>>> Lada
>>>>
>>>
>>> Andy
>>>
>>>
>>>>>
>>>>> 7.1.6.  The include Statement
>>>>>
>>>>>  [para 1].....  Modules are only allowed to
>>>>>  include submodules that belong to that module, as defined by the
>>>>>  "belongs-to" statement (see Section 7.2.2).
>>>>>
>>>>> 7.2.2.  The belongs-to Statement
>>>>>
>>>>>  The "belongs-to" statement specifies the module to which the
>>>>>  submodule belongs.  The argument is an identifier that is the name of
>>>>>  the module.
>>>>>
>>>>> There is only one XML namespace for the entire module.
>>>>> QNames within that namespace must be unique.
>>>>> There is no namespace per submodule in YANG.
>>>>>
>>>>>> Lada
>>>>>>
>>>>>
>>>>> Andy
>>>>
>>>> --
>>>> Ladislav Lhotka, CZ.NIC Labs
>>>> PGP Key ID: E74E8C0C
>>>>
>>>>
>>>>
>>>>
>>
>> --
>> Ladislav Lhotka, CZ.NIC Labs
>> PGP Key ID: E74E8C0C
>>
>>
>>
>>

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 2 Apr 2013 10:11:41 -0700

> I think the message that module names should be kept unique is quite clear. After all, even the uniqueness of URIs is not enforced and works only as long as people follow the rules.

It is quite clear from sec 7.1 that private module names are not reliable.

7.1 (para 2 & 3)

   Names of modules published in RFC streams [RFC4844] MUST be assigned
   by IANA, see Section 14.

   Private module names are assigned by the organization owning the
   module without a central registry.  It is RECOMMENDED to choose
   module names that will have a low probability of colliding with
   standard or other enterprise modules and submodules, e.g., by using
   the enterprise or organization name as a prefix for the module name.

> Yes, that's why it is important for the client to be able to find a complete list of submodules in the main module.

This is a minor implementation detail.
The client can usually figure out the submodules to use
by following the include-chain.

I don't agree that submodules have their own namespace.

The include-stmt is used to make a subset of all module symbols
available to that suibmodule.  IMO this is rather pointless and
just adds extra complexity (like we are discussing right now).

The "belongs-to" statement has the module prefix as
a sub-statement.  Any symbol from the module being
compiled must use that prefix.  Note that the include-stmt
does not have a prefix sub-statement.

The symbol resolution logic for the current design is not broken.
Each parent module or submodule MUST pull in all definitions
from its own included submodules to compile statements.
So saying that these symbols are not visible is incorrect.
All the submodules for the module are accounted for in the
dependency tree.

It is just an implementation detail how the compiler determines
that the requirements of sec 6.2.1 are met.  The human writing
the YANG module does not need to add anything to the main
module.  Listing all the submodules in the main module
defeats the whole purpose of nesting submodules in
the first place.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 2 Apr 2013 20:47:20 +0200

> by following the include-chain.

There is no support for this include chain in 6020. 

> I don't agree that submodules have their own namespace.

I don't think I have ever said this.

> just adds extra complexity (like we are discussing right now).

I agree with this part, but sec. 7.1.6 also has this text:

   When a module includes a submodule, it incorporates the contents of
   the submodule into the node hierarchy of the module.

This means it is the include statement that pulls a submodule tree into the module hierarchy.

IMO a sensible solution would be to require the main module to include all submodules, and have belongs-to in submodules in order to declare the prefix. That is, no includes in submodules, submodules could automatically refer to top-level definitions in the main module and all other submodules.

I suspect we may be in a violent agreement. :-)

Lada 

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Tue, 2 Apr 2013 12:13:31 -0700

> There is no support for this include chain in 6020.

The include chain is the conceptual dependency tree specified
by the include-stmts in the main module and each submodule
included by the main module or a submodule.

There has never been any support in YANG for
module search paths or other implementation-specific details
which would impact the YANG compiler's ability to
process an include-stmt.

> I don't think I have ever said this.

I thought this came up in the discussion of private submodules.

>    the submodule into the node hierarchy of the module.

IMO this means that all symbols included into the submodule
via include-stmts or import-stmts are available to the compiler
for resolving symbols.

Consider a simple nested example:

   submod2 --> submod1 --> main

submod1 includes all of submod2 into its context, and
main includes all of submod1+submod2 into its context.

If this did not happen then groupings would not compile
correctly.  I can use a grouping from submod1 in main.
That grouping can use a typedef from submod2.
That typedef is used in the main module even though
the typedef name is not referenced directly in the main module.

Having extra include-stmts in the main module for symbols
that are not referenced in the main module is just busy work
for the YANG designer.  The compiler does not need
any extra includes to resolve a symbol.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 2 Apr 2013 21:37:54 +0200

> via include-stmts or import-stmts are available to the compiler

My understanding is this is only true for includes in submodules, see the next sentence in 7.1.6:

   When a submodule includes another submodule, the target submodule's
   definitions are made available to the current submodule.

The difference seems clear:

- include in the main module incorporates submodule contents into the node hierarchy,
- include in a submodule makes definitions available.

> any extra includes to resolve a symbol.

In my view, an include statement in the main module is equivalent to having the submodule contents directly in the main module.

Lada