Subject: Re: [netmod] ietf-yang-smiv2

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 9 Nov 2011 16:24:46 +0100

On Wed, Oct 12, 2011 at 03:12:34PM -0400, David Reid wrote:
> I have a few suggestions for ietf-yang-smiv2.yang in 
> draft-ietf-netmod-smi-yang-01:
> 
> 1. Add an extension to indicate if an index object is IMPLIED. I believe that
>    with this one addition a tool could get all of the MIB information it 
>    needs from the converted yang module.
> 
>    extension implied {
>      argument "index";
>      description
>       "If an SMIv2 INDEX object is preceded by the IMPLIED keyword, then 
>        the implied statement is present in the yang module and takes as 
>        an argument the name of the IMPLIED index object.";
>      reference
>       "RFC2578: Structure of Management Information Version 2 (SMIv2)";
>    }

I am not strictly against it but I do not really understand the use
case for this. For data shipped over NETCONF, it does not matter
whether the INDEX is IMPLIED or not.

> 2. Modify the description clause of the oid extension to allow a more 
>    user-friendly format. For example, ifEntry could be written as
>    ifTable.1 instead of 1.3.6.1.2.1.2.2.1, or perhaps just .1 as a
>    short hand notation.
> 
>    extension oid {
>      argument "value";
>      description
>       "The oid statement takes as an argument the object identifier
>        assigned to an SMIv2 definition. The object identifier value
>        is written in dotted decimal notation. If an oid statement
>        exists for the parent object identifier, then the value may
>        be written in a short hand notation of the form 
>        <parent descriptor>.<last subidentifier> or just 
>        .<last subidentifier>. For example, the oid value for ifTable from
>        IF-MIB would be '1.3.6.1.2.1.2.2'. The value for ifEntry could be 
>        '1.3.6.1.2.1.2.2.1', 'ifTable.1', or simply '.1'. Similarily, 
>        ifDescr could be '1.3.6.1.2.1.2.2.1.2', 'ifEntry.2', or '.2'";
>      reference
>       "RFC2578: Structure of Management Information Version 2 (SMIv2)";
>    }

This already was discussed and so far it seems Andy and me prefer to
keep the value a full OID. If other representations are really needed,
we should have additional aliases instead of complicating life by
allowing translations to generate whatever they prefer.
 
> 3. The alias extension is a useful addition. I would like to see it mentioned
>    in the body of the document much like the oid extension is mentioned. Here
>    are the suggested changes:
> 
>    Section 7:
> 
>    The SMIv2 uses OBJECT IDENTIFIER assignments to introduce names for
>    intermediate nodes in the OBJECT IDENTIFIER tree.  OBJECT IDENTIFIER
>    assignments MAY be translated into smiv2:alias statements, see the 
>    YANG extension defined in Section 11.

OK
 
>    Section 8.6, first paragraph:
> 
>    An OBJECT-TYPE macro invocation defining an augmenting conceptual
>    table MAY be translated into an smiv2:alias statement, see the    
>    YANG extension defined in Section 11. The clauses of the macro are
>    translated as follows:
> 
>    o  The SMIv2 SYNTAX clause is ignored.
> 
>    o  The SMIv2 UNITS clause is ignored.
> 
>    o  The SMIv2 MAX-ACCESS clause is ignored.
> 
>    o  The SMIv2 STATUS clause is mapped to the YANG status statement.
>       The generation of the YANG status statement is skipped if the
>       value of the STATUS clause is current.
> 
>    o  The SMIv2 DESCRIPTION clause is mapped to the YANG description
>       statement.
> 
>    o  The SMIv2 REFERENCE clause is mapped to the YANG reference
>       statement.
> 
>    o  The value of the SMIv2 OBJECT-TYPE macro invocation MAY be
>       translated into an smiv2:oid statement, see the YANG extension
>       defined in Section 11.

I am not sure I understand what you are trying to achieve. Is your
idea to stick the status/description/reference statements into the
alias statement?
 
>    Section 8.6, second paragraph, add one more bullet:
> 
>    o  The name of the SMIv2 OBJECT-TYPE macro invocation MAY be
>       translated into an smiv2:alias statement, see the YANG extension
>       defined in Section 11.

If we add this here, would we not also have to add this at other
places, e.g. sections 8.1, 8.3, 9 and likely a few more places?
Perhaps we should agree on what we are producing. Right now, smidump
translates ifEntry into a YANG leaf that has an embedded smiv2:oid
statement. In addition, I am creating a top-level smiv2:alias
statement for all defined SMIv2 identifiers containing another
smiv2:oid statement, that is, for the ifEntry I have:

    // ..

      list ifEntry {
        key "ifIndex";
        description
         "An entry containing management information applicable to a
          particular interface.";
        smiv2:oid "1.3.6.1.2.1.2.2.1";

	// ...
     }

   // ...

   smiv2:alias "ifEntry" {
     smiv2:oid "1.3.6.1.2.1.2.2.1";
   }

> 4. With respect to generating the smiv2 extensions, I would prefer MAY to be
>    changed to SHOULD to encourage implementations to include the extensions.

I am OK with that change.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: David Reid <[email-masked]>
Date: Wed, 09 Nov 2011 12:45:39 -0500

> whether the INDEX is IMPLIED or not.

We use the same method routines to supply data to the netconf server and
the SNMP agent (and CLI and web). The current smiv2 extensions provide 
every other useful piece of information from the SMIv2 module except this one.

With this one addition, my yang compiler can read the converted module and
produce method routines that can be used by both netconf and SNMP.
Maybe this is a silly argument since I can of course use my MIB compiler to
read the original MIB document and get the same result, which is what I
normally do. But I can imagine that a netconf/yang shop might prefer to
drive all the tools from yang modules and expect SNMP to come along for free.

Additionally, our sort order is the same for netconf and SNMP. This is not
a requirement, but it seems logical. The implied extension gives the netconf
client a hint about the ordering.

> allowing translations to generate whatever they prefer.

The main reason I wanted to add this was to make the yang module more readable.
It does not help my tools in any way. It also makes it easier for the yang
module writer. Of course, this document is aimed at automated tools doing the
translation. But I plan to use these extensions in some cases to manually 
assign OIDs to objects defined in yang modules so that I can access them from
SNMP.

> idea to stick the status/description/reference statements into the

I was trying to move the information from comments to machine readable fields.
I think the text in the current document was written before we had the alias
extension, so adding comments was the only way to preserve the information.
The only parts I really need are the descriptor and oid. ifXTable would look
like this:

  smiv2:alias ifXTable {
    smiv2:oid "1.3.6.1.2.1.31.1.1";
    description
           "A list of interface entries.  The number of entries is
            given by the value of ifNumber.  This table contains
            additional objects for the interface table.";
  }

> smiv2:oid statement, that is, for the ifEntry I have:

The only reason I wanted alias here was because the descriptor for augmenting
tables is not otherwise in the yang module. It is not needed in the other
sections since the descriptor is translation to yang. I think I do the 
translation a little different than you, see below.

>      smiv2:oid "1.3.6.1.2.1.2.2.1";

I don't create an alias for ifEntry since the descriptor and smiv2:oid are
included in the list definition. I only create an alias for things not
otherwise included like ifMIB, ifMIBObjects, and interfaces.

The problem I had was that I was not translating the descriptor for
augmenting entries like ifXEntry.

ifEntry looks something like this:

    list ifEntry {
      ...

ifXEntry looks somthing like this:

  augment /ifTable/ifEntry {
    smiv2:oid "1.3.6.1.2.1.31.1.1.1";
    description
    ....

The missing piece is the descriptor ifXEntry. I wanted to do something like
this:

  augment /ifTable/ifEntry {
    smiv2:alias "ifXEntry" {
        smiv2:oid "1.3.6.1.2.1.31.1.1.1";
    }
    description

I guess I could just as easily put the alias outside the augment container
and no changes would be required. But I kind of like having it inside the
augment container.

> I am OK with that change.

-David Reid

---

From: David Spakes <[email-masked]>
Date: Wed, 9 Nov 2011 17:39:58 -0500 (EST)

> >       "RFC2578: Structure of Management Information Version 2 (SMIv2)";

For tables that are indexed by a human-readable string, the writer of an
SMIv2 document uses the IMPLIED keyword to specify that the order in
which the data should be presented is alphabetic.  Otherwise, such a
table would be returned in order by length of string from shortest to
longest; e.g., 'fred' returned before 'barney'.

RFC 6020 has a Section 7.7.1 on Ordering that introduces the ordered-by
substatement.  All of the data described by a converted SMIv2 document
represents state data (as far as NETCONF is concerned), so by definition
it is all "system ordered" and the order does not matter.  A NETCONF
client interacting with a multiprotocol entity must be prepared to
accept the data in any order, not just the "typical" order suggested
by the RFC; i.e.,

   For example, a list of valid users would typically be sorted
   alphabetically, since the order in which the users appeared in the
   configuration would not impact the creation of those users' accounts.

Still, I believe the presence or absence of the IMPLIED keyword is a
detail that should not be lost when a document is converted to YANG.
The absence of the IMPLIED keyword may be more meaningful than the
presence of the IMPLIED keyword, but this information would not be
available if the converted document discards it.

-dss

-------------------------------------------------------------
 David Spakes                       email:   [email-masked]
 SNMP Research                      voice:   +1 865 573 1434
 3001 Kimberlin Heights Road          fax:   +1 865 573 9197
 Knoxville, TN  37920-9716  USA      http://www.snmp.com
-------------------------------------------------------------

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 9 Nov 2011 15:26:28 -0800

Hi -

> From: "David Spakes" <[email-masked]>
> To: "Juergen Schoenwaelder" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Wednesday, November 09, 2011 2:39 PM
> Subject: Re: [netmod] ietf-yang-smiv2
...
> which the data should be presented is alphabetic.  ...

But IMPLIED doesn't even accomplish that.  For UTF-8 a naive sort by
code point gives incorrect results for most languages.  Even for pure
ASCII values the order is not alphabetic by any reasonable definition:

"O'Hara" would precede "Oahu" but not "O'connor". 
"aardvark" would follow "ZEBRA", and "Zebra" would
be between "Aardvark" and "aardvark".  IMPLIED doesn't
even accomplish what it set out to do, so I remain to be
convinced that there is any value in carrying that semantic
into the NETCONF world.

Randy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 10 Nov 2011 10:14:01 +0100 (CET)

Hi,

> client a hint about the ordering.

I agree with David, for the same reason.  We do the same - drive both
the SNMP agent and NETCONF server from the same module - and currently
we have a vendor-specific extension for implied.

> assign OIDs to objects defined in yang modules so that I can access them from

Again I am with David.  Our current vendor-specific oid statement
accepts this shorthand notation, for the same reason.

>             additional objects for the interface table.";

I like David's idea.  I prefer a formal statement instead of comments.

> > I am OK with that change.

This makes me wonder what the 2119 language tries to convey in this
document.  What does interoperability mean for this spec?  The text
says that it is used for:

   enabling read-only access to data objects defined in SMIv2 MIB
   modules via NETCONF

With the possible exception of smiv2:oid, none of the extensions are
needed in order to fulfill this goal.

>From this perspective, the extensions should be OPTIONAL.

OTOH, maybe our goal is actually more than just providing read-only
access via NETCONF.  Maybe what we really want is that David's tool
should be able to use the YANG module produced by smidump, and my tool
to use the YANG module produced by David's compiler.  This can happen
e.g. if my NETCONF client uses <get-schema> to read a YANG module from
David's server.

If this is the case, all these extensions should be REQUIRED.

/martin

---

From: David Spakes <[email-masked]>
Date: Thu, 10 Nov 2011 11:25:10 -0500 (EST)

> convinced that there is any value in carrying that semantic

The objective is not to preserve the idiosyncrasies of SNMP in NETCONF.

The objective is to preserve information from the source SMIv2 document
in the converted YANG document.  There is no reason to arbitrarily
discard the IMPLIED information.

Since we are talking about a YANG extension here, the application that
handles the converted document can disregard the IMPLIED information.
It has no obligation to even parse the statement.

-dss

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 11 Nov 2011 22:20:24 +0100

> accepts this shorthand notation, for the same reason.

Are we producing extensions for machine generated translations or for
stuff written by humans? For machine generation and processing, I
think a simple predicatable format is to be preferred.
 
> I like David's idea.  I prefer a formal statement instead of comments.

So if we do this, would if be valid to repeat descriptions etc. for
other mappable constructs as well?
 
> needed in order to fulfill this goal.

Even the smiv2:oid is not needed for this.

> If this is the case, all these extensions should be REQUIRED.

What do _you_ think is the goal?

Implementation wise, all this does not really matter since generating
the smiv2:* statements is not a big deal and ignoring the smiv2:*
statements in a YANG document is also easy. And both David's and my
implementation have the code, the question is whether it is
permanently on or not. Perhaps you are right, we should either turn it
permanently on or really treat it as an optional feature. I personally
do not care much.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 11 Nov 2011 23:31:47 +0100 (CET)

> What do _you_ think is the goal?

Here's a use case.  Suppose you want to implement a proxy that
provides this read-only access to SNMP data over NETCONF, i.e., a
NETCONF server that speaks SNMP southbound to provide this data (which
happens to be close to what I am working on right now).  In this case,
the "oid" statement is necessary, as is "implied".  The others
("display-hint", "defval" etc), are not really important for this use
case.

So, I prefer the extensions to be REQUIRED.  If get a module converted
by some tool that is compliant with this spec, I can use it in my
proxy.  If these extensions are OPTIONAL, I cannot work with all
compliant tools.

Since non-important extensions are trivial to ignore, it is not a
problem for YANG tools to always get them.

/martin

---

From: David Spakes <[email-masked]>
Date: Fri, 11 Nov 2011 17:41:51 -0500 (EST)

> problem for YANG tools to always get them.

I am also working on multiprotocol entities that will support both
NETCONF and SNMP (proxy being one example).  I agree with Martin.
I prefer that the extensions be REQUIRED.

-dss

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 14 Nov 2011 09:23:48 +0100 (CET)

Hi,

>    enabling read-only access to data objects defined in SMIv2 MIB

The document says that all nodes MUST be config false, which is
natural since the interoperability goal is read-only access.  But I
think we should allow implementations to mark nodes as config true, in
an implementation dependent manner.  This would not affect our
read-only interoperability goal.

We already know from this ML that some vendors (including myself) will
provide config true access to parts of the MIBs.  (We're using a bunch
of directives to the compiler to control which parts).  As the
document is written today, the output from my tool is non-compliant,
but I do not think this is necessary.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 14 Nov 2011 10:22:28 +0100

> but I do not think this is necessary.

If my tool generates config false but yours does config true, will a
YANG manager not be potentially confused? Both will have the same
revision timestamp.

/js

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Nov 2011 07:31:17 -0500

>but I do not think this is necessary.

Wouldn't this be a list of deviations that your tool would
generate (read-only as required plus a deviation)?

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 14 Nov 2011 18:30:58 +0100 (CET)

> YANG manager not be potentially confused? Both will have the same

A client can only rely on config false for full interoperability, just
like today, I don't want to change that.  So if a client that knows a
module w/ config false gets a capability with the right namespace and
revision, it knows how to <get/> from the server, regardless of how
many config trues that server is using.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 14 Nov 2011 18:31:45 +0100 (CET)

> generate (read-only as required plus a deviation)?

I would have to use deviations if I am not compliant.  But I would
like to be compliant, even if I have a few config trues in there.

/martin

---

From: David Reid <[email-masked]>
Date: Mon, 14 Nov 2011 12:36:15 -0500

> but I do not think this is necessary.

Our tool also allows config true under some conditions.

-David Reid

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Nov 2011 12:59:09 -0500

>like to be compliant, even if I have a few config trues in there.

So you want "config maybe-if-i-feel-like-it"?  I think the
spec needs to be one way or the other, so you have some
consistency.  Clients that are open to treating these as
config can use your deviations to find that.

All this points out the weakness of not having a clear way to bind
operational data to configuration data, which is especially important
when the set of possible values are distinct and one can't treat
them using deviations like this.  We really need a mechanism to say
"this knob is the config version of that knob".  Otherwise only the
description tells me where to find the current link speed for an
interface when I say "link-speed auto".

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 14 Nov 2011 11:12:21 -0800

> interface when I say "link-speed auto".

We have description-stmt for this purpose.
There are many different semantic relationships possible between data nodes.
A special mechanism for the admin-duplex=auto and oper-duplex=N corner-case
is too simplistic.  Until a formal language definition is developed,
ad-hoc text (i.e., description-stmt) is the only reasonable common solution.

IMO, a more interesting operational state problem is 'operational config' --
data nodes set only by the server at runtime to some non-constant value,
which must persist with the data node across a reboot (e.g.,. UUID assigned by server).
(But this is not a thread on operational data, so back to your original topic...)

>   Phil

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 14 Nov 2011 20:25:21 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> So you want "config maybe-if-i-feel-like-it"?

Not at all.

I want this translation algorithm to allow implementations to treat
some nodes as config.

> I think the
> operational data to configuration data

I agree with you on the importance of that, but I think that is a
different topic.  I would love a solution to this problem; it pops up
every now and then, and we don't have a good answer.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Mon, 14 Nov 2011 15:02:57 -0500

>I want this translation algorithm to allow implementations to treat

Maybe I'm lost; I thought you wanted the output of the translation
algorithm to be standard, meaning there's no room for implementation
differences like this.

>every now and then, and we don't have a good answer.

The only answer that works IMHO is to have one of the leaf pairs
point to the other, like:

    leaf oper-link-speed {
        config false;
        ...
    }
    leaf link-speed {
        config true;
        config-knob-for ../oper-link-speed;
    }

Using the description for this means hand coding the bindings for
each pair, which is just bad news.

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 15 Nov 2011 03:15:16 +0100

> I want this translation algorithm to allow implementations to treat

Which one? We are talking about an algorithm so there needs to be an
algorithmic answer.

/js