Subject: [netmod] [[email-masked]: New Version Notification for draft-schoenw-netmod-smi-yang-02]

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 14 Mar 2011 23:08:04 +0100

Hi,

here is an update of the SMIv2 to YANG translation document. Comments
are more than welcome.

/js

----- Forwarded message from IETF I-D Submission Tool <[email-masked]> -----

Date: Mon, 14 Mar 2011 14:59:09 -0700
From: IETF I-D Submission Tool <[email-masked]>
To: [email-masked]
Subject: New Version Notification for draft-schoenw-netmod-smi-yang-02
Message-ID: <[email-masked]>

A new version of I-D, draft-schoenw-netmod-smi-yang-02.txt has been successfully submitted by Juergen Schoenwaelder and posted to the IETF repository.

Filename:	 draft-schoenw-netmod-smi-yang
Revision:	 02
Title:		 Translation of SMIv2 MIB Modules to YANG Modules
Creation_date:	 2011-03-14
WG ID:		 Independent Submission
Number_of_pages: 31

Abstract:
YANG is a data modeling language used to model configuration and
state data manipulated by the NETCONF protocol, NETCONF remote
procedure calls, and NETCONF notifications.  The Structure of
Management Information (SMIv2) defines fundamental data types, an
object model, and the rules for writing and revising MIB modules for
use with the SNMP protocol.  This document defines a translation of
SMIv2 MIB modules into YANG modules, enabling read-only access to
data objects defined in SMIv2 MIB modules via NETCONF.

The IETF Secretariat.

----- End forwarded message -----

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: David Reid <[email-masked]>
Date: Fri, 25 Mar 2011 17:10:41 -0400

> here is an update of the SMIv2 to YANG translation document. Comments

I have a few comments and questions on the three TODO: items and a few other 
items. Overall, I think this is a good draft which the working group should 
standardize. We plan to implement it (We have a partial implementation from a 
long time ago that we will update to match the draft).

Section 6.1 DISPLAY-HINTS TODO item: I think the current text is 
sufficient. I would also be OK with just not translating it at all. 
If someone wants to define a more complete translation algorithm, I 
would have no objection. But I don't really think it is worth the effort. 
I would like to see an additional sentence saying that the display-hint 
extension from ietf-yang-smiv2 should be used. In fact, I would like to 
see that sentence in the other places where it makes sense to use 
extensions from that module.

Section 7.1 TODO: Can someone explain the issue here. I didn't understand 
the problem.

Section 9.2 TODO, identities: I think the OBJECT-IDENTITY macro should be 
mapped to a yang identity rather than a container (and generate the 
appropriate base).

Section 2: Using DISPLAY-HINT to determine whether the yang type is string 
or binary is problematic because SMIv2 revision rules allow DISPLAY-HINT to 
be added, but that would change the yang type, which is not a legal revision.
I hate to always use binary since most OCTET STRINGS in MIBs are text strings,
so I hope we can come up with a better solution.

Section 4 states "Since SMIv2 module names are unique...", but SMIv2 module 
names are not guaranteed to be unique. Maybe it's OK to make this assumption,
otherwise, I don't know how we'll get unique namespaces.

Section 6.2: pattern "\p{IsBasicLatin}{0,255}". Should this be 
"\\p{IsBasicLatin}{0,255}"?

In notifications, why are the non-index objects not of type leafref?

-David Reid

---

From: David Spakes <[email-masked]>
Date: Fri, 25 Mar 2011 20:27:32 -0400 (EDT)

Juergen,

I have a few comments on the SMIv2 to YANG translation document.
Good job, by the way!

[1]  On page 6, I was confused by the statement: "on average, much shorter
prefxies are generated."  Are you planning to grow the list in Table 2
with all of the pulished RFCs containing MIB modules?

[2]  In Table 3 on page 7, SNMPv2-TC is supposed to be an ignored IMPORTS,
but your example in section 4.1 on page 8 does not ignore it.

[3]  Typo in section 4.1 on page 8:
"(consisting of two token  and this no further abbrevation)."
.........................s

[4]  Typo in section 8.1 on page 14:
"Embedded clauses are generates as described in Section 2."
..............................d

[5]  Regarding the mapping of SMIv2 types to YANG types, I have a suggestion
for OCTET STRING.  According to RFC 2579, page 24, "(4)  A DISPLAY-HINTS
clause may be added or updated."  In practice, I believe there are a lot
of OCTET STRING objects that are intended to contain only printable
strings, but the MIB document does not [yet] provide a DISPLAY-HINT.
For example, RFC 1213 defines a DisplayString on page 12 as:

          DisplayString ::=
              OCTET STRING

Then, sysDescr is defined with SYNTAX DisplayString on page 13:

          sysDescr OBJECT-TYPE
              SYNTAX  DisplayString (SIZE (0..255))
              ACCESS  read-only
              STATUS  mandatory

Later, RFC 3418 (SNMPv2-MIB) added the DISPLAY-HINTS to sysDescr by
importing the definition of DisplayString from SNMPv2-TC (RFC 2579).

I propose that an OCTET STRING object that does not have a DISPLAY-HINT
should convert to a union of string and binary so a NETCONF server has
some flexibility on how these are handled.  Perhaps in this situation,
a smart NETCONF server should able to discriminate the enclosed value
by adding a double-quote character to the beginning and end; e.g.,

    typedef OctetString {
       type union {
          type string {
              pattern "\".*\""
          }
          type binary;
       }
    }

[6]  Similar to OCTET STRING, I have a concern about the conversion of
an enumerated INTEGER to a YANG enumeration.  A leaf of type enumeration
can not be extended, but new enumerations are sometimes added to MIBs.
The most recognizable example is ifType:

   http://www.iana.org/assignments/ianaiftype-mib

As the mapping is stated today, the definition of ifType in RFC 1213 would
be converted to this:

    leaf ifType {
      type enumeration {
        enum other                  { value 1; }
        enum regular1822            { value 2; }
        enum hdh1822                { value 3; }
        enum ddn-x25                { value 4; }
        enum rfc877-x25             { value 5; }
        enum ethernet-csmacd        { value 6; }
        enum iso88023-csmacd        { value 7; }
        enum iso88024-tokenBus      { value 8; }
        enum iso88025-tokenRing     { value 9; }
        enum iso88026-man           { value 10; }
        enum starLan                { value 11; }
        enum proteon-10Mbit         { value 12; }
        enum proteon-80Mbit         { value 13; }
        enum hyperchannel           { value 14; }
        enum fddi                   { value 15; }
        enum lapb                   { value 16; }
        enum sdlc                   { value 17; }
        enum ds1                    { value 18; }
        enum e1                     { value 19; }
        enum basicISDN              { value 20; }
        enum primaryISDN            { value 21; }
        enum propPointToPointSerial { value 22; }
        enum ppp                    { value 23; }
        enum softwareLoopback       { value 24; }
        enum eon                    { value 25; }
        enum ethernet-3Mbit         { value 26; }
        enum nsip                   { value 27; }
        enum slip                   { value 28; }
        enum ultra                  { value 29; }
        enum ds3                    { value 30; }
        enum sip                    { value 31; }
        enum frame-relay            { value 32; }
      }
      config false;
      description
             "The type of interface, distinguished according to
              the physical/link protocol(s) immediately `below'
              the network layer in the protocol stack.";
    }

It would not be legal to add more enumerations; e.g., rs232(33).

The page http://www.yang-central.org/twiki/bin/view/Main/YangFAQ#whenempty
suggests a technique of using a choice of type empty instead of enumeration,
and then it is possible to use an augment to add enumerations.  But I don't
really like this approach.  I would rather see the enumerations remain
enumerations.

If ifType were instead converted to a union of enumeration and int32,
then it would be possible to convey any new enumerated values as their
numeric equivalents.

    leaf ifType {
      type union {
        type enumeration {
          enum other                  { value 1; }
          enum regular1822            { value 2; }
          enum hdh1822                { value 3; }
          enum ddn-x25                { value 4; }
          enum rfc877-x25             { value 5; }
          enum ethernet-csmacd        { value 6; }
          enum iso88023-csmacd        { value 7; }
          enum iso88024-tokenBus      { value 8; }
          enum iso88025-tokenRing     { value 9; }
          enum iso88026-man           { value 10; }
          enum starLan                { value 11; }
          enum proteon-10Mbit         { value 12; }
          enum proteon-80Mbit         { value 13; }
          enum hyperchannel           { value 14; }
          enum fddi                   { value 15; }
          enum lapb                   { value 16; }
          enum sdlc                   { value 17; }
          enum ds1                    { value 18; }
          enum e1                     { value 19; }
          enum basicISDN              { value 20; }
          enum primaryISDN            { value 21; }
          enum propPointToPointSerial { value 22; }
          enum ppp                    { value 23; }
          enum softwareLoopback       { value 24; }
          enum eon                    { value 25; }
          enum ethernet-3Mbit         { value 26; }
          enum nsip                   { value 27; }
          enum slip                   { value 28; }
          enum ultra                  { value 29; }
          enum ds3                    { value 30; }
          enum sip                    { value 31; }
          enum frame-relay            { value 32; }
        }
        config false;
        description
               "The type of interface, distinguished according to
                the physical/link protocol(s) immediately `below'
                the network layer in the protocol stack.";
      }
      type int32;
    }

So, the encoding for any of the original enumerations would be as

   <ifType>ethernet-csmacd</ifType>

and for any new enumerations it would be as:

   <ifType>33</ifType>

MIB documents that never extend their enumerations would not be affected
by the switch to a union conversion.

[7]  I have concerns about YANG-centric applications that will inevitably
be created in the future that need to access SNMP agents in deployed
devicess as a source of raw data.  My main concern is that YANG
documents converted from MIB documents won't necessarily have all of
the necessary information.  It disturbs me that a lot of information
from the original MIB document is lost when it is converted to YANG;
e.g., in several places:

  "The SMIv2 ... clause is ignored."

The YANG Language Extension Definition provides the means for information
that would otherwise be lost during the conversion to be preserved in
the resulting YANG document.  I suggest that more extensions should be
added to section 11 to preserve even more of the MIB information;
e.g., UNITS, the original SYNTAX of an object (so the application can
know if a string was converted from a DisplayString or UTF8String, for
example), etc.

Section 11 counts for nothing, though, if the use of extensions is optional.
I would like to propose that every occurrance of

  "The SMIv2 ... clause is ignored"

in this draft should be changed in the next draft to

  "The SMIv2 ... clause MUST be converted to extension ... as described
   in Section 11."

This requirement to generate the extensions in the resulting YANG document
in no way impacts YANG compilers that care nothing about SNMP, since
"unknown-statement MAY be ignored by the compiler." (RFC 6020, page 38)

-dss

On Mon, 14 Mar 2011, Juergen Schoenwaelder wrote:

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

-------------------------------------------------------------
 David Spakes                       email:   [email-masked]
 SNMP Research                      voice:   +1 865 573 1434
 3001 Kimberlin Heights Road          fax:   +1 865 573 9197
 Knoxville, TN  37920-9716  USA      http://www.snmp.com
-------------------------------------------------------------

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 28 Mar 2011 12:28:13 +0200

> I have a few comments on the SMIv2 to YANG translation document.

Thanks for the review, this is highly appreciated.
 
> with all of the pulished RFCs containing MIB modules?

No, table 2 is supposed to be a fixed list of well known special
prefixes. It is built statically into my implementation.
 
> but your example in section 4.1 on page 8 does not ignore it.

Table 3 says that the import of 'TEXTUAL-CONVENTION' from the
'SNMPv2-TC' is ignored. It does that say anything about say RowStatus.

> .........................s

fixed 
 
> ..............................d

fixed

> by adding a double-quote character to the beginning and end; e.g.,

How we distinguish textual strings from binary strings is now an open
issue. The union approach, though a nice idea, can cause ambiguities
unless we introduce something that reliably distinguishes string
values from binary values (represented in base64 encoding). The quotes
serve that purpose, but they likely also get into the way since we
start to mess around with the value contained in an XML element. This
requires more discussion I think.

> can not be extended, but new enumerations are sometimes added to MIBs.

What makes you believe enums can not be extended in YANG? RFC 6020 says:

   o  An "enumeration" type may have new enums added, provided the old
      enums's values do not change.

That said, SMIv2 allows to change the label associated with an
enumerated INTEGER:

(1)  A SYNTAX clause containing an enumerated INTEGER may have new
     enumerations added or existing labels changed.  Similarly, named
     bits may be added or existing labels changed for the BITS
     construct.

YANG does not seem to allow such changes.

> by the switch to a union conversion.

See above, I think adding enumerations is not a problem. The problem I
see is that SMIv2 allows to change labels.

> know if a string was converted from a DisplayString or UTF8String, for

The UNITS clauses are translated to YANG units statements (perhaps I
am missing that somewhere) so there is nothing lost. A DisplayString
SYNTAX is translated into a type statement using
snmpv2-tc:DisplayString so nothing is lost here either.

> "unknown-statement MAY be ignored by the compiler." (RFC 6020, page 38)

Whether the generation of these extensions statements is a MUST
generate or a SHOULD generate needs to be discussed. I can go either
way. In any case, I agree that the "is ignored" text needs to be
rewritten.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 28 Mar 2011 12:28:22 +0200

> long time ago that we will update to match the draft).

Thanks David for the review.
 
> extensions from that module.

OK

> Section 7.1 TODO: Can someone explain the issue here. I didn't understand 

In SMIv2, this is perfectly legal:

foo OBJECT IDENTIFIER ::= { some 1 }
bar OBJECT IDENTIFIER ::= { some 1 }

In other words, you can have multiple descriptors for a single OID. If
you generate a nested container structure in YANG, which of the
multiple descriptors do you choose? Note that additional descriptors
can be added and the order of appearance can change as well when a
module is revised.

> mapped to a yang identity rather than a container (and generate the 

Even though wired, you can do

foo OBJECT-IDENTITY
     STATUS current
     DESCRIPTION ""
     ::= { some 1 }

bar OBJECT-TYPE
     -- stuff removed
     ::= { foo 1 }

and hence the container may be needed if we want to recreate the
hierarchical structure in YANG (but see above).

> so I hope we can come up with a better solution.

Good catch, bad news. I agree that relying on DISPLAY-HINT is broken,
not sure what a better solution could be.

> otherwise, I don't know how we'll get unique namespaces.

RFC 2478 says:

   The names of all standard information modules must be unique (but
   different versions of the same information module should have the
   same name).  Developers of enterprise information modules are
   encouraged to choose names for their information modules that will
   have a low probability of colliding with standard or other enterprise
   information modules.

My understanding here is that modules names really must be unique
because SMIv2 imports simply do not work if module names are not
unique. However, there is no registry to enforce that module names are
unique and hence this encouraging text to choose module names with low
collision probability. In other words, an SMIv2 parser (at least the
once I have seen) already makes the assumption that module names are
unique.

> "\\p{IsBasicLatin}{0,255}"?

What makes you believe there should be two slashes?

> In notifications, why are the non-index objects not of type leafref?

I have to look at this. Perhaps this is simply dating back from the
times where leafref was a keyref...

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 28 Mar 2011 05:32:18 -0700

> On Fri, Mar 25, 2011 at 05:10:41PM -0400, David Reid wrote:
...
> times where leafref was a keyref...

If the notification varbind identifies an OBJECT-TYPE macro that is MAX-ACCESS
accessible-for-notify, then the YANG leaf will be inline within the notification.

What if the varbind is MAX-ACCESS not-accessible (INDEX clause)?
Then is a leafref used or inline?  My guess: leafref since keys are accessible in YANG.

sec. 9, bullet 1:

typo:
       The name of this container
       is the name of the notification and the name of the current
       concatenated by a hyphen.

name of the current object, concatenated ...

comment:

What if the OBJECTS clause contains objects that have the same descriptor
but from different modules  (FOO-MIB.x and BAR-MIB.x)?

Then you have container notif-x and container notif-x as siblings which is illegal
in YANG.

> /js

Andy

---

From: David Reid <[email-masked]>
Date: Mon, 28 Mar 2011 09:33:44 -0400

> What makes you believe there should be two slashes?

I thought double-quoted strings in yang required a backslash to be
represented as \\.

-David Reid

---

From: David Reid <[email-masked]>
Date: Mon, 28 Mar 2011 09:41:31 -0400

> once I have seen) already makes the assumption that module names are

I agree. Our tools make that assumption. It just bothered me to see the
statement that "SMIv2 module names are unique" when the rfc only suggests
picking names with a low probability of colliding. But I think I'm being 
too picky and your text is just fine.

-David Reid

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 28 Mar 2011 15:42:16 +0200

> Then is a leafref used or inline?  My guess: leafref since keys are accessible in YANG.

Yes, I think leafrefs should just work.

fixed

> Then you have container notif-x and container notif-x as siblings which is illegal

You can have the same if you include the same object twice in a
notification. I checked my code and to my surprise it handles this
situation by generating the names notif-x and notif-x-2. ;-)

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 28 Mar 2011 17:09:50 +0200

> I thought double-quoted strings in yang required a backslash to be

Indeed. I changed to single quotes. Thanks.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 28 Mar 2011 17:24:34 +0200 (CEST)

> Indeed. I changed to single quotes. Thanks.

Actually, 6020, section 6.1.3 is not clear on what "\p" means... It
could mean that since \p is not listed as a special combination, it
means '\p'.  Or it could mean that since \p is not special, it means
'p'.

pyang treats it as '\p'.  What do other implementations do?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 28 Mar 2011 09:16:40 -0700

On 03/28/2011 08:24 AM, Martin Bjorklund wrote:
> Juergen Schoenwaelder<[email-masked]>  wrote:

yangdump does the same thing.  Only the special sequences listed in 6.1.3 are converted.

Just found a bug in yangdump --format=yang or --format=html.  The converted string
replaces the original string instead of preserving the original format.
So  description "\\t\"" turns into description "\t"", which is of course invalid.
oops.

Andy