Subject: [netmod] YANG 1.1 issue: data-specific attributes

From: Andy Bierman <[email-masked]>
Date: Tue, 29 Apr 2014 10:53:56 -0700

Hi,

There is a need to support meta-data for data returned by a NETCONF
or RESTCONF server.  YANG only supports attributes with hacks
like the <get> operation in ietf-netconf.yang:

http://www.netconfcentral.org/modules/ietf-netconf/2011-06-01#get-filter-element-attributes.56

The most common attribute (that applies to a subset of all data definitions)
is the "xml:lang" attribute used in the error-message field.

      error-message:  Contains a string suitable for human display that
      describes the error condition.  This element will not be present
      if no appropriate message is provided for a particular error
      condition.  This element SHOULD include an "xml:lang" attribute as
      defined in [W3C.REC-xml-20001006] and discussed in [RFC3470].

We implement attributes in YANG with yet another YANG extension called
'metadata'.
It is used in the 'LangString' typedef to model <error-message> correctly.

http://www.netconfcentral.org/modules/yuma-netconf/2013-07-07#LangString.215

The use of YANG extensions is problematic because they are completely
optional
and outside the language (even the <get> hack in RFC 6241).  A normative
solution
is needed.

Proposed Solution:

  * A general mechanism is needed to declare data-specific and
typedef-specific metadata

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 29 Apr 2014 20:24:23 +0200

On 29 Apr 2014, at 19:53, Andy Bierman <[email-masked]> wrote:

> We implement attributes in YANG with yet another YANG extension called 'metadata'.
> The use of YANG extensions is problematic because they are completely optional
> and outside the language (even the <get> hack in RFC 6241).  A normative solution
> is needed.
> 
> 
> Proposed Solution:
> 
>   * A general mechanism is needed to declare data-specific and typedef-specific metadata

Is it something else than Y33?

Lada

> 
> 
> Andy
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 29 Apr 2014 11:35:56 -0700

> Is it something else than Y33?
Yes -- as it says -- this is data-specific meta-data.
Note this text from issue Y33:

  A new YANG statement, 'attribute', is proposed to allow defining such
  attributes. It would be allowed only at the top level of a module so
  that it could be used for defining global attributes that can be
  attached to any data node.

How does the client know which leafs use the xml:lang attribute?
Which leafs have mandatory xml:lang attributes and which ones are optional?

A top-level statement might be part of the solution for this problem.
I am just identifying the problem.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 29 Apr 2014 20:52:25 +0200

> How does the client know which leafs use the xml:lang attribute?

It may be attached to any element, not only leafs. This is how it works in XML.

> Which leafs have mandatory xml:lang attributes and which ones are optional?

So you want to model attributes in the same way as the XML schema languages do? I think this would be a significant change to the design of YANG - attributes were deliberately left out.

Practically speaking, I can see the need for global attributes but not for local ones. What are the use cases that cannot be modelled in YANG now, e.g. via changing a leaf to a container?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 29 Apr 2014 12:32:19 -0700

> It may be attached to any element, not only leafs. This is how it works in
I am talking about YANG schema. How XML works is not important.

> So you want to model attributes in the same way as the XML schema
> languages do? I think this would be a significant change to the design of
> YANG - attributes were deliberately left out.
>
>
I did not specify the syntax of any solution.
The ncx:metadata uses 1 line.  It is not XSD.
Nobody suggesting using XSD.

> Practically speaking, I can see the need for global attributes but not for
> local ones. What are the use cases that cannot be modelled in YANG now,
> e.g. via changing a leaf to a container?
>
>
The global attribute is not that useful.
Associating specific metadata with specific data nodes is more useful.

Not all nodes are strings, so we have the YANG type-stmt
to clarify the generic XML field. Not all strings are administrative
strings that need the language identified. (So we need a sub-statement
to specify that xml:lang attribute is expected).

YANG parsers already have to accept a new subtree instead of ';'
after every statement. The must-stmt has sub-trees.  Adding a complex
statement, if that was the solution, would not be any more complex than now.

Lada

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 30 Apr 2014 08:02:39 +0200 (CEST)

Hi,

Added as Y46.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 30 Apr 2014 09:45:42 +0200

> I am talking about YANG schema. How XML works is not important.

You took xml:lang as a use case. The definition of error-message in RFC 4741 predated YANG, so xml:lang seemed to be a logical choice for specifying the language at that time. With YANG, it is not necessarily the case.

> Nobody suggesting using XSD.

Neither do I - I am talking about making attributes first-class data modelling objects in YANG.

> Associating specific metadata with specific data nodes is more useful.

Metadata associated with specific nodes can be modelled as standard YANG data nodes, perhaps using a grouping.

My serious concern is that people will start using attributes as normal data nodes in YANG, for example

<ip-address netmask="255.255.0.0">1.2.3.4</ip-address>

Extensive use of attributes is also rather unfriendly to JSON encoding.  

> to specify that xml:lang attribute is expected).

Nothing prevents you from using a language *leaf* where needed. For me, a typical use case for attributes is "inactive", which cannot be modelled this way.

> statement, if that was the solution, would not be any more complex than now.

Shall we end up with something as large as XSD and twice as natural?

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 30 Apr 2014 10:00:00 +0200 (CEST)

> nodes, perhaps using a grouping.

But this it what we want to avoid...

> My serious concern is that people will start using attributes as normal data
> nodes in YANG

+1

I agree with you that having a mechanism to define global meta data is
the right way to go.  It is quite simple, even if it doesn't solve all
use cases as strict as you might like.  data node specific meta data
can be be handled with global meta data, even if it isn't as elegant.

/martin

---

From: [email-masked]
Date: Wed, 30 Apr 2014 16:07:43 +0800

Ladislav Lhotka <[email-masked]> wrote:
> Metadata associated with specific nodes can be modelled as standard YANG 
data
> nodes, perhaps using a grouping.

+1

/martin

I suggest data node specific meta data  must only be used in get opertion.

global meta data can be used in set or get operation.
/frank
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s).  If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited.  If you have received this mail in error, please delete it and notify us immediately.

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 30 Apr 2014 10:19:26 +0200

On 30 Apr 2014, at 10:00, Martin Bjorklund <[email-masked]> wrote:

> But this it what we want to avoid…

What’s wrong with that?

> can be be handled with global meta data, even if it isn't as elegant.

I guess it depends on how you define metadata. In a sense, “inactive” or “nc:operation” is more “meta” than say “language” or “media-type”.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 30 Apr 2014 02:21:19 -0700

On Wed, Apr 30, 2014 at 1:19 AM, Ladislav Lhotka <[email-masked]> wrote:

> >> Metadata associated with specific nodes can be modelled as standard
> YANG data
this is a strawman.
The solution does not have to be inappropriate or abused.

> What's wrong with that?

data model designers can decide what is appropriate

> >> My serious concern is that people will start using attributes as normal

people could be designing containers when they should be using leafs.
There are lots of ways one can make bad decisions with YANG.

> I guess it depends on how you define metadata. In a sense, "inactive" or
> "nc:operation" is more "meta" than say "language" or "media-type".
>

very good - you picked another attribute that can apply globally.
You are ignoring the 'language string' extension since it clearly does not
apply
to every data node.  It is not XML-specific (see the NCX extension).
Adding <language> sibling leafs everywhere there is an admin string is not
practical.

The point of the YANG schema is to let the client know what the server will
implement,
such as which metadata will be returned for which nodes.

Lada

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 30 Apr 2014 02:56:10 -0700

> > My serious concern is that people will start using attributes as normal
I did not mention that these attributes are read-only, so it would be
impossible
to replace config leafs with attributes.  This abuse you are concerned about
is not possible.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 30 Apr 2014 12:04:37 +0200 (CEST)

> to replace config leafs with attributes.  This abuse you are concerned about

This seems like an arbitrary limitation.  The "inactive" attribute is
an example of a read/write (global) attribute.  Why would data-node
specific attributes be read only?

For both global and local attributes, it must be possible to define
their read/write properties.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 30 Apr 2014 05:33:16 -0700

On Wed, Apr 30, 2014 at 3:04 AM, Martin Bjorklund <[email-masked]> wrote:

> YANG
> > > > My serious concern is that people will start using attributes as
> normal
> > > data
> > > > nodes in YANG
> > >
> > > +1
> > >
> > >
why is this mandatory for local?

---

From: Andy Bierman <[email-masked]>
Date: Wed, 30 Apr 2014 05:48:11 -0700

> their read/write properties.

So what is supposed to stop someone from defining "netmask" as a
global attribute instead of a config leaf?  Global vs. local is irrelevant,
except global is worse because the YANG schema does not say
the <address> leaf is going to reject any edit that does not include
the 'netmask' attribute.

Using attributes for editing doesn't work.  If you make a CLR that global
attributes
can't be "part of the data model", then the same CLR can be made for local
attributes.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 30 Apr 2014 14:49:48 +0200

> data model designers can decide what is appropriate

XML attributes were quite often misused, and the schema designers probably also decided it was appropriate. IMO, that’s why JSON (so far) has resisted all attempts to define attributes.

> You are ignoring the 'language string' extension since it clearly does not apply
> Adding <language> sibling leafs everywhere there is an admin string is not practical.

Is the role of <language> specified for a text any different from the role of <address-family> specified for a network address or route? Where is the borderline between data and metadata?

> 
> The point of the YANG schema is to let the client know what the server will implement,

Metadata can be returned as normal data. Attributes might look more compact in XML encoding but certainly not in JSON. This would be the encoding according to draft-ietf-netmod-yang-json-00:

"message": "…",
"@message": {
  "language": "en"
} 

I think the following model is better in that it works for both XML and JSON:

container message {
  uses lang-string;
}

and (maybe in another module)

grouping lang-string {
  leaf language {
    type language-tag;
    default "en";
  }
  leaf text {
    type string;
  }
}

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 30 Apr 2014 15:02:53 +0200

> global attribute instead of a config leaf?  Global vs. local is irrelevant,

Nothing except it would look extremely stupid.

> Using attributes for editing doesn't work.  If you make a CLR that global attributes

The significant difference is that global attributes can be defined without changing YANG in any way, e. g. via capabilities. I only think it is more convenient to do it in a YANG module.

Local attributes that need to be placed in specific places of the data tree require making them a new data node type. This IMO already qualifies as a "fundamentally new data modeling concept”.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 30 Apr 2014 06:17:37 -0700

On Wed, Apr 30, 2014 at 6:02 AM, Ladislav Lhotka <[email-masked]> wrote:

> > > > > >> Practically speaking, I can see the need for global attributes
> but
> > > > not for
> > > > > >> local ones. What are the use cases that cannot be modelled in
> YANG
> > > > now,

???

so why wouldn't it also be stupid to do as a local attribute?
So you would allow 'netmask' as a global attribute, right?
But there is no way to indicate which leafs require the attribute to be
present.

In the general case, any attribute could be considered mandatory to provide
with any data node. How does the client know what to send?

> The significant difference is that global attributes can be defined
> without changing YANG in any way, e. g. via capabilities. I only think it
> is more convenient to do it in a YANG module.
>
> Local attributes that need to be placed in specific places of the data
> tree require making them a new data node type. This IMO already qualifies
> as a "fundamentally new data modeling concept".
>

???
Any new statement in YANG  is the same here.
You cannot add an "attribute-stmt" to YANG 1.1 using a protocol capability.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 30 Apr 2014 16:51:01 +0200

> So you would allow 'netmask' as a global attribute, right?

No, I would fiercely oppose anything like this becoming part of any standard.

The point is that if attributes become data nodes, people will use and misuse them as they did in XML. On the other hand, there is no benefit in defining “netmask” as a global attribute.

> But there is no way to indicate which leafs require the attribute to be present.

No, because global attributes convey information that’s by (my) definition not part of the data model. If the server indicates support for disabling any part of the data tree, it doesn’t mean the “inactive” attribute becomes part of any data model. The same goes for the attributes defined in NETCONF.

> with any data node. How does the client know what to send?

This can be again specified without any reference to a data model. It was possible even before YANG. 

> You cannot add an "attribute-stmt" to YANG 1.1 using a protocol capability.

No, but I can use a protocol capability for defining a new global attribute with specific semantics that will work even with YANG 1.0 - that’s why it cannot be considered a new data modeling concept.

The advantage of having a new “attribute” statement in YANG is that

- the attribute will automatically get a namespace and recommended prefix,

- the namespace encoding is also readily defined in JSON,

- it can possibly be assigned a datatype,

- descriptions, references etc. can be used for specifying the attribute’s semantics.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 30 Apr 2014 08:14:31 -0700

On Wed, Apr 30, 2014 at 7:51 AM, Ladislav Lhotka <[email-masked]> wrote:

> > > > > > >> Practically speaking, I can see the need for global
> attributes but
> > > > > not for
> The point is that if attributes become data nodes, people will use and
> misuse them as they did in XML. On the other hand, there is no benefit in
> defining "netmask" as a global attribute.
>
>

I agree on that point.  I do not want data model properties defined
in attributes.

But you missed my point -- if we can define these rules for global
attributes, we obviously can define them for local attributes as well.

> No, because global attributes convey information that's by (my) definition
> not part of the data model. If the server indicates support for disabling
> any part of the data tree, it doesn't mean the "inactive" attribute becomes
> part of any data model. The same goes for the attributes defined in NETCONF.
>
>

1) there is really no way for the YANG compiler to tell the difference
between
an attribute for "enabled" vs. an attribute for "netmask".

2) global attributes that only apply to a subset of nodes are needed, such
   as the language of an administrative string.

Here is a solution to demonstrate that we are actually in agreement::

 - there are only global attributes, which are not allowed to contain data
model
   properties.  The attribute will define if it is read-write/read-only,
etc.

 - there is also a sub-statement to use specific attributes.

attribute language {
   description
     "Identifies the language for the associated string data type.
      as defined in [W3C.REC-xml-20001006] and discussed in [RFC3470]."
 }

 leaf admin-string {
   type string;
   uses-attribute foo:language;
 }

> No, but I can use a protocol capability for defining a new global
> attribute with specific semantics that will work even with YANG 1.0 -
> that's why it cannot be considered a new data modeling concept.
>
>
this is what we have today

> The advantage of having a new "attribute" statement in YANG is that
>
> - descriptions, references etc. can be used for specifying the attribute's
> semantics.
>
>
So some attributes will be defined in the protocol and accessible via XML
only,
and some will be defined in YANG, so they can be accessed in XML and JSON.
Isn't that a problem? How will protocol defined attributes be supported in
JSON?

Lada
Andy