Subject: Re: [netmod] metadata in JSON

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 26 Aug 2014 15:05:25 +0200

Hi,

I would like to resolve the issue of metadata encoding in JSON, so let
me summarize the options we have. There are actually two valid questions:

1. Does the specification of the mapping of XML attributes in JSON
   belong to the yang-json document, and if so, in what form?

Possible answers:

1.1. No, as YANG doesn't model XML attributes.
1.2. Yes, as normative text.
1.3. Yes, as non-normative text.

For 1.2 and 1.3, we have to answer the second question because I assume
most XML attributes in use with YANG-modelled data will have non-null namespace:

2. How do we represent the namespace of an attribute in JSON?

Possible answers, using

  <elem xmlns:x="http://example.com/ns/URI" x:attr="foo">bar</elem>

as an example:

2.1. Use Clark's notation [1]:

  "elem": "bar",
  "@elem": {
    "{http://example.com/ns/URI}attr": "foo"
  }

2.2. Associate somehow every attribute with a YANG module, and then use
     the same notation as for encoding the namespace of YANG data nodes:

  "elem": "bar",
  "@elem": {
    "somemodule:attr": "foo"
  }

2.3. Associate somehow the XML namespace URI with a prefix in JSON and
     then use the same notation as for XML:

  "elem": "bar",
  "@elem": {
    "x:attr": "foo"
  }

In both 2.2 and 2.3, we need to say what "somehow" actually means. If we
accept 1.3, it could I guess involve a certain amount of hand-waving but
leaving it completely unanswered IMO makes the whole thing useless.

I personally think that 2.3 would be a step in a wrong direction because
the URI-prefix duality was recognized as a serious problem of XML
namespaces [2].

Please indicate your preferences or propose something else.

Thanks, Lada

[1] http://tools.ietf.org/html/draft-saintandre-json-namespaces-00
[2] http://blog.jclark.com/2010/01/xml-namespaces.html

Juergen Schoenwaelder <[email-masked]> writes:

> On Thu, Jul 31, 2014 at 10:49:07AM +0200, Ladislav Lhotka wrote:
>> 
>> On 31 Jul 2014, at 07:52, Juergen Schoenwaelder <[email-masked]> wrote:
>> 
>> > On Wed, Jul 30, 2014 at 05:53:30PM +0200, Ladislav Lhotka wrote:
>> >> 
>> >>>>> understand that this makes the solution somewhat incomplete but it
>> >>>>> allows us to finish this document now and to move forward.
>> >>>> 
>> >>>> That is my priority, too.
>> >>> 
>> >>> Good. So what speaks against saying that if attribute names clash and
>> >>> if attributes are scoped by a module name, then the encoding is XYZ?
>> >> 
>> >> I thought YANG 1.1 was a good opportunity to decide about how to do this scoping, provided that standard attributes are really needed.
>> >> 
>> > 
>> > Lets keep the discussion focused on the JSON document. Are you OK with
>> > the proposal to add text that attribute names may be scoped?
>> 
>> Yes, if there is no other choice. It is unclear what “scoped” means.
>> 
>
> Scoped by a namespace, such as 'namespace:identifier'.
>
> /js
>
> -- 
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
> Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 26 Aug 2014 15:19:55 +0200 (CEST)

> 1.3. Yes, as non-normative text.

1.2

>      then use the same notation as for XML:

2.1

And I think the text should specify that the usage of this meta data
must be defined by a specification that uses this mapping.  This means
that RESTCONF must specify which attributes are supported in RESTCONF,
and how they are encoded (i.e. if they have a namespace or not).

/martin

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 26 Aug 2014 15:32:16 +0200

> and how they are encoded (i.e. if they have a namespace or not).

Hmm, do you mean that the set of attributes allowed in (a given version of) RESTCONF will be fixed and no other attributes can be used? In this case we probably needn’t bother with attribute namespaces at all.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 26 Aug 2014 15:36:08 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> Hmm, do you mean that the set of attributes allowed in (a given
> version of) RESTCONF will be fixed and no other attributes can be
> used?

This will be up to the RESTCONF specification - but yes, RESTCONF
probably won't need namespaces for the attributes.

But IMO it is better if the json mapping document is complete and
handle the namespace case as well.  We don't know what other
specifications might want to use.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 26 Aug 2014 15:50:42 +0200

> and how they are encoded (i.e. if they have a namespace or not).

Martin,

does it make sense to have two different namespace notations? For
attributes, you seem to like to use

  {<url>}<attribute-name>

while for regular leafs, the I-D currently says we use

  <modulename>:<leaf-name>

Would it note make sense to consider being consistent, i.e., to
use

  {<url>}<leaf-name>

as well? The counter argument I have heard so far is that module names
are likely shorter than URLs. While true, they are still way longer
than prefixes we use in XML land.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 26 Aug 2014 07:19:49 -0700

Hi,

This issue and been opened and closed again at least 3 times.
Are we spinning on this issue until you get the answer you like?
Which choice is the solution from Martin we already agreed to use?
That is the one I prefer.   The namespace URI is much longer than
the module-name, and not easy to remember like the module name.
But if YANG will not have attributes, then there may be no other choice.

I do not agree with the CLR that only the RESTCONF protocol doc
can define meta-data.  What is the purpose of that rule?  NETCONF
has "with-defaults" in a separate document.  The protocol
needs to be modular so new optional features can be added over time.
Vendors should be able to add meta-data as well.

Andy

On Tue, Aug 26, 2014 at 6:05 AM, Ladislav Lhotka <[email-masked]> wrote:

> >> On 31 Jul 2014, at 07:52, Juergen Schoenwaelder <
> [email-masked]> wrote:
> >>
> >> >> I thought YANG 1.1 was a good opportunity to decide about how to do
> this scoping, provided that standard attributes are really needed.
> >> >>
> >> >
> >> Yes, if there is no other choice. It is unclear what "scoped" means.
> >>
> >

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 26 Aug 2014 16:28:40 +0200

> Which choice is the solution from Martin we already agreed to use?

I am now seriously confused. Can you point me to that Martin’s solution? Did it take into account attribute namespaces?

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 26 Aug 2014 16:38:10 +0200

> than prefixes we use in XML land.

Module names are also more regular in that they are YANG identifiers whereas URIs may contain pretty much any mess. I don’t think your previous objection that (non-IETF) module names aren’t guaranteed to be unique is really relevant: a server implementing two different modules with the same name must be considered seriously broken.

Lada

---

From: Reinaldo Penno <[email-masked]>
Date: Tue, 26 Aug 2014 07:47:49 -0700

I agree that vendors could define metadata as long as there is a 
procedure for that such as a IETF document and a IANA registry.

My reasoning is that I see this metadata is much like decorators in 
Java. In the case of JAVA<->JSON there are multiple libraries that can 
do serialization and they use different decorators, therefore not very 
interoperable. So, we I would suggest metadata/decorators are registered 
somewhere and there is possibility for standard, experimental and 
private notations.

On 8/26/14 7:19 AM, Andy Bierman wrote:
> On Tue, Aug 26, 2014 at 6:05 AM, Ladislav Lhotka <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi,
>
>         "{http://example.com/ns/URI}attr
>     <http://example.com/ns/URI%7Dattr>": "foo"
>       }
>
>     Juergen Schoenwaelder <[email-masked]
>     <mailto:[email-masked]>> writes:
>
>     >> On 31 Jul 2014, at 07:52, Juergen Schoenwaelder
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >>
>     >> >> I thought YANG 1.1 was a good opportunity to decide about
>     how to do this scoping, provided that standard attributes are
>     really needed.
>     >> >>
>     >> >

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 26 Aug 2014 16:56:03 +0200

> specifications might want to use.

I don’t know, perhaps the document can say that those who really can’t do without namespaces are on their own. This is also worth reading:

https://www.mnot.net/blog/2011/10/12/thinking_about_namespaces_in_json

Let me cite one paragraph from it:

"Starting with the obvious, I’d say that if you can use JSON without namespaces, you really, really should. In other words, if you really need distributed extensibility, you need something like namespaces, but for all other purposes, they should be avoided like the plague; they make it too complex, and simplicity is the name of the game in JSON.”

I tend to agree with you that metadata we are talking about are mostly protocol-specific and then distributed extensibility is not needed.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 26 Aug 2014 08:06:07 -0700

> procedure for that such as a IETF document and a IANA registry.
We don't need this for XML. Why do we need it for JSON?

> My reasoning is that I see this metadata is much like decorators in Java.
> In the case of JAVA<->JSON there are multiple libraries that can do
> serialization and they use different decorators, therefore not very
> somewhere and there is possibility for standard, experimental and private
> notations.
>
>
The client parser does not need to understand the semantics of an attribute
in order to be able
to parse it in a reply.   The client won't be sending any requests with
attributes unless
it is programmed to understand specific attributes.

I see JSON attributes being used the same way XML attributes are being used
now
to represent meta-data associated with datastores.  (with-defaults,
enabled, etc.)
If the feature is designed correctly, then it won't break clients that don't
know about the attribute (client has to ask in the request for certain
attributes
to be present in the reply).

I don't see how Java programming practices impact the operational
requirements for datastores.
Vendors are going to use datastore meta-data whether the standard supports
it or not.

Andy

On 8/26/14 7:19 AM, Andy Bierman wrote:
> netmod mailing [email-masked]://www.ietf.org/mailman/listinfo/netmod
>
>
>

---

From: Reinaldo Penno <[email-masked]>
Date: Tue, 26 Aug 2014 08:14:22 -0700

On 8/26/14 8:06 AM, Andy Bierman wrote:
>
>
>
> On Tue, Aug 26, 2014 at 7:47 AM, Reinaldo Penno <[email-masked] 
> <mailto:[email-masked]>> wrote:
>

[RP] Because the metadata work for JSON is being done in the IETF

>
>
>     My reasoning is that I see this metadata is much like decorators
>     in Java. In the case of JAVA<->JSON there are multiple libraries
>     that can do serialization and they use different decorators,
>     therefore not very interoperable. So, we I would suggest
>     metadata/decorators are registered somewhere and there is
>     possibility for standard, experimental and private notations.
>
>
> The client parser does not need to understand the semantics of an 
> attribute in order to be able
> to parse it in a reply.

[RP] That should be true.

> The client won't be sending any requests with attributes unless

[RP] Yes...

>
> I see JSON attributes being used the same way XML attributes are being 
> used now

[RP] Standards based attributes could be present in request or response 
without necessary a previous request, much like certain HTTP headers, or 
X-based HTTP headers. This goes back to my previous point about IANA 
registry.

> Vendors are going to use datastore meta-data whether the standard 
> supports it or not.
>
>
> Andy
>
>
>>         [1]
>>         http://tools.ietf.org/html/draft-saintandre-json-namespaces-00
>>         >> >> I thought YANG 1.1 was a good opportunity to decide
>>         about how to do this scoping, provided that standard
>>         attributes are really needed.
>>         >> >>
>>         >> >
>>     [email-masked]  <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 26 Aug 2014 08:51:30 -0700

On Tue, Aug 26, 2014 at 7:56 AM, Ladislav Lhotka <[email-masked]> wrote:

> I don't know, perhaps the document can say that those who really can't do
> without namespaces are on their own. This is also worth reading:
>
> "Starting with the obvious, I'd say that if you can use JSON without
> namespaces, you really, really should. In other words, if you really need
> distributed extensibility, you need something like namespaces, but for all
> other purposes, they should be avoided like the plague; they make it too
> complex, and simplicity is the name of the game in JSON."
>
>
Too bad we can't just force the use-cases to change to match our solutions.
We want "distributed naming authority" instead of "centralized naming
authority".
This allows multiple vendors and SDOs to create modules independently with
near-zero
possibility of naming collisions.

I agree we pay a lot for this feature, but it is important.

I tend to agree with you that metadata we are talking about are mostly
> protocol-specific and then distributed extensibility is not needed.
>
>
???
How do I tell the difference between the standard "with-defaults" attribute
and some vendor attribute with the same name?

I am more concerned about URI query parameter extensions than attributes.
You should expect to see lots more of those than custom attributes.

Lada

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 26 Aug 2014 18:34:27 +0200

> I tend to agree with you that metadata we are talking about are mostly protocol-specific and then distributed extensibility is not needed.

Unfortunately, name collisions will happen. So we need to deal with
namespaces and (speaking as technical contributor) I prefer a solution
that is solving this problem in a consistent manner.

Even if your assumption is correct that metadata is mostly
protocol-specific, this does not guarantee that meta-data attribute
names will be collision free.

If we go with YANG module names as namespaces in JSON, another option
is to say that the general format is [<namespace>:]<name>. For YANG
defined names, the namespace is the module name while for names not
defined in YANG modules (like attribute names), it is unspecified how
namespaces with low collision probability will be derived. We could
provide guidelines about possible ways to choose namespaces with low
collision probability.

My understanding is that the JSON I-D only requires to use explicit
namespaces if there is actually a name collision. So ideally,
namespace qualification would be rather an exception than the
default. Perhaps advice should be given that vendor extensions
for meta data should pick names that likely won't collide with
a future standard usage.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 26 Aug 2014 18:39:17 +0200

On 26 Aug 2014, at 17:51, Andy Bierman <[email-masked]> wrote:

> We want "distributed naming authority" instead of "centralized naming authority".
> This allows multiple vendors and SDOs to create modules independently with near-zero

This is certainly true for YANG data nodes, I am not so sure about attributes.

> and some vendor attribute with the same name?

How do you tell the difference between the standard HTTP “GET” method and some vendor method with the same name? If it is part of the protocol, the vendor simply cannot use the name.

> You should expect to see lots more of those than custom attributes.

Yes, but this has nothing to do with yang-json, right?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 26 Aug 2014 09:43:54 -0700

On Tue, Aug 26, 2014 at 9:34 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > I don't know, perhaps the document can say that those who really can't
> do without namespaces are on their own. This is also worth reading:
> >
I like this proposal.  We give the same guidance for capability URIs
without specifying an exact format.

> for meta data should pick names that likely won't collide with
I think Lada's draft says SHOULD use where namespace changes (including top)
even if there are no collisions right now.  As Phil pointed out, a new
module can
be loaded later, which causes a new naming collision. (Then old client send
/foo
and it is suddenly an "ambiguous node error".)

/js
Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 26 Aug 2014 22:28:21 +0200 (CEST)

Andy Bierman <[email-masked]> wrote:
> I like this proposal.

+1

RESTCONF still needs to specify how all standard attributes are
encoded (probably w/o namespace).

> We give the same guidance for capability URIs

I have suggested before that it should be MUST use the explicit
namespace whenever the namespace changes.  This gives determistic and
stable names, even if modules are revised or added.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 27 Aug 2014 09:01:03 +0200

> stable names, even if modules are revised or added.

What do you mean with "whenever the namespace changes"? The I-D
currently says:

   The namespace identifier MUST be used for local names that are
   ambiguous, i.e., whenever the data model permits a sibling data node
   with the same local name.  Otherwise, the namespace identifier is
   OPTIONAL.

I think the phrase 'the data model permits a sibling data node with
the same local name' was supposed to mean 'a server supports a sibling
data node with the same local name' (and if I am correct I think this
phrase should be changed since in general every YANG defined data node
'permits' a sibling data node with the same local name).

So what do you mean by 'whenever the namespace changes'? I do not think
the YANG to JSON I-D has a notion of a current namespace and namespace
changes.

/js

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 Aug 2014 00:45:06 -0700

> the YANG to JSON I-D has a notion of a current namespace and namespace

Nested nodes which result from augment-stmt from an external module.
Otherwise the node is the same namespace as its parent.  If there
is no module-name, then the parent module-name applies.

Perhaps the draft text is wrong -- there is no ambiguity
determining the module name for a nested node without a prefix.
It is simply the module name of its parent.  Only the sibling from
the external module needs a module-name to clarify its namespace.

 Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 27 Aug 2014 09:51:38 +0200 (CEST)

> the YANG to JSON I-D has a notion of a current namespace and namespace

Consider this situation:

  module a {
    ...
    container x {
      container y  { ... }
    }
  }
  
  module b {
    ...
    augment "/a:x/a:y" {
      container z { ...}
    }
  }

If this is all there is, the current draft allows:

   "x": {
      "y": {
         "z": {
            ...
       }
   }

I.e., "z" can be encoded w/o the prefix since there is no conflict.

I propose that since the module of "z" is different from the module of
"y" (a namespace change), the prefix is MANDATORY:

   "a:x": {
      "y": {
         "b:z": { ... }
       }
   }

This also means that if module a has:

  module a {
    ...
    container x {
      container y  {
        container z { ... }
      }
    }
  }

then we could have:

   "a:x": {
      "y": {
         "z": { ... }
         "b:z": { ... }
       }
   }

I.e., the "z" from module "a" is encoded w/o prefix even though "b" is
there.  This means that the data looks the same regardless of whether
module "b" is implemented or not.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 Aug 2014 01:15:46 -0700

> I.e., "z" can be encoded w/o the prefix since there is no conflict.

The text about needing a module-name because there is a
sibling with the same local-name is wrong.

The node "z" is not qualified above.  There is no child "z"
in parent "y".  The module name "b" is ALWAYS needed
because it is not from the module "a".

There does not need to be any child nodes defined for "y"
in module "a", let alone a duplicate.  The module name
is needed to correctly identify namespace "b".

> module "b" is implemented or not.
It is really simple -- if a node has no prefix, then it is in the same
namespace as its parent. Siblings have nothing to do with it.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 27 Aug 2014 10:24:48 +0200 (CEST)

Andy,

It seems we agree, but this it not how the current draft is specified!

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 10:29:05 +0200

> namespaces if there is actually a name collision. So ideally,

For YANG data nodes, we can find out whether there is a collision or not
from the data model. To be able do it for metadata, we need to know
whether/how the server will advertise support for particular metadata.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 10:40:09 +0200

Andy Bierman <[email-masked]> writes:

> I think Lada's draft says SHOULD use where namespace changes
> (including top) even if there are no collisions right now.  As Phil

This is possible.

> pointed out, a new module can be loaded later, which causes a new
> naming collision. (Then old client send /foo and it is suddenly an
> "ambiguous node error".)

Well, loading a new module means a different contract, so the client
should be notified.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 27 Aug 2014 01:46:18 -0700

> It seems we agree, but this it not how the current draft is specified!
Consider:

  module c {
    ...
    augment "/a:x/a:y" {
      leaf z { ...}
    }
  }

It doesn't matter that module "a" does not define any child nodes for "y".
The node "z" always needs a prefix so we know if it is from module "b" or
"c".

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 27 Aug 2014 10:52:17 +0200 (CEST)

> The node "z" always needs a prefix so we know if it is from module "b" or

I agree.

  If the node is not in the same module as its parent (i.e., an it is
  an augmented node), then it MUST be encoded with the module name as
  prefix.

I would even say:

  If the node is in the same module as its parent, then it MUST NOT be
  encoded with the module name as prefix.

The reason for these rules is simplicitly.  Each node is *always*
encoded the same way, regardless of which modules are loaded in a
particular server.  Simple, unambigous, and easy to implement.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 11:24:42 +0200

Hi,

Andy once proposed to define metadata as YANG identities. I don’t think it is a good idea to overload identities for this purpose but, well, maybe defining metadata with a top-level statement in YANG modules is the best way for solving the metadata issues after all. There are actually two options:

1. Reconsider YANG 1.1 issue Y33 - the new statement could be called e.g. “annotation” to avoid any XML connotations.

2. Define this statement as an extension.

Either way, benefits of doing so are:

- we immediately get the module name, namespace URI and prefix, so the encoding can be the same as for YANG data (both in XML and JSON);
- YANG module revision mechanism could be also handy;
- the semantics can be explained in a description;
- a data type can be specified.

For example:

annotation inactive {
  type boolean;
  description
    "If this annotation is attached to a configuration
     data node, and its value is 'true', then the server
     MUST behave as if the node was not present."; 
}

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 27 Aug 2014 11:39:04 +0200

Hi,

when Y33 was discussed, there was generally an agreement that we will
only have a small set of metadata 'attributes' and there was a concern
that making 'attributes' a first class citizen of YANG 1.1 may
actually encourage misuse.

That said, if a formal way to define metadata 'attributes' can be
provided using a YANG extension, someone should write a concrete
proposal up as an I-D and then this may be considered independent of
the YANG 1.1 effort. It is not clear to me why this presumably rarely
used mechanism needs to be part of YANG 1.1.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 11:41:56 +0200

>  encoded with the module name as prefix.

OK, if it is the consensus, I am fine with it. In our previous discussion, Andy wanted to retain the possibility for using the qualified name anywhere, even if it’s not necessary.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 27 Aug 2014 11:44:24 +0200 (CEST)

> actually encourage misuse.

Agreed.

> used mechanism needs to be part of YANG 1.1.

This is certainly possible, but I think it has the same drawbacks as
if it was a built-in statement.  I.e., if there is an IETF-defined
extension, it could encourage misuse.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 11:51:44 +0200

> actually encourage misuse.

This has never been proposed as a first-class citizen. Since it would be allowed only at the top-level there would be no way for specifying the place in the data model where it can appear.

If we only had a small set of metadata, then we should avoid namespaces as the plague, but I had the impression that you consider support for distributed extensibility a necessity.

> used mechanism needs to be part of YANG 1.1.

I can write such a draft.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 12:00:28 +0200

> extension, it could encourage misuse.

Why is the danger of misuse higher than in the case when vendors are allowed to throw in metadata without further ado?

If the set of permitted attributes is small and relatively fixed, then let’s forget about namespaces and just use what we already have. On the other hand, distributed extensibility of metadata warrants a new YANG statement.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 27 Aug 2014 12:06:15 +0200

> This has never been proposed as a first-class citizen. Since it would be allowed only at the top-level there would be no way for specifying the place in the data model where it can appear.

Your example shows the definition of a metadata 'attribute' that as
far as I understand can appear on any data node. So apparently there
is a difference where something is defined and where it is applied.

> If we only had a small set of metadata, then we should avoid namespaces as the plague, but I had the impression that you consider support for distributed extensibility a necessity.

I have heard vendors use their own metadata 'attributes' already today
and since I doubt they will coordinate their metadata 'attribute'
names, we have to be prepared to handle collisions. My goal here is
robustness on the protocol encoding level. For me, this is separate
from how metadata 'attributes' are defined. The data encoding IMHO
must be robust and interoperable.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 13:07:22 +0200

> is a difference where something is defined and where it is applied.

No, that’s why I wrote “top-level statement”. It would be a new alternative in the “body-stmts” production and nowhere else.

> names, we have to be prepared to handle collisions. My goal here is

But then the set of metadata attributes needn’t be necessarily small, I’d expect rather the opposite.

> must be robust and interoperable.

I think that defining them in YANG modules would make it as robust and interoperable as possible.

An advantage of having it as a regular statement and not an extension is that these attributes could then also be used in XPath expressions, if the module defining the attribute is imported.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 27 Aug 2014 13:24:55 +0200

> No, that’s why I wrote “top-level statement”. It would be a new alternative in the “body-stmts” production and nowhere else.

We talk past each other. I meant: The metadata 'attribute' defined in
the toplevel statement can be attached to every data node.

> But then the set of metadata attributes needn’t be necessarily small, I’d expect rather the opposite.

Depends on how we define small. The point is that it might be small
for each vendor but these small sets might have a high collision
probability.

> An advantage of having it as a regular statement and not an extension is that these attributes could then also be used in XPath expressions, if the module defining the attribute is imported.

Personally, I hope we can avoid that. It seems you really want to make
metadata attributes first class citizens.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 27 Aug 2014 14:13:11 +0200

> the toplevel statement can be attached to every data node.

This confusion is due to YANG terminology:

   o  data node: A node in the schema tree that can be instantiated in a
      data tree.  One of container, leaf, leaf-list, list, and anyxml.

So what you mean is “an attribute can be attached to any instance of a data node”. Then yes, but this will be true for these metadata attributes in any case, right?

> for each vendor but these small sets might have a high collision

It won't be a problem unless one device uses attributes from different vendors with colliding names. Using Randy’s earlier example, a client can use “gift” attribute unambiguously with an English and German server at the same time because the context is specific for each server.  

> metadata attributes first class citizens.

I don’t know, you should tell me what you mean by “first class citizen”. It is certainly, and intentionally, not the same as, e.g., the “attribute” pattern in RELAX NG. I am just trying to solve the problems we have in an effective way. Metadata is not my idea.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 27 Aug 2014 14:42:33 +0200

> It won't be a problem unless one device uses attributes from different vendors with colliding names. Using Randy’s earlier example, a client can use “gift” attribute unambiguously with an English a…

Vendors buy other vendors. Modular netconf servers may allow plugins
from third parties, one written by a German programmer and one written
by an English coder. My expectations that vendors will always sort
these kind of things out when they merge are probably lower than
yours. I would not be surprised if even different product groups
within a big vendor manage to produce collisions.
 
> I don’t know, you should tell me what you mean by “first class
> citizen”. It is certainly, and intentionally, not the same as, e.g.,
> the “attribute” pattern in RELAX NG. I am just trying to solve the
> problems we have in an effective way. Metadata is not my idea.

As far as I can tell, making metadata 'attributes' accessible in
must/when expressions was not proposed in any of the YANG 1.1
issues. The closest might have been solution Y48-01 but then Y48 got
rejected. I prefer to keep YANG 1.1 scoped.

/js

---

From: Reinaldo Penno <[email-masked]>
Date: Wed, 27 Aug 2014 09:41:21 -0700

Hi Lada,

I would be willing to contribute to this effort.  We are looking into 
using such metadata Yang statements in Opendaylight.

thanks,

Reinaldo

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 28 Aug 2014 13:51:31 +0200

Hi Reinaldo,

> using such metadata Yang statements in Opendaylight.

Great! I made a GitHub project for this work:

https://github.com/yang-metadata/yang-metadata

It already contains the module with extension definition.

Please send me your GitHub ID, I will give you push access.

Thanks, Lada