Subject: [netmod] yang json and anyxml

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 28 Aug 2014 09:48:29 +0200

Hi,

would it not make sense to encode anyxml data as a JSON string
containing the XML content? I believe this would be more interoperable
than saying the "translation of anyxml contents is necessarily
application-specific and outside the scope of this document".

The anyxml type says that the data is an arbitrary chunk of XML and
hence we should just transport it without inviting implementations to
try any fancy JSON encodings. [And if we introduce anydata in YANG
1.1, hopefully the need to use anyxml goes away and anydata content
may be YANG encoded with the caveat that everything will be treated
as a string due to a lack of a data model.]

/js

PS: An extreme position would be to encode everything in general as a
    JSON string since obviously the JSON type system and the YANG type
    system do not line up very well - I guess Lada will not like this
    at all.

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 28 Aug 2014 12:48:39 +0200 (CEST)

> application-specific and outside the scope of this document".

Here is one problem with this approach (the root cause of the problem
is how anyxml is defined (and the reason for this goes back to how
netconf is defined...)):

  module x {
    namespace "urn:x";
    prefix x;
  
    container x {
      anyxml y;
    }
  }

Now, this is a valid XML instance:

  <x xmlns:x="urn:x"
     xmlns:foo="urn:foo">
    <y>
      <a>
        <foo:b>4668</foo:b>
      </a>
    </y>
  </x>

If we translate this to json with a simple string, it would look like
this (modulo whitespaces):

  "x": {
     "y": "<a> <foo:b>4668</foo:b> </a>"
  }

Note how the xml data in "y" cannot be properly interpreted, b/c the
xmlns declaration for "foo" is missing.

I guess we *could* solve this by stating that any xmlns declarations
that are used within the anyxml data must be inserted by the encoder
at the proper place, giving:

  "x": {
     "y": "<a xmlns:foo='urn:foo'> <foo:b>4668</foo:b> </a>"
  }

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 Aug 2014 04:39:05 -0700

Hi,

I do not agree that anyxml needs to be encoded as a string, or treated
special somehow.  NETCONF does not allow mixed mode XML.
Our tools fully support anyxml (for XML valid for NETCONF).
This easily maps to JSON without any problems. Collections, arrays
and leafs can be encoded and decoded just fine.

I doubt the original arbitrary YANG data tree can be reproduced by
the receiver from a string, vs. simple JSON structures.

I do not view the message encoding as a first order problem.
It should not get in the way of the protocol. We should find
an encoding that suites our protocol, not the other way around.

Andy

On Thu, Aug 28, 2014 at 12:48 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 28 Aug 2014 13:58:01 +0200

>     "y": "<a xmlns:foo='urn:foo'> <foo:b>4668</foo:b> </a>"

Right, this is one reason. Another is that some applications might want to do something more clever, see the example in sec. 3.2.5.

In any case, applications are free do what you propose, if it makes sense, and this should be stated in the description of the anyxml statement so interoperability is possible.

Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 28 Aug 2014 14:03:40 +0200

> special somehow.  NETCONF does not allow mixed mode XML.

RFC 6020 section 7.10:

   The "anyxml" statement is used to represent an unknown chunk of XML.
   No restrictions are placed on the XML.

So are there restrictions or not?

> and leafs can be encoded and decoded just fine.

How do you tell whether <a>123</a> is to be encoded as a string or a
number in JSON?

If there is a way to encode anyxml content in JSON, then I would say
the YANG to JSON I-D needs to define that and not say this is
implementation dependent.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 28 Aug 2014 14:05:36 +0200

> In any case, applications are free do what you propose, if it makes sense, and this should be stated in the description of the anyxml statement so interoperability is possible.

I want robust and interoperable encoding. I do not care about
cleverness that only works for some applications.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 28 Aug 2014 14:06:20 +0200

>      "y": "<a xmlns:foo='urn:foo'> <foo:b>4668</foo:b> </a>"

Yes, this might be the price to pay.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 Aug 2014 05:13:51 -0700

> So are there restrictions or not?

words on a page vs. running code.
No customer has complained that the anyxml we support is too restrictive.

> How do you tell whether <a>123</a> is to be encoded as a string or a

we encode as a string.  Since there is no data type, what does it matter?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 Aug 2014 05:36:12 -0700

> So are there restrictions or not?

RFC 6020 defines a language that is only intended for use in NETCONF.
Since NETCONF does not allow arbitrary XML, YANG anyxml is
poorly defined.  There are restrictions if the YANG is used in the
NETCONF protocol.  Since that is its only official purpose,
there is no reason for YANG to be defined incorrectly.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 28 Aug 2014 14:45:05 +0200

> No customer has complained that the anyxml we support is too restrictive.

So are you saying the discussion we had in Toronto about anyxml and
anydata is pointless since in reality anyxml really is anydata? Can
other implementations confirm this? If this is generally true, we
should perhaps revisit the discussion and clarify in RFC 6020 bis
that anyxml really means a rather restricted set of XML.

Question: How do you deal with namespaces in the XML when you render
it in JSON? The JSON I-D requires that you know the module name while
all you have is a URI...

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 28 Aug 2014 14:53:19 +0200

> cleverness that only works for some applications.

One size fits all isn’t the same as robust and interoperable. Every anyxml is defined in a YANG module so why couldn’t each have a specific purpose, format, semantics and handling? Why is this non-interoperable or non-robust? IMO moving around pieces of invalid XML is worse.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 Aug 2014 06:25:24 -0700

> it in JSON? The JSON I-D requires that you know the module name while
I have not implemented JSON attributes yet.  In XML, the code needs to
preserve attributes (including xmlns).

I should stress that we rarely use anyxml as supposedly intended.
It is usually tagged with a YANG extension so it is treated as a document
root (<config>, <data>, etc.), and even then, these "root" containers are
only used within RPC input or output.  We support anyxml as config or state
data,
but recommend that real YANG be used instead.

I would prefer to have a special container in YANG 1.1 (called "docroot"?)
instead of anyxml or anydata.

/js
Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 28 Aug 2014 17:14:27 +0200

> preserve attributes (including xmlns).

My question was not about attributes...

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 28 Aug 2014 17:22:46 +0200

> One size fits all isn’t the same as robust and interoperable. Every anyxml is defined in a YANG module so why couldn’t each have a specific purpose, format, semantics and handling? Why is this non-…

I am not sure "every anyxml is defined in a YANG module" is true. I
heard that Kent uses configlets that are anyxml and the configlets may
actually be defined in XSD.

Quoting draft-ietf-netmod-yang-json-00.txt:

   An XML element that is modeled as a YANG anyxml data node is
   translated to a name/object pair.  The content of such an element is
   not modelled by YANG, and there may not be a straightforward mapping
   to JSON text (e.g., if it is a mixed XML content).  Therefore,
   translation of anyxml contents is necessarily application-specific
   and outside the scope of this document.

I do not think "translation of anyxml contents is necessarily
application-specific" is interoperable. (I note that the second quoted
sentence contradicts your statement that "every anyxml is defined in a
YANG module").

And the example given in draft-ietf-netmod-yang-json-00.txt is
actually supporting my concern. Why should something translate

       <bar>
         <p xmlns="http://www.w3.org/1999/xhtml">
           This is <em>very</em> cool.
         </p>
       </bar>

into an arbitrary markup notation

       {
         "bar": {
           "p": "This is *very* cool."
         }
       }

if bar is of type anyxml? I would strongly prefer to get the
unmodified xhtml content for bar and not some arbitrary rendering of
it.

/js

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 Aug 2014 08:24:43 -0700

> My question was not about attributes...

OK -- I would implement your proposed solution (use the XML namespace as
the module name).
 That is not supported yet.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 28 Aug 2014 08:32:32 -0700

On Thu, Aug 28, 2014 at 8:22 AM, Juergen Schoenwaelder <
[email-masked]> wrote:

> > One size fits all isn't the same as robust and interoperable. Every
> anyxml is defined in a YANG module so why couldn't each have a specific
> purpose, format, semantics and handling? Why is this non-interoperable or
> non-robust? IMO moving around pieces of invalid XML is worse.
> >
>
But this would not be allowed in a NETCONF message.
Why would we add this complexity to RESTCONF (the lightweight version)?
How important is it for RESTCONF to manage arbitrary chunks of HTML?
Nobody has asked for the NETCONF XML restrictions to be removed.

> unmodified xhtml content for bar and not some arbitrary rendering of
Agreed -- if mixed mode XML and entities were allowed, then encoding
the XML as a string would be best.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 28 Aug 2014 18:59:28 +0200 (CEST)

Juergen Schoenwaelder <[email-masked]> wrote:
> > One size fits all isn’t the same as robust and interoperable. Every anyxml is
> > defined in a YANG module so why couldn’t each have a specific purpose,
> > format, semantics and handling? Why is this non-interoperable or non-robust?
> > IMO moving around pieces of invalid XML is worse.
> > 
> 

The main reason for anyxml was to be able to define the NETCONF
operations in YANG, and NETCONF can (theoretically) be used with any
xml.

The question is if this still is important.  Currently, the only
interoperable way of using NETCONF is with YANG.

I think the same applies to anyone that implements the zerotouch stuff,
including the YANG modules it defines; the data for the initial config
in the configlets will be modelled with YANG.

Thus, if we add anydata in YANG 1.1, and explain that anyxml really
shouldn't be used, I think Lada's current solution for anyxml is good
enough.

/martin

---

From: Radek Krejčí <[email-masked]>
Date: Fri, 29 Aug 2014 10:07:13 +0200

Hi,

> that anyxml really means a rather restricted set of XML.

libnetconf put no restrictions on anyxml content. On the other hand, we
don't use anyxml and I don't have any feedback regarding anyxml from the
mailing list, so it wasn'theavily tested.

Radek

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 29 Aug 2014 10:31:37 +0200

> actually be defined in XSD.

I know the term “anyxml” only from YANG context. Therefore, if it’s not defined in a YANG module, it is not anyxml.

> YANG module”).

No, the outermost XML element is defined in YANG and what’s inside (“content of such an element”) is not.

> if bar is of type anyxml? I would strongly prefer to get the

Why not? The definition of this anyxml data note could be e. g.

anyxml bar {
  description
    "Text of the banner page that is displayed to every user
     after login. XHTML markup can be used in XML and Markdown
     in JSON.";
}

Why should implementors have any trouble with this? I think it could be actually quite useful.

And yes, I agree with you that RFC 6020 allows anyxml node instances to contain really any well-formed XML, including mixed content.

> unmodified xhtml content for bar and not some arbitrary rendering of

Sorry, I still don’t see the reason. Could you give an example where use-case-specific rendering can cause any problems?

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 29 Aug 2014 11:31:01 +0200

> the XML as a string would be best.

OK, so would a string be better than an object? Something like this:

An XML element that is an instance of a YANG anyxml data node is translated to a name/value pair where the value is a JSON string. The XML-JSON mapping of this string is not defined by this document. That is, the content of the XML instance may appear unchanged in this string value, or it may be processed in any way that’s appropriate for a given application. In the latter case, the definition of the anyxml data node SHOULD contain a description specifying the mapping of the XML content to JSON and vice versa. 

Lada