Subject: [netmod] development plan

From: Andy Bierman <[email-masked]>
Date: Fri, 2 Aug 2013 23:41:32 -0700

Hi,

I wold like the NETMOD and NETCONF WGs to establish a
development plan and roadmap for the work that will be attempted
over the next 1 to 3 years.

One draft after another gets presented.
Some, like the ACL draft or <get2> draft have had almost unanimous
support as a problem that needs to be solved. Then quickly forgotten.

The complete lack of any development plan discourages people
from participating and probably encourages others who are just
wasting their time submitting drafts to these WGs.

I don't agree with the assertion that NETCONF/YANG will only
be useful when it "manages everything" (like the proprietary CLI).
It is useful now, but in order to be essential, the community needs
to know that it is stable and understand when and how new functionality will
be added.

Andy

---

From: "Ersue, Mehmet (NSN - DE/Munich)" <[email-masked]>
Date: Sat, 3 Aug 2013 08:58:23 +0000

We need to start somewhere to discuss the priorities. So, let's start here.

What would be your proposal for the development plan? 

Cheers, 
Mehmet 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of ext
> Andy Bierman
> Sent: Saturday, August 03, 2013 8:42 AM
> To: [email-masked]; Netconf
> Subject: [Netconf] development plan
> 
> Hi,
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Sat, 3 Aug 2013 02:12:54 -0700

> What would be your proposal for the development plan?

This will take some time to work out, so I will just start with some
design principles
and fill in the details later.

1) current functionality that the community agrees is useful MUST NOT be broken
    in any way

2) current functionality that the community agrees is not useful
SHOULD be removed

3) YANG needs to be a multi-protocol DML, but that does not mean written for
    unknown protocols.  The protocol-independent and protocol-dependent aspects
    need to be clearly separated, allowing specific protocol mapping RFCs to be
    easily standardized

4) Important module infrastructure such as ACLs and topology need to
be standardized
    in an extensible manner, allowing future (currently unknown)
applications to build
    on these modules and not start over

5) The private candidate datastore is a disaster for multi-client
deployments.  A real
    transaction mechanism that does not rely on global or even
predictive locking is needed.
    Concurrent transactions that do not collide can be processed in parallel.

6) Protocol performance and scaling properties need to be addressed

7) Network-wide applications need to be supported in a way that does not break
    existing single-server use cases

Andy

>> From: [email-masked] [mailto:[email-masked]]  Behalf Of ext
>> Andy Bierman

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 05 Aug 2013 11:56:14 +0200

>     need to be clearly separated, allowing specific protocol mapping RFCs to be

IMO, YANG 2.0 should be protocol-independent, and a light-weight adaptation layer should be defined for each protocol, including NETCONF.

Apart form this, there are several other areas that should be considered. One that comes to my mind are XPath extension functions.

An interesting question is whether YANG 2.0 could/should also be made independent of the output encoding so that e.g. JSON is directly supported.

>     on these modules and not start over

The priority for the NETMOD group should be to work on vendor-neutral data models that can be used right away or mapped to vendor-specific data models.

>     Concurrent transactions that do not collide can be processed in parallel.

+1, and I'd also add a sound model of configuration, state data and their interaction.

Lada

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Aug 2013 05:23:56 -0700

> Apart form this, there are several other areas that should be considered. One that comes to my mind are XPath extension functions.

I don't think it is a good time to start a 6020bis RFC and
tell the community YANG 1.0 is unstable and being replaced.

> An interesting question is whether YANG 2.0 could/should also be made independent of the output encoding so that e.g. JSON is directly supported.

I don't want YANG 2.0.
New standard YANG extensions are OK for now.

> +1, and I'd also add a sound model of configuration, state data and their interaction.

I think Kent's conditional enablement draft is the foundation for all the
corner-case config state and operational CM requirements.
We currently have a very simplistic model for the device configuration.
The running config is either there or it is deleted and forgotten.
IMO this is #1 or 2 priority for the NETCONF WG.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Aug 2013 05:35:30 -0700

> What would be your proposal for the development plan?

Here is my development plan priorities:

NETCONF WG:

   #1) RESTCONF (was YANG-API)
   #2) Config state, operational state, conditional enablement
         (IMO, all these are related problems)
   #3) Private Candidates (concurrent transactions)

NETMOD WG:

   #1) Finish current base modules (either publish or abandon)
   #2) Transition to IETF support mode -- help other WGs
         create YANG modules for NETCONF and RESTCONF.
         Start with ACLs and topology. Help other WGs like SACM
         and I2RS use YANG.  Coordinate all new work
         through YANG Doctors.
    ** Wait 2 years or so before attempting YANG 2.0

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Aug 2013 14:50:41 +0200

> tell the community YANG 1.0 is unstable and being replaced.

I had the impression that my proposal to decouple YANG from NETCONF was generally supported, and this is impossible to do without changing 6020, because some important things like XPath context, 'choice' and 'when' semantics, or even the concept of validity itself, are defined in NETCONF-specific terms.

> New standard YANG extensions are OK for now.

I agree with Martin that extensions should be restricted in scope, so one can do only as much with them.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Aug 2013 06:31:12 -0700

> I had the impression that my proposal to decouple YANG from NETCONF was generally supported, and this is impossible to do without changing 6020, because some important things like XPath context, 'c…

I agree with Dave Harrington's concerns expressed at the NETMOD WG meeting.
Introducing a new language version is a serious and drastic step that should
be done very carefully.  IMO we do not have enough operational experience
with YANG to do a good job at YANG 2.0.

> I agree with Martin that extensions should be restricted in scope, so one can do only as much with them.

Yes -- we should specify what best practices are YANG extensions.
An update to the the YANG Usage Guidelines would probably be
a good idea.  Consensus on best practice has changed and the
guidelines are no longer 100% current.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Aug 2013 16:03:55 +0200

> with YANG to do a good job at YANG 2.0.

It is a fact of life that YANG is being used outside NETCONF, and there are more plans for doing do, but with 6020 it is a slippery slope. For example, such non-NETCONF applications of YANG will most likely want to use "must" for expressing semantic constraints. However, the accessible tree for "must" is defined like so:

   o  If the context node represents configuration, the tree is the data
      in the NETCONF datastore where the context node exists.  The XPath
      root node has all top-level configuration data nodes in all
      modules as children.

   o  If the context node represents state data, the tree is all state
      data on the device, and the <running/> datastore.  The XPath root
      node has all top-level data nodes in all modules as children.

   ...

This cannot be unambiguously interpreted in a non-NETCONF context, so it is effectively undefined, which is rather dangerous.

So I don't mean to completely redesign YANG, but we cannot, on the other hand, enable non-NETCONF YANG data modelling without providing new definitions.

Lada

---

From: Kent Watsen <[email-masked]>
Date: Mon, 5 Aug 2013 14:10:23 +0000

My short list:

NETCONF WG:  

  (essentially Andy's list with a couple items I'm already into on top)

  #1) Reverse SSH
  #2) Zero Conf
  #3) RESTCONF
  #4) Config state, operational state, conditional enablement
  #5) Private Candidates (concurrent transactions)

NETMOD WG:

  #1) Extend as needed to minimize number of proprietary Juniper YANG
modules

  #2) Modules for basic EMS support (hardware, software, licenses, etc.)
  #3) Data Model Transformation Engine

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Aug 2013 07:22:40 -0700

Hi,

I omitted NETCONF stuff that is currently chartered.
I agree Reverse SSH and TLS-bis are #1 for NETCONF.
Zeroconf came out of left field.  What does
that mean in the context of these 2 WGs?

Andy

---

From: Kent Watsen <[email-masked]>
Date: Mon, 5 Aug 2013 15:28:54 +0000

>From the WG summary Mehmet sent out:

"...and Zero-configuration.  The latter has been seen as essentially
important and should be done in a separate draft. Steve Hanna volunteered
to work on this together with Kent.  Manual configuration will be done in
the Reverse SSH draft."

My understanding was that there was interest in doing ZeroConf (for both
SSH and TLS), especially now that we have the Security experts attention.
I think the chairs were going to check if the current charter covers it or
not...

K.

---

From: Balazs Lengyel <[email-masked]>
Date: Mon, 5 Aug 2013 17:49:53 +0200

> So I don't mean to completely redesign YANG, but we cannot, on the other hand, enable non-NETCONF YANG data modelling without providing new definitions.
In my view Netconf contains the "pure" protocol (encoding, operations, 
transactional behavior...) and background plumbing (datastores, 
separation of state and config data, hierarchical data organisation, 
filtering, connection requirements, session concept, capability 
negotiation, many of the capabilites, error messages).
The background plumbing is valid for all access methods (CLI, Netconf, 
YANG-Api, I2RS, GUI, etc.) while the "pure" protocol needs to be redefined.
regards Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
System Manager
ECN: 831 7320                        Tel: +36-1-437-7320
Mobile: +36-70-330-7909              email: [email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Mon, 5 Aug 2013 09:53:53 -0700

...
>    #1) RESTCONF (was YANG-API)

This item also includes a complete protocol encoding
for JSON (in addition to XML).  Lada's JSON draft
is required for this item.

http://datatracker.ietf.org/doc/draft-lhotka-netmod-yang-json/

Andy

---

From: "Lisa Huang (yihuan)" <[email-masked]>
Date: Mon, 5 Aug 2013 17:20:39 +0000

Hi,

I echo what Andy said below. When I invited people from other WGs, like
BGP etc, to join netmod session, they were disappointed when they found
that the WG has only a limited models and were still discussing basic
models. Encouraging more model submission is very important and should be
part of development plan.

Thanks,

Lisa

On 8/2/13 11:41 PM, "Andy Bierman" <[email-masked]> wrote:

>netmod mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/netmod

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Aug 2013 19:45:54 +0200

On Aug 5, 2013, at 5:49 PM, Balazs Lengyel <[email-masked]> wrote:

> In my view Netconf contains the "pure" protocol (encoding, operations, transactional behavior...) and background plumbing (datastores, separation of state and config data, hierarchical data organisation, filtering, connection requirements, session concept, capability negotiation, many of the capabilites, error messages).
> The background plumbing is valid for all access methods (CLI, Netconf, YANG-Api, I2RS, GUI, etc.) while the "pure" protocol needs to be redefined.

I don't think this is true. I2RS will probably have no distinction between config and state data, though they may have read-write and read-only. Sessions and capability negotiation are by no means necessary (RESTCONF doesn't have it). I can actually imagine data modelling without any specific protocol for data exchange.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Aug 2013 19:50:33 +0200

> part of development plan.

Reviews help to finish up things faster.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Aug 2013 19:52:45 +0200

> IMO, YANG 2.0 should be protocol-independent, and a light-weight adaptation layer should be defined for each protocol, including NETCONF.

Someone needs to define 'protocol-independent'. If you want to make
YANG fully protocol-independent, then you will likely not finish and
create just another example of the 2nd systems effect [1].

/js

[1] http://en.wikipedia.org/wiki/Second-system_effect

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Aug 2013 19:53:40 +0200

>          (IMO, all these are related problems)

What is to be done about config state and operational state?

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Aug 2013 19:55:04 +0200

>   #1) Extend as needed to minimize number of proprietary Juniper YANG

Any Juniper submissions I can look at to see what this means?
 
>   #3) Data Model Transformation Engine

What is a 'data model transformation engine' doing?

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Aug 2013 20:16:26 +0200

> create just another example of the 2nd systems effect [1].

XML schema languages are protocol-independent and IMO they are good examples of what YANG specification should aim at. In fact, DSDL mapping of YANG (RFC 6110) can help in this respect because the resulting DSDL schemas can be used, by extension, in any non-NETCONF context already now.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Aug 2013 21:25:52 +0200

> XML schema languages are protocol-independent and IMO they are good examples of what YANG specification should aim at. In fact, DSDL mapping of YANG (RFC 6110) can help in this respect because the…

YANG was designed to support NETCONF and as such it has a number of
features that are related to NETCONF - and NETCONF is not just an XML
editing tool. Do you propose to remove them? If so, do you propose
these 'elements' become YANG 2.0 extensions?

If the DSDL mapping of YANG already does the job and others are fine
to use YANG as it is today, perhaps there is no problem to be solved.

Note: I am just trying to find out what people mean when they use
certain terms in this thread since we might easily speak past each
other (or worse agree on something just to find out later that we have
N different interpretations of 'something').

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Aug 2013 22:04:38 +0200 (CEST)

Ladislav Lhotka <[email-masked]> wrote:
> >> It is a fact of life that YANG is being used outside NETCONF, and
> >> there are more plans for doing do, but with 6020 it is a slippery
> >> slope. For example, such non-NETCONF applications of YANG will most
> >> likely want to use "must" for expressing semantic
> >> constraints. However, the accessible tree for "must" is defined like
> >> so:
> >> 
> >> This cannot be unambiguously interpreted in a non-NETCONF context, so
> >> it is effectively undefined, which is rather dangerous.
> >> So I don't mean to completely redesign YANG, but we cannot, on the
> >> other hand, enable non-NETCONF YANG data modelling without providing
> >> new definitions.
> >> 
> >> Lada
> I don't think this is true. I2RS will probably have no distinction
> between config and state data

FWIW, this is not how I understand I2RS - I think they explicitly
mention the configuration data as being one thing, and that they want
another "direct" method to modify the operational state.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 6 Aug 2013 08:45:07 +0200

> another "direct" method to modify the operational state.

Yes, this is also my reading of draft-atlas-i2rs-architecture-01.txt,
which is likely going to become the I2RS architecture document soon.

/js

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 6 Aug 2013 11:47:08 +0200

> I don't think this is true. I2RS will probably have no distinction between config and state data, though they may have read-write and read-only. Sessions and capability negotiation are by no means…
IMHO we can debate whether some protocols need some of the features, or 
whether some of the features belong to the plumbing or the pure 
protocol, but I still believe that many of the concepts like "NETCONF 
datastore" above are actually easily understandable without considering 
the protocol on the wire. If you would just remove the word Netconf from 
your example paragraphs IMHO they are quite understandable in a protocol 
independent way.

Let me just try as an exercise:
Running Datastore:

  A configuration datastore is defined as the complete set of configuration data that
    is required to get a device from its initial default state into a
    desired operational state.  The configuration datastore does not
    include state data or executive commands.

    The running configuration datastore holds the complete configuration
    currently active on the network device.  Only one configuration
    datastore of this type exists on the device, and it is always
    present.

The above defeinition is extracted from RFC6241 (I just removed 2 
sentences). I think it is a very nice protocol independent definition.

Does anyone have a list of features that make YANG protocol (Netconf) 
dependent?

Balazs

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 06 Aug 2013 12:20:40 +0200

> these 'elements' become YANG 2.0 extensions?

It depends. YANG 2.0 could work with the general notion of a "document" and an adaptation specification for a particular protocol would have to say what the document is supposed to be. For NETCONF, it could be (depending on the context) one of the datastores, running + state data, etc.

The "rpc" and "rpc-reply" statements in YANG 2.0 would only determine the data contents, and the adaptation spec for a protocol would specify encapsulation, framing etc. - for instance, for NETCONF it would be <rpc> while for RESTCONF it would be HTTP POST.

And, for example, the "config" statement could possibly be a NETCONF specific extension.
 
> to use YANG as it is today, perhaps there is no problem to be solved.

The problem to be solved is to decouple YANG from NETCONF. What I meant is that semantics of DSDL languages are not defined in terms of NETCONF concept, so it might be possible to "backport" that semantics to YANG.

> N different interpretations of 'something').

I hope I explained what I meant by 'protocol-independent DML' satisfactorily. :-)

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Aug 2013 03:38:13 -0700

Hi,

I implemented the entire YANG-API (-01) draft within our server.
There was no problem adapting the XPath at all, because the
running config + state is exactly the same context in both.

The <rpc-input> XPath can be easily applied for calls
such as /yang-api/oerations/get-config because the
exact same message payload structure is expected.

When we add notifications to RESTCONF, we will also make sure that
the notification context is exactly the same.

The only part of YANG we had to ignore was the insert operation
parameters, encoded as XML attributes.  The 'point' query string
parameter is used instead of embedded XML attributes in the config.
The yang-api-01 draft only allows the target node to be inserted,
not arbitrary subnodes like an <edit-config> payload.

IMO, all proposals to all WGs should be backed up by some running code,
even experiments.  When theorizing about solutions, it is easy to identify
huge problems that actually do not exist, and just as easy to miss huge
problems hiding in the implementation details.

Andy

On Tue, Aug 6, 2013 at 2:47 AM, Balazs Lengyel
<[email-masked]> wrote:
>>>> It is a fact of life that YANG is being used outside NETCONF, and there
>>>> are more plans for doing do, but with 6020 it is a slippery slope. For
>>>> example, such non-NETCONF applications of YANG will most likely want to use
>>>> "must" for expressing semantic constraints. However, the accessible tree for
>>>> "must" is defined like so:
>>>>
>>>> This cannot be unambiguously interpreted in a non-NETCONF context, so it
>>>> is effectively undefined, which is rather dangerous.
>>>> So I don't mean to completely redesign YANG, but we cannot, on the other
>>>> hand, enable non-NETCONF YANG data modelling without providing new
>>>> definitions.
>>>>
>>>> Lada
>>>
>>> transactional behavior...) and background plumbing (datastores, separation
>>> of state and config data, hierarchical data organisation, filtering,
>>> connection requirements, session concept, capability negotiation, many of
>>> the capabilites, error messages).
>> I don't think this is true. I2RS will probably have no distinction between
>> config and state data, though they may have read-write and read-only.
>> Sessions and capability negotiation are by no means necessary (RESTCONF
>> doesn't have it). I can actually imagine data modelling without any specific
>> protocol for data exchange.
>>
>> Lada
>
> whether some of the features belong to the plumbing or the pure protocol,
> but I still believe that many of the concepts like "NETCONF datastore" above
> are actually easily understandable without considering the protocol on the
> wire. If you would just remove the word Netconf from your example paragraphs
> IMHO they are quite understandable in a protocol independent way.
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 06 Aug 2013 12:55:37 +0200

> which is likely going to become the I2RS architecture document soon.

That document clearly states (Sec. 5.3): "... local device configuration is considered to be separate from the I2RS data store." So if Fig. 1 indicates that I2RS agents interact with Local Config, I assume it has to do so as a compliant NETCONF client. So "config true" data stay fully in the NETCONF realm.

My conclusion from the discussion with the authors of draft-nitinb-i2rs-rib-info-model-01 is that I2RS data store is a subset of "config false" data from the NETCONF viewpoint, except that they are all read-only for NETCONF whereas the I2RS agent can write some of them.

Maybe I2RS folks reading this thread can shed more light on this issue.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Aug 2013 04:15:37 -0700

> My conclusion from the discussion with the authors of draft-nitinb-i2rs-rib-info-model-01 is that I2RS data store is a subset of "config false" data from the NETCONF viewpoint, except that they are…

I have not finished the code yet, so I will not comment on details,
but it should be quite possible to allow writable operational state
to be supported so it is compatible with NETCONF.

This meeting it was decided the injected state will not
be tied to local config (i.e., made to persist).  It only
lasts until explicitly removed or the next server reboot.

If you remember from the BoF, they want a fast path.
So implementing I2RS with RESTCONF or NETCONF is just
a matter of eliminating the slow parts ;-)

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 6 Aug 2013 13:25:43 +0200

> That document clearly states (Sec. 5.3): "... local device configuration is considered to be separate from the I2RS data store." So if Fig. 1 indicates that I2RS agents interact with Local Config,…

My reading of the I-D is that the want to read config, not write it.
Nothing seems to persists in I2RS.

   The I2RS Agent will not attempt to retain or reapply state across
   routing element reboot.

Anyway, we may have to check this document to make sure it really
clearly says this when it gets final using a terminology we do
understand.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 6 Aug 2013 13:27:51 +0200

> I hope I explained what I meant by 'protocol-independent DML' satisfactorily. :-)

Lets see: You want YANG to evolve into a general XML schema language
if you say "protocol-independent". Is that a correct summary?

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 6 Aug 2013 13:36:24 +0200

On Aug 6, 2013, at 11:47 AM, Balazs Lengyel <[email-masked]> wrote:

> IMHO we can debate whether some protocols need some of the features, or whether some of the features belong to the plumbing or the pure protocol, but I still believe that many of the concepts like "NETCONF datastore" above are actually easily understandable without considering the protocol on the wire. If you would just remove the word Netconf from your example paragraphs IMHO they are quite understandable in a protocol independent way.
> 
> The above defeinition is extracted from RFC6241 (I just removed 2 sentences). I think it is a very nice protocol independent definition.

This is certainly NOT a NETCONF-independent definition! Consider a data model for an entire network: there is no single device to apply the data to, and the "protocol" for updating the data store could be Emacs. But then one may be able to transform such a document into real configurations for real devices.
 
> 
> 
> Does anyone have a list of features that make YANG protocol (Netconf) dependent?

Apart from many definitions, there are (CL) rules that follow from how NETCONF was designed. For example: in general it is not necessary that lists have keys. Databases such as mongodb work fine without keys, although they allow for defining optional keys for indexing (performance reasons).

YANG has mandatory list keys for config data because edit-config cannot work without them.

Maybe it is easier to start with features that should be common to all YANG data models:
They should be able to describe a schema for a data hierarchy following the (restricted) XML model, including grammar, data types, semantic rules and defaults, and similar schemas for RPCs and notifications.

I believe it is possible to write a new YANG spec without referring to any NETCONF data stores, state data or operations.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Aug 2013 04:46:17 -0700

...
>>> > create just another example of the 2nd systems effect [1].
....
> It depends. YANG 2.0 could work with the general notion of a "document" and an adaptation specification for a particular protocol would have to say what the document is supposed to be. For NETCONF,…

We got chartered to create YANG because it is not
a general instance document definition language.

I misspoke when I said "separate" protocol-independent parts.
I meant just identify any differences in the protocol mapping
document. No update to RFC 6020 is needed for this purpose.

Your YANG-to-JSON draft is not quite what is needed.
The real translation is YANG to RESTCONF (to NETCONF
is already embedded in RFC 6020).  Then there are JSON
or XML encoding rules for the RESTCONF protocol.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 6 Aug 2013 13:47:13 +0200

> if you say "protocol-independent". Is that a correct summary?

Essentially, yes. However, since YANG only addresses a subset of XML, it could also work for JSON.
And then there would be special parts defining RPCs and notifications.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Aug 2013 04:59:54 -0700

...
> YANG has mandatory list keys for config data because edit-config cannot work without them.

I started implementing the optional-key feature in yang-api-01 but bailed
on it because it was not going to be compatible with NETCONF, and the
corner-cases for network device data structures that don't need keys
are very limited.

Fortunately we had the foresight to model the operation layer in YANG,
so it is trivial to add a custom operation to let the server pick the
key value (that is what mongodb is doing as well) like
<create-loopback-interface>.

> I believe it is possible to write a new YANG spec without referring to any NETCONF data stores, state data or operations.

So far I have not heard anything that indicates RFC 6020 is broken.
A new type of choice-stmt or XPath extensions, etc. are new features,
not bug fixes.  They can go in new RFCs that may update RFC 6020.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 6 Aug 2013 14:18:07 +0200

> document. No update to RFC 6020 is needed for this purpose.

RESTCONF (YANG-API) is pretty close to NETCONF in that it uses the same datastores, but still, you have to use RFC 6020 in a very casual way. For example: 6020 defines a precise encoding for "rpc" nodes, but there is no <rpc> element inside in the data part of the HTTP POST method. 

> or XML encoding rules for the RESTCONF protocol.

Show me an XML snippet that can be modelled with YANG and cannot be converted to JSON using my draft (and please don't use anyxml:-).

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 6 Aug 2013 05:27:09 -0700

> RESTCONF (YANG-API) is pretty close to NETCONF in that it uses the same datastores, but still, you have to use RFC 6020 in a very casual way. For example: 6020 defines a precise encoding for "rpc"…

That will have to get fixed before RFC publication.

> Show me an XML snippet that can be modelled with YANG and cannot be converted to JSON using my draft (and please don't use anyxml:-).

You missed the point.
NETCONF or RESTCONF peers do not send YANG messages.
They send NETCONF or RESTCONF protocol messages.
Various protocol operations return protocol-related meta-data
associated with particular YANG data nodes.

Your draft does not support protocol-related meta-data.
Maybe this should be specified in the protocol document as an
extension to your base encoding document.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 6 Aug 2013 14:51:49 +0200

> corner-cases for network device data structures that don't need keys

I do see the opposite: there is no sensible key to be assigned to routes in routing tables (or a list of static routes). It would be nice to be able to locate a route using *any* subset of route attributes. Of course, preformance-wise it might be desirable to create indices for some attributes, but using other attributes should still be allowed.
  
> So far I have not heard anything that indicates RFC 6020 is broken.

Well, I am sure you've heard at least about the "when" statement: Its context node is ill-defined, and 6020 states no constraints for its XPath expression, which could lead to various race conditions. This has been discussed several times in the NETMOD mailing list.

> not bug fixes.  They can go in new RFCs that may update RFC 6020.

Introducing a new (alternative) choice statement via an extension should be a taboo. Instead, it is necessary to change the semantics of the existing choice statement. 

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 6 Aug 2013 15:11:09 +0200

> associated with particular YANG data nodes.

The document deals just with the data part and it is not intended only for RESTCONF.

> extension to your base encoding document.

Yes, this is possible and IMO better.

Lada

---

From: t.petch <[email-masked]>
Date: Tue, 6 Aug 2013 11:34:34 +0100

----- Original Message -----
From: "Juergen Schoenwaelder" <[email-masked]>
To: "Andy Bierman" <[email-masked]>
Cc: "Netconf" <[email-masked]>; <[email-masked]>
Sent: Monday, August 05, 2013 6:53 PM

> What is to be done about config state and operational state?

Define them in a standards track RFC.

I did look to review some of the modules in Last Call, decided I lacked
sufficient understanding of the current thinking on how many 'states'
there were and what the differences between them are, went looking for
definitions, was unable to find any in any RFC or current I-Ds - and
gave up:-(

I suspect that if you are full time Netxxx, then this is not a problem;
I am not and it is.

Tom Petch

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 6 Aug 2013 15:29:40 +0200

> definitions, was unable to find any in any RFC or current I-Ds - and

What about RFC 6244 Section 4.3 as a start?

/js

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 06 Aug 2013 11:42:42 -0400

If there is an arrow in figure 1 of the I2RS architecture implying that 
an I2RS agent manipulates the config data, then we need to fix that. 
The intention of the authors (and the intention agreed in earlier 
documents) is that I2RS does not mes with config data.
The amplification in the architecture document is that I2RS simply does 
not mess with any persistent data storage at all.  I2RS state is lost at 
reboot.  The working group sounded happy with this proposal in Berlin. 
We will see what happens on the list.

Yours,
Joel M. Halpern,
I2RS Architecture coauthor and netconf/netmod spectator.

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 9 Aug 2013 12:43:01 +0200

See the end!

> I believe it is possible to write a new YANG spec without referring to any NETCONF data stores, state data or operations.
{BALAZS}:  I don't see why we would want to get read of multiple 
datastores or state versus configuration data. These are basic device 
management concepts that are relevant to any protocol managing a device. 
E.g. the exsistence of a startup datastore is dependent on the data 
storage and and restart methods used by the device, but is independent 
of the protocol. They are already used by Netconf, CLI, GUI so three 
ways to manipulate the config.

Yes, I can imagine a device, that doesn't use these concepts , but do we 
want to accomodate them as well, or are trying to provide additional 
interfaces that depend on the same basic concepts used by Netconf. If we 
throw out such basic concepts, the deviation from Netconf becomes too 
big for my liking, that is I would not like to see.

Balazs
IMHO:

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Aug 2013 13:11:14 +0200

On Aug 9, 2013, at 12:43 PM, Balazs Lengyel <[email-masked]> wrote:

> {BALAZS}:  I don't see why we would want to get read of multiple datastores or state versus configuration data. These are basic device management concepts that are relevant to any protocol managing a device. E.g. the exsistence of a startup datastore is dependent on the data storage and and restart methods used by the device, but is independent of the protocol. They are already used by Netconf, CLI, GUI so three ways to manipulate the config.
> 
> Yes, I can imagine a device, that doesn't use these concepts , but do we want to accomodate them as well, or are trying to provide additional interfaces that depend on the same basic concepts used by Netconf. If we throw out such basic concepts, the deviation from Netconf becomes too big for my liking, that is I would not like to see.

Do you mean we should recommend the authors of draft-clemm-netmod-yang-network-topo-00 to drop YANG and use XML schema languages instead? Their data model is clearly outside the scope of device-oriented network management - in fact, the very question whether the contents are config true or false doesn't make much sense to me.

What I had in mind with YANG 2.0 was a more generic data modelling language that could be used for such cases, too. A small "adaptation" document could then provide specific info for a particular use or protocol. The result of this combination for NETCONF could be pretty much the same as RFC 6020.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Aug 2013 14:10:58 +0200

> What I had in mind with YANG 2.0 was a more generic data modelling language that could be used for such cases, too. A small "adaptation" document could then provide specific info for a particular u…

I do not see why YANG can't be used to read/write configuration to a
network manager that generates device configuration. I do not know
whether this is the intention behind the I-D but ultimately I hope we
start working at some point in time on _network_ management rather
than just _device_ management.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Aug 2013 15:16:05 +0200

> than just _device_ management.

The fact of the matter is that YANG is used there as an XML schema language, unrelated to NETCONF.

I can imagine YANG to be a good candidate language for modelling the I2RS data store, which by definition isn't NETCONF data store.

On a practical side: YANG might be useful to people that have no idea about NETCONF and don't want to get any. For them, reading RFC 6020 would be an unnecessarily difficult and long exercise, with all the references to NETCONF data stores, <edit-config> operations, server behaviour etc.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Aug 2013 15:47:51 +0200

On Aug 9, 2013, at 3:38 PM, Juergen Schoenwaelder <[email-masked]> wrote:

> On Fri, Aug 09, 2013 at 03:14:42PM +0200, Ladislav Lhotka wrote:
>> 
> This is your interpretation.

Sure.

> This is your interpretation.

Sure.

> It would be nice if those people trying to use YANG for completely
> different purposes would stand up and tell us that there indeed is a
> problem with using RFC 6020 and what exactly this problem is.

At the end of the NETMOD session I heard several voices in support of my proposal to decouple YANG from NETCONF.

Lada

> 
> Your position seems clear to me - no need to repeat it.
> 
> /js
> 
> --

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Aug 2013 15:56:22 +0200

> At the end of the NETMOD session I heard several voices in support of my proposal to decouple YANG from NETCONF.

The problem is that people sometimes agree to something where the same
people afterwards find out it means very different things to them.
Hence I am trying to find out what people really want - there are
apparently different definitions for "decouple YANG from NETCONF" that
people have in their minds.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 Aug 2013 07:11:45 -0700

>>> What I had in mind with YANG 2.0 was a more generic data modelling language that could be used for such cases, too. A small "adaptation" document could then provide specific info for a particular…

I don't agree YANG 2.0 is needed at this time.

>> than just _device_ management.

Just implement a mid-level-manager and you have network management.
There is nothing in NETCONF/YANG that says it can only be used by
an NMS to manage an end-device.

Write the code -- have the MLM talk to multiple southbound devices.
Then you will have network management.

I don't think the industry needs the IETF to do everything for them.
I don't see any barriers preventing the IETF from standardizing
network configuration.  Write a draft if you think it is that
important.

> The fact of the matter is that YANG is used there as an XML schema language, unrelated to NETCONF.

This is incorrect.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Aug 2013 16:44:47 +0200

>> The fact of the matter is that YANG is used there as an XML schema language, unrelated to NETCONF.

Why? It just models a single XML document without any special relationship to NETCONF datastores, state data, RPCs or whatever. Tell me a single aspect that goes beyond plain XML.

During Jan's presentation in Berlin, Martin asked via Jabber: "Why did you model this as configuration data?" And indeed, how can one decide which part of the data is configuration and which is operational state data? The answer actually must be "It depends." But this is important, because several rules in YANG rely on the distinction between config true and false nodes.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 Aug 2013 08:38:04 -0700

> Why? It just models a single XML document without any special relationship to NETCONF datastores, state data, RPCs or whatever. Tell me a single aspect that goes beyond plain XML.

I think a protocol mapping document (YANG for I2RS) could specify
the relationship to NETCONF datastore concepts and new datastores.
I like the 'dynamic' datastore that Juniper proposed awhile back for
I2RS purposes.

The NETCONF-specific aspects of YANG are not hard to map
to other protocols.  The concept of a running datastore (local config)
is fairly universal.  The <rpc> and <notification> XPath context details
can be ignored if those messages do not apply.

The "dynamic" datastore is a stretch (wrt/ standard text) because
YANG has datastore validation requirements that may get bypassed
if fast-path edits do not use full validation.

> During Jan's presentation in Berlin, Martin asked via Jabber: "Why did you model this as configuration data?" And indeed, how can one decide which part of the data is configuration and which is ope…

We don't have the right YANG meta-data to describe writable
operational state.

IMO, config=false clearly means that no editing is allowed.
Unfortunately, config=true means "validate and persist as part
of the running datastore" in NETCONF.  A simple YANG
extension to overwrite that property can fix that.

The YANG RFC does not discuss the startup datastore.
Any YANG-to-??? protocol mapping document needs to
define how new datastores interact with the NETCONF datastores.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Aug 2013 18:29:08 +0200

> to other protocols.  The concept of a running datastore (local config)

Sometimes there even needn't be any protocol involved.

> if fast-path edits do not use full validation.

It would be possible to define various validation steps in a general way, e.g., (i) grammar + data types and (ii) semantical rules, and then state which of them are to be applied to a particular datastore.

> We don't have the right YANG meta-data to describe writable

Yup.

> extension to overwrite that property can fix that.

Hmm, how can you override the semantics of an existing YANG statement with an extension? Isn't it the kind of extension that Martin warned against?

A much simpler approach would be to talk just in terms of RW and RO, and specify special properties and behaviour of each datastore/protocol where needed. 

Lada

---

From: Balazs Lengyel <[email-masked]>
Date: Fri, 9 Aug 2013 18:52:43 +0200

> A much simpler approach would be to talk just in terms of RW and RO, and specify special properties and behaviour of each datastore/protocol where needed.
IMHO it was a mistake to combine two concepts writability and 
persistence in the single config=true/false statement. So I very much 
like your comment.
Balazs

---

From: "Jan Medved (jmedved)" <[email-masked]>
Date: Fri, 9 Aug 2013 16:55:38 +0000

[Snip]

On 8/9/13 9:29 AM, "Ladislav Lhotka" <[email-masked]> wrote:

>>>During Jan's presentation in Berlin, Martin asked via Jabber: "Why did
>>>you model this as configuration data?" And indeed, how can one decide
>>>which part of the data is configuration and which is operational state
>>>data? The answer actually must be "It depends." But this is important,
>>>because several rules in YANG rely on the distinction between config
>>>true and false nodes.

+1. 

The topology model described in the draft is a relatively easy use case -
it's read-only, so it can be (and should have been :-)) modeled as
operational data. 

But we need to model any kind of service services, which in general can be
read-write and writes could be either ephemeral or persistent (config).

[snip]

Thanks,
Jan

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 09 Aug 2013 19:08:10 +0200 (CEST)

> > IMO, config=false clearly means that no editing is allowed.

I disagree.  (but maybe we mean the same thing anyway).  config false
means that the data node is not part of any configuration datastore.
Since NETCONF just has two classes, config and state, and currently
the only standard operations to modify anything are edit-config and
copy-config, which are used to modify the configuration datastores, it
follows that you cannot edit a config false node with these two
operations.  But nothing prevents us from defining a new
edit-operational operation, for example.  Or special rpcs to modify
say a routing table.  So in this sense config false does not mean that
the node is strictly read-only.

> Hmm, how can you override the semantics of an existing YANG statement
> with an extension? Isn't it the kind of extension that Martin warned
> against?

No, since you do not override the existing semantics.  A config false
node with an extension "writable-operational" is still not part of the
configuration datastores.  Old clients can still communicate properly
with new servers.

> A much simpler approach would be to talk just in terms of RW and RO,
> and specify special properties and behaviour of each
> datastore/protocol where needed.

I strongly disagree.  config true is (obviously) very specific to
configuration management, but this is one thing that makes NETCONF /
YANG different compared to other solutions.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 Aug 2013 10:13:04 -0700

> the node is strictly read-only.

I agree this is a better description of config=false.

> YANG different compared to other solutions.

+1

Andy