Subject: Re: [netmod] container netconf

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 02 Jan 2009 13:08:11 +0100

Martin Bjorklund píše v Po 29. 12. 2008 v 09:57 +0100:
> > 
> > Every NETCONF model (rooted under /netconf) is going
> > to have to augment the notifications data model.
> > That doesn't seem very clean, but multiple /netconf containers
> > in different XML namespaces is even worse.
> > 
> > Suggestions to clean this mess up before it starts?
> 
> That would be nice, and I believe that Sharon's intention has always
> been to have a single 'netconf' container.
> 
> However, we're sort of stuck with the limitations of XSD.  YANG is
> designed to handle this situation, but the YANG models are
> non-normative.

On the other hand, this is a typical application for NVDL, see for
example this paper:
http://2007.xtech.org/public/schedule/detail/48

Lada

> 
> Assuming we cannot change the structure of the data model in the
> notification RFC, and that we want one single 'netconf' container, it
> would have to be called:
> 
>    {urn:ietf:params:xml:ns:netmod:notification}netconf
> 
> which is not optimal...
> 
> Anyway, here are a few alternatives:
> 
>   1)  Add a substitionGroup to the 'netconf' element, as an errata to
>       rfc 5277.   The monitoring schema would then use this
>       substitionGroup.
> 
>   2)  As an errata to 5277, add text that explains that the XSD is
>       supposed to be completely open (<openContent> in XSD 1.1)  The
>       monitoring schema would then have text that explains that the
>       contents would go under the
>       {urn:ietf:params:xml:ns:netmod:notification}netconf element.
> 
>   3)  Live with it, and do better next time.  Maybe as soon as in the
>       monitoring spec - it could define a simple schema / YANG module
>       with a single 'netconf' container in a generic namespace, and
>       then augment this container with the monitoring stuff.   When we
>       do the third data model for netconf, we will then have this
>       'netconf' container element in place.
> 
> 
> But there are still a couple of unanswered questions - should we have
> one 'netconf' container for config and one for operational data?  If
> we have one single container for both kinds, should we have two
> distinct subtrees underneath that one?
> 
> 
> /martin
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 Jan 2009 08:58:27 -0800

>> been to have a single 'netconf' container.

This is where the difference between good intentions
and good experience starts to show up, and why we
need to get started with NETMOD, on the right track.

Q wrap-up:

1) Is the netconf root a presence container or an NP container?
    Since we defined no meaning for the root, it must be NP.

2) Can the errata process be used to adjust the XSD as
    Martin suggests? (adding a 'netconf-component' substitution
    group that other XSDs can use, to match the YANG augment).

3) Should there be separate <netconf> roots, one for monitoring
    (the existing one) and another one for configuring the
    NETCONF protocol? (if and when that is needed)

4) What is the right way to organize the NETCONF MIB tree?
    IMO, separating non-config is really going against the grain wrt/
    aggressively hierarchical nature of XML.  I prefer
    one subtree for the 'foo service' containing everything
    there is to know about that service.  OID values in SMIv2
    are totally random and have no semantics.  I still cannot
    remember any of them by heart after 20 years.
    But "/interfaces/interface/ifMtu" could not be more clear.
    Scattering data randomly (over time) will ruin this important
    feature of XML.

Andy

---

From: "David Harrington" <[email-masked]>
Date: Fri, 2 Jan 2009 12:27:54 -0500

Hi,

Some comments from SNMP-land:

We tried separating SNMP into a separate subtree, paralel to mib-2,
and did not find it that useful. Subsequently, we recommned keeping
everything under mib-2, whether it is SNMP-protocol-specific or not. I
am not sure having a separate XML subtree for netconf-protocol config
will be found to be useful.

For a purely technical backwards-compatibility reason, mib modules are
now recommended to use a subtree for notifications, another for
objects, and a third for conformance, under the mib module
registration. It might be good to standardize something similar for
netconf. It might have been useful to separate config info from state
info in mib modules, but we never found it justified, given that we
only had read-only vs read-write to distinguish the two. Since netconf
does a better job of separation, this might be useful in nnetconf
models. 

Of course, netconf could create a netconf/notifications/subtree and a
netcoinf/config subtree and a netconf/state subtree and then require
modules to register as a notification model, a config model, or a
state model.

dbh

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Andy Bierman
> Sent: Friday, January 02, 2009 11:58 AM
> To: Ladislav Lhotka
> Cc: [email-masked]
> Subject: Re: [netmod] container netconf
> 
> Ladislav Lhotka wrote:
element.

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 Jan 2009 10:55:17 -0800

> modules to register as a notification model, a config model, or a

This is sort of Apples and Oranges.
Either that, or "We're not in Kansas anymore" :-)

RPCs and notifications do not show up in the conceptual NETMOD
data tree.  They only show up in PDUs in the module's XML namespace.

The contents of a standard NETCONF database would benefit from
an organizational structure other than "ad-hoc".
An operator who knows even a little XPath could easily
remember '/netconf' or '/system' or /interfaces' or
even "/interfaces/interface[name='eth1']".

However, if data is not maintained in an organized manner,
then an operator has no hope of remembering all the 'places'
where interesting data 'lives' in NETCONF.

IMO, the ease-of-use to operators of the long term
is easily neglected by the WG, yet it should be the #1 priority.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 2 Jan 2009 19:56:19 +0100

On Tue, Dec 23, 2008 at 04:20:14PM -0800, Andy Bierman wrote:

> I was just looking at netconf-state.yang
> and then I looked at notifications.yang.

Which ones? I must admit I am not sure.

> They both define a top-level <netconf> container.
> One of them defines it 'config true' and the other one
> defines it 'config false'.
>
> I know it is early for real data models and all,
> but notifications is a standards track RFC,
> so it must be right ;-)
>

You seem to suggest that netconf-state.yang and notifications.yang
become YANG submodules included by a netconf.yang module since
otherwise the definitions will be in different namespaces and hence
there is no common root anyway.

For the SNMP VACM module (see the testing report), I have done exactly
that.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 Jan 2009 12:31:23 -0800

> there is no common root anyway.

I am not suggesting that, but it is an issue.
That would be the only way to keep 'netconf stuff' in
1 XML namespace.  I don't think that is very important
in XPath.  It's an issue in subtree filtering though.
The <notification> is already in a different namespace
than the rest of NETCONF.  Augments using different namespaces
to avoid coordination of the naming.
Putting 'everything' in one namespace would impact augment.

You seem to be suggesting, that if we wanted to
consider an organizational algorithm other than "ad-hoc",
one of them would be:

   1 main module per 'category':

   netconf.yang: (just a bunch of includes)

     include netconf-root.yang
     include notifications.yang
     include nc-notifications.yang
     include netconf-state.yang
     include partial-lock.yang

This would only really work if the main netconf.yang was
kept online on IANA.  Republishing that as an RFC every
time a new module came out seems cumbersome.

I prefer N namespaces, but starting out with an
augment-able container that gets designated for the category.
All standard extensions (and vendor if they want) would
be required to use the most appropriate existing container.
The YANG MIB Doctors should have to approve new standard
containers, as part of their review process.

   module netconf-root {

      // no objects except this empty NP container
      container netconf;
   }

   module partial-lock {

     import netconf-root { prefix root; }

     augment "/root:netconf" {
        // all the partial-lock objects in this namespace
     }

     rpc partial-lock { ... }

     rpc partial-unlock { ... }
   }

This requires no IANA support, and separate RFCs for each
data model in the category do not impact each other
(especially with import-by-revision added to the mix).

> For the SNMP VACM module (see the testing report), I have done exactly

I noticed.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 04 Jan 2009 12:13:03 +0100 (CET)

> in XPath.  It's an issue in subtree filtering though.

Do you mean because of this sentence from 6.2.1 of rfc4741:

   If namespaces are used, then the filter output will only include
   elements from the specified namespace.

This is another area in need of clarification.  What is the intention
of this feature?   What exactly does "if namespaces are used" mean?

Putting pieces of information together, I *think* the intention is
that this expression:

   <filter>
     <netconf xmlns="urn:ietf:params:xml:ns:netmod:notification"/>
   </filter>

would suppress any augementing data models under the 'netconf' root,
but that this expression:

   <filter>
     <netconf xmlns=""/>
   </filter>

would return all augmenting data models.  (and also return other data
models with a top element 'netconf').

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 04 Jan 2009 15:58:17 +0100 (CET)

> http://2007.xtech.org/public/schedule/detail/48

NVDL is part of DSDL, so I guess it would be ok to use it in our DSDL
mapping.  Could NVDL be used to solve our "super-model" issue?  The
idea would be to generate one RelaxNG schema per YANG module, with no
additional genererated top-level elements.  Then a device can make a
susper-schema in NVDL that describes how the individual data models
are combined.  This super-schema could even be advertised in the
schema discovery list so clients can learn it from the device.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 05 Jan 2009 09:40:05 +0100

> schema discovery list so clients can learn it from the device.

Yes, I think this could be done, but NVDL can also serve as a formal
glue between NETCONF and data model schemas. Note that NVDL can also use
XSD as the schema language for some sections of the composite schema.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Balazs Lengyel <[email-masked]>
Date: Mon, 05 Jan 2009 12:21:20 +0100

+1

I think we absolutely should provide some structure. This looks like a good idea, that IMHO
conforms to the basic idea we had when we defined the notifications draft.

However we would probably need to verify tis decision on the Netconf list as well.

Balazs

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Jan 2009 13:14:33 +0100

> be required to use the most appropriate existing container.

Which problem does this solve?

> containers, as part of their review process.

My understanding is that MIB doctors review and provide input to the
AD and the IESG but do not approve anything.

Back to the subject. Can we make a list of options and list the pros
and cons? Here is a start:

a) do nothing - modules pick namespaces and roots at random

   pro: no coordination required
   con: potentially confusing

b) related definitions share a single namespace by using submodules

   pro: all related definitions share the same namespace making
        filtered retrieval can cross references easier
   con: requires coordination and an IANA maintained module

   com: since vendor extensions must use augmentations, there is
        organizationally a difference between extensions provided by
        the owner of the module (namespace) and extensions provided by
        other parties

c) related definitions in separate modules (namespaces) augment a
   common root

   pro: requires no coordination via IANA since everyone is free to
        augment anything
   con: related definitions exist in different namespaces potentially
        complicating filtered retrieval and cross references

Are there more options or pros and cons? What do operators prefer?  is
having all SNMP/NETCONF/you name it definition in one namespace
preferrable over a solution where things are split into N different
namespaces, reflecting the evolution of the definitions over time?

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 07:12:06 -0800

> Which problem does this solve?

Multiple namespaces solves naming collision problems.
Maintaining some coherence and predictability across
the entire set of managed objects makes the system easier
to navigate and understand.

Individual WGs (and dev-groups within companies)
rarely consider the managed system outside their little
piece of it.  The result is what we have now with SNMP and CLI.
Ad-hoc, almost random data organization.  Lots of data silos
requiring independent maintenance and application code design.

Subtree filtering could be changed so that including
a node in the filter automatically means include all
its descendants, regardless of namespace (so it works
the same as XPath).

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Jan 2009 16:19:21 +0100

> requiring independent maintenance and application code design.

So how is this fixed by having different namespaces for all the
various pieces that belong to SNMP / NETCONF / OSPF etc?

> its descendants, regardless of namespace (so it works

In which version of NETCONF do you want to do this?

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 07:41:32 -0800

> various pieces that belong to SNMP / NETCONF / OSPF etc?

Why are multiple XML namespaces a problem?
Each YANG module maps to a (hopefully)
globally unique namespace.  This removes any need
for coordinating selection of identifiers across
modules (which is the point of XML Namespaces).
Augment is robust instead of fragile, only if different
namespaces are used.

IMO, standards based submodules are problematic
because there is a main module 'include-stmt'
that must be added every time a new submodule
is added.  The main module revision-date needs to be changed
every time this happens.  Every module importer
will need to check if the submodules they care
about really changed or not.

Maintaining the main module on IANA
puts a huge burden on tools which have to deal
with this special case, and get the main module from IANA.
There are significant standards conformance issues
as well if a standard version of a data model
is not published in any RFC.  Normative reference
resolution would get really complicated over time.

It effectively removes version control from the main module
and moves it to the submodules instead.  This has
an impact on the 'importer' module that needs
a specific version. There is no visibility into
submodules A.1 or A.2 when module B is importing module A.

> In which version of NETCONF do you want to do this?

in 1.1

How many implementations deal with
subtree filtering of multiple namespaces correctly
anyway?  I doubt any implementations even have
a nested node in a different namespace than its parent yet.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Jan 2009 17:02:37 +0100 (CET)

> a nested node in a different namespace than its parent yet.

We have nested nodes in different namespaces, but we do not deal with
them correctly according to the spec - once a subtree is selected, we
print the entire subtree, including any mounted (augmenting)
namespaces.

Here's another unclear issue: will these filters produce the same
result?  (assuming some other namespace has augmented /top/system)

  <filter> 
    <top xmlns="http://example.com/cfg">
      <system/>
    </top>
  </filter>

  <filter> 
    <x:top xmlns:x="http://example.com/cfg">
      <x:system/>
    </x:top>
  </filter>

  <filter> 
    <top xmlns="http://example.com/cfg">
      <system xmlns="http://example.com/cfg"/>
    </top>
  </filter>

The unclarity comes from this sentence:

   A namespace is considered to match (for filter purposes) if the
   content of the 'xmlns' attributes are the same in the filter and
   the underlying data model.

What does "the content of the 'xmlns' attributes" really mean??

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 08:14:24 -0800

> print the entire subtree, including any mounted (augmenting)

So then the 1.1 spec can be clarified to specify
this behavior, right?  This has always been a
problem with subtree filtering.

> What does "the content of the 'xmlns' attributes" really mean??

(My bad.)
It is one of the sentences that needs to be rewritten.
The intent is the expanded-name is the same. The
actual xmlns directives are irrelevant.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 5 Jan 2009 17:15:32 +0100

> Augment is robust instead of fragile, only if different

Yeah, we all agree.

> every time this happens.

Yes.

> Every module importer

Please elaborate.

> with this special case, and get the main module from IANA.

That has been true for IANA maintained MIB modules as well.

> resolution would get really complicated over time.

I am not sure I follow. Please elaborate.

> submodules A.1 or A.2 when module B is importing module A.

Isn't B important a certain revision of A? And do the YANG
rules not make it clear what revision A stands for?

I am wondering whether I should ask the nasty question: if submodules
don't work, why do we have them in YANG?  Or what needs to be changed
to make them work?

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Jan 2009 17:24:28 +0100 (CET)

> problem with subtree filtering.

So far, I don't think we have any other change that would require
us to bump the base version to 1.1.  The only version change I recall
is for the :validate capability.

Do we really want to bump the base version for this?

> actual xmlns directives are irrelevant.

Ok.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 08:42:53 -0800

>> about really changed or not.

If the main module is like this example:

module foo {
   namespace "blah";
   prefix foo;

   include foo-root { version 2008-01-17; }
   include abc { version 2008-01-05; }
   include def { version 2008-04-25; }
   include hij { version 2008-11-12; }
   // ... more submodule includes
   include xyz { version 2009-01-05; }

   revision 2009-01-05 {
      description "updated xyz submodule";
   }
   // 27 more revision statements

}

Every time the main module is republished, the revision
date will be changed.

Even though the importer will technically be safe:

module bar {
   namespace "bar-blah";
   prefix bar;

   import foo { version 2008-04-25; }

   revision 2008-11-05 {
      description "initial version";
   }
   //...
}

The implementer of 'bar' may eventually want to check
what changed in the 'foo' module.  If the 'bar' module
really only uses the 'def' sub-module  from 'foo',
then the foo module revision date is not the relevant
trigger, the 'def' submodule revision date is the trigger.
Since the revision clause is optional in the import
or include, a change to 'def' may not be apparent to
the 'bar' module implementer at all.

> don't work, why do we have them in YANG?  Or what needs to be changed

I don't think the IETF should use them.
It's like "this amp go to 11."
Why not rename submodule to be module,
if that's how they are going to be used?

They add complexity (like top-level augments
into the same namespace, supposedly only needed
to augment across modules).  Some people really
like them so I don't care if vendors use them,
but the IETF should not.

NETCONF is already defining a different namespace
for every little thing that gets added.  Dealing
with 7 namespaces in the PDU instead of 3 or 4 is not a
big deal.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 6 Jan 2009 14:40:12 +0100

> the 'bar' module implementer at all.

But this is then a question of what a good "size" of a module is and
it is pretty much independent of using submodules or not. Even in SNMP
land, if I import BridgeId for BRIDGE-MIB and the module gets revised,
I have to find out by reading the module or using tools such as
smidiff whether I am affected or not. And in your example, the
module revision statement says

>      description "updated xyz submodule";

so it is rather clear what changed.

> with 7 namespaces in the PDU instead of 3 or 4 is not a

I don't care about the PDU - I care about the perspective from an
operator. As you know, the IETF process causes us to sometimes break
strongly related things into rather strange pieces. See for example
the IF-MIB and the IF-INVERTED-STACK-MIB. Or take the modular approach
taken to define management interfaces for SNMP itself. Or take the
NETCONF stuff where we start walking before we learn how to walk well.
As an operator scripting stuff, I prefer having everything related to
a particular protocol or service in one or very few logically well
organized namespaces (a standard one plus a vendor specific extension
one). Perhaps the question is whether we simplify for the operator,
the implementor, or the IETF processes. I have my preference...

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 06 Jan 2009 07:22:28 -0800

> the implementor, or the IETF processes. I have my preference...

I agree that using submodules could be made to work,
although not as easily as just using modules.
I do not agree that the PDU aspect does not matter.
If the NETCONF protocol has retrieval mechanisms
that allow all the IF info to be returned in one
simple request, that also has value to the operator.
The number of XML namespaces involved is not driving the
complexity factor, it's the number of subtrees involved.

Andy