Subject: [netmod] some yang-03 comments

From: Andy Bierman <[email-masked]>
Date: Mon, 19 Jan 2009 14:44:19 -0800

Hi,

Sections 7.1.5 (import) and 7.1.6 (include) do not clearly state
what happens if the optional revision sub-clause
specifies a revision that the compiler cannot find.

I assume a fatal error occurs, and no [sub-]module is
used at all for that import or include.

These sections do not say which version of a module
the compiler should use if the optional revision
is not specified.  I thought it was supposed
to indicate that the most recent must be used.

There is no text in the entire document formally defining
a module version field, or how to associate the
contents of the revision sub-clause of import/include
to the contents of a YANG module somehow.

It is assumed that the most recent revision-stmt
date identifies the module, but this is not specified,
and it is not enforceable.

Section 7.1.9 says a module SHOULD have a revision clause,
but it is not mandatory.

There needs to be a standard way to associate some
portion of the contents of a YANG module with
the revision requested in an import or include statement.
It MUST be usable for any valid YANG module.
It MUST be deterministic, and MUST NOT rely on
suggested file naming conventions.

There is also a change in the draft that says the
specified YANG clause order SHOULD be used (not MUST).
IMO, the ABNF order MUST be followed.

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 20 Jan 2009 10:27:54 +0100

> to indicate that the most recent must be used.

Well, but what is the most recent? The upgrades should be controlled,
I'd prefer a mechanism similar to XML catalogs, which is a local
configuration file where an URI is associated to a fixed resource,
either local or remote.

> Section 7.1.9 says a module SHOULD have a revision clause,

Yes, and moreover the reverse order of revision statements is not
prescribed (just SHOULD), so one cannot even rely on the first statement
being the most recent revision.

> suggested file naming conventions.

+1.

> IMO, the ABNF order MUST be followed.

Where is this?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 Jan 2009 06:53:44 -0800

> configuration file where an URI is associated to a fixed resource,

The most recent is the one with the highest value
string for the revision date (strcmp, not XPath cmp).

> being the most recent revision.

no, you have to search through them for the
highest value, if there are any revision statements
at all.

SMIv2 has the LAST-UPDATED clause (but it's optional).
It should be very easy to map the revision date to a specific
named field.  I do not understand why YANG relies
so heavily on module versioning, yet there isn't even
a version identifier for a module or submodule.

The supposed use case is vendor data models.
They should not have to be burdened with
including a revision-stmt.  IMO, this is nonsense.
There are plenty of mandatory clauses (like prefix)
which don't even need to be used, yet they
are mandatory.

I do not think it is a good idea to allow for
the possibility that a managed system contains
some data models with versions and some without.
This just adds complexity without any benefit.
A missing revision date in the import cannot
mean "pick whatever revision you want".
That is not inter-operable.

I do not like import-by-revision because
it is quite sloppy.  I think the manager
(and agent) need to be aware of the the exact
module versions being used.   Why bother
with exact deviations when you can't even figure out
which version of the main module is being modified?

>> IMO, the ABNF order MUST be followed.

7.1, para 4.
IMO, it should say this is a suggested order,
but the real order in the ABNF needs to be followed.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 20 Jan 2009 16:10:41 +0100

> string for the revision date (strcmp, not XPath cmp).

OK, but this depends on where you search: the most recent revision
available locally on disk may be older than another revision that is
somewhere online.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 Jan 2009 07:24:54 -0800

> available locally on disk may be older than another revision that is

This seems out of scope for NETMOD.
It seems traditional in the IETF NM area to
ignore such practical matters as how a file
is named, or where to find it.

It is assumed the tools know where to find the
modules, know how to tell them apart, and know
how to pick the right one out of N when no
revision date is specified.  This is considered
just an implementation detail.

I agree with you that this approach is not good enough.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 20 Jan 2009 17:30:37 +0100

> SMIv2 has the LAST-UPDATED clause (but it's optional).

Probably not important, but LAST-UPDATED is not optional.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 Jan 2009 13:33:09 -0800

> Probably not important, but LAST-UPDATED is not optional.

It is important.
It means it has not been a burden at all to maintain
this field in MIBs, so it should not be a problem
to maintain it in YANG either.

I prefer a new mandatory 'last-updated' or 'version' clause,
rather than hunting down the optional revision-stmt
with the highest value revision date.

This should be way at the top, in the [sub-]module-header,
before the linkage statements, so it will be easy for people
and tools to identify the version immediately.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 20 Jan 2009 22:59:33 +0100

> Juergen Schoenwaelder wrote:

That is not quite correct - or at least I see this differently.  Until
we added code to libsmi to check that the LAST-UPDATED value and the
REVISION clauses do match, they were often out of sync. Our MIB
guidelines meanwhile require that a REVISION clause is added when a
new MIB is published which is a good thing since the guidelines
require more information to be in the description clause. Having a
redundant "last-updated" date in another clause is for me just noise,
asking for trouble if things get out of sync. Since libsmi
automatically puts revision clauses into reverse chronological order,
it is really not difficult to find out when a module was last updated.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 Jan 2009 14:17:59 -0800

> On Tue, Jan 20, 2009 at 01:33:09PM -0800, Andy Bierman wrote:

So what is the version of a module which has no revision clauses?

What version is retrieved if no revision is specified in
the import or include?

I do not care what field is used, as long as it is mandatory.
I prefer that it is before the linkage statements.
It is important that import or include by version
be inter-operable, and IMO it currently is not.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 21 Jan 2009 00:31:29 +0100

> So what is the version of a module which has no revision clauses?

I thought I made it clear in my original comment that my SMIv2
clarification might not be relevant for this thread.

> I do not care what field is used, as long as it is mandatory.

And I prefer to avoid redundant fields. Perhaps we are even in
agreement (except perhaps the placement).

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 Jan 2009 15:48:02 -0800

> On Tue, Jan 20, 2009 at 02:17:59PM -0800, Andy Bierman wrote:
> 

I am referring to YANG modules.  Forget SMIv2.
Sorry I brought it up.

> agreement (except perhaps the placement).

I thought the consensus from the interim meeting
was that the revision-stmt was going to be mandatory,
and if the import/include did not specify any version,
then the newest available version would be used.

I do not understand what changed, or the rationale
for changing it, but I object to the change.
The 'suggested' file format is not a solution.
There are plenty of applications (HTTP POST) that
may provide a YANG module without a 'file'
to contain it.

Having to store a 'file name' plus the entire contents
of the YANG module seems redundant and kludgey to me.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 21 Jan 2009 07:10:12 +0100

> may provide a YANG module without a 'file'

I agree that the version number must be defined within the YANG
module.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 21 Jan 2009 09:58:48 +0100 (CET)

> > And I prefer to avoid redundant fields.

+1

> was that the revision-stmt was going to be mandatory,

I cannot find that in my notes.  (and I cannot find any offical
minutes from the interim)

> then the newest available version would be used.

Yes.  This would work like with SMIv2 MIBs.

> for changing it, but I object to the change.

I don't think there is any change.

We have talked about many similar issues in the past - there are
several statements which are optional but will probably be required in
IETF modules (description, contact, organization...)  IETF modules
will always have revision statements, and they will always be in
reverse order.

If anything, I'd rather make both 'namespace' and 'prefix' optional.
'namespace' is required, but it has no meaning for "type library
modules", like e.g. yang-types.

> may provide a YANG module without a 'file'

Of course.  The suggested file name format has always been there, and
it has absolutely nothing to do with this issue.  The suggested file
name format is just that - a recommended way to name yang modules when
they are stored as files.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Jan 2009 01:29:21 -0800

> Andy Bierman <[email-masked]> wrote:

YANG needs to define the behavior for all implementations
to follow. There is a difference between mandatory-to-implement
and mandatory-to-use.  The standard needs to define what
happens when the revision is not found, not specified, etc.
The current draft is under-specified.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 21 Jan 2009 11:03:59 +0100 (CET)

> The standard needs to define what

I guess this applies to an import w/o revision as well?  I.e. we need
to say somthing like "It is an error if the module, or the specified
revision of the module, does not exist."

Was this what you meant?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Jan 2009 02:44:04 -0800

> revision of the module, does not exist."

1) there needs to be explicit instructions for
associated an import revision clause with a YANG module.

module A {

    import foo {
        prefix foo;
        version "2009-01-20";
    }
}

These seems obvious, but what is the deterministic
mechanism that associates the string "2009-01-20"
with a particular instance of the foo module?

2) there needs to be explicit instructions for selecting
an import or include file if the import revision is not
specified:

x.yang:

module foo {

   namespace "foo";
   prefix foo;
   leaf a { type string; }

}

y.yang:

module foo {

   namespace "foo";
   prefix foo;
   leaf b { type string; }

}

module bar {

   namespace "bar";
   prefix bar;
   import foo { prefix foo; }

}

Which file is used for the import into module bar:

x.yang or y.yang?

How does the DM designer know for sure which one will
be used?  Why should this even be allowed in YANG?

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 21 Jan 2009 12:29:30 +0100 (CET)

> Martin Bjorklund wrote:

Ok.  We need to say that it must match the most recent revision
clause.

> be used?  Why should this even be allowed in YANG?

IMO this is outside the scope of YANG.  We cannot mandate that files
must be used, as you said.  The modules might be stored in a
database.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Jan 2009 07:51:54 -0800

> Ok.  We need to say that it must match the most recent revision

And unless there is an exact match with the highest value
revision date, there is no import or include (fatal error).

> must be used, as you said.  The modules might be stored in a

For get that module foo is stored in a file 'x' or 'y'.
It could be a URL 'x' or 'y. or SQL database object name.
Sorry for adding irrelevant data for you to focus on
instead of the real problem.

The correct answer is that an import without a revision
date is exactly what we had before -03, so whatever YANG did to
properly select the correct module before -03, is what it
still does now.

This problem has been in YANG all along.

module foo {

   namespace "foo";
   prefix foo;
   typedef A { type string; }

}

module foo {

   namespace "acme-foo";
   prefix foo;
   typedef A { type float32; }

}

module bar {

   namespace "bar";
   prefix bar;

   leaf which-is-it { type foo:A; }

}

There is a simple fix, which has apparently been rejected
by the WG:

  * require a revision-stmt in each module or submodule
  * if no specific revision is requested, then an instance
    with the most recent revision date (earlier or equal
    to the current date) is selected.

The current solution, which is "it is implementation-specific"
is one of the ways to punt the standards issue, but the
problem doesn't go away.  It does not get detected
at YANG compile-time either.

In SMIv2, any module 'find' is going to be a deterministic
hit because every module version preserves all the previous
version identifiers, and the conformance section can also
be analyzed.  There is really only the 'latest version'
to deal with.  Not a problem in SMIv2.

In YANG, the previous versions are not identified, and it
is part of the architecture that agents and applications
MUST be capable of concurrently supporting multiple versions
of a module.  There is no conformance section to analyze either.

E.g.:

  module X {
    import Y { revision 2009-01-01; prefix y; }
    import Z { revision 2009-01-02; prefix z; }
    ...
  }

  module Y {
    import Z { revision 2008-11-22; prefix y; }
    ...
  }

YANG uses an 'N concurrent versions' approach.
In, YANG the problem is built into the architecture,
not a corner-case.  Picking randomly amongst N versions
is a new twist, much worse than the unlikely collision
of 2 vendors picking the same module name.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 21 Jan 2009 22:26:37 +0100 (CET)

> revision date, there is no import or include (fatal error).

Ok.

> properly select the correct module before -03, is what it

Yes.

> This problem has been in YANG all along.

I really do not understand the problem.  If module X imports Y w/o
specifying a revision, the result will be implementation dependent.
This is also how it works in SMIv2.  IF-MIB imports IANAifType, and
two agents that both implement the same version of IFMIB might have
different versions of the IANAifType.

The new thing in YANG is that this problem can be avoided by using
import by revision, and if no revision was used in the import,
the capability exchange and schema discovery can be used to find out
exactly what the agent implements.

> In YANG, the previous versions are not identified

I don't get this.  Can you elaborate why this is different in SMIv2
vs. YANG?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Jan 2009 13:41:58 -0800

> I don't get this.  Can you elaborate why this is different in SMIv2

I do not agree that SMIv2 has the same properties as YANG
because there is a special corner-case module called IANAifType
that is maintained online.  SMIv2 modules are not allowed
to have nested imports like YANG, so multiple concurrent
versions of the same definition are not possible.

SMIv2 does not allow a TEXTUAL-CONVENTION to be
defined from another one, and it does not have groupings.
AUGMENTS is also N:1, and SMIv2 is not allowed to change
in a way that would break an augmenting table,
so any number of versions can work together.
YANG has more features than SMIv2, which
lead to instabilities and corner-cases that
do not occur in SMIv2.

SMIv2 has a usage guidelines RFC, so standard MIB modules
follow much stricter rules.  I think the same thing is
needed for YANG -- now, not someday in the future.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 21 Jan 2009 22:49:57 +0100 (CET)

> versions of the same definition are not possible.

I thought the problem you brought up was how a particular module is
found?  We've already discussed the implications of import by
revision, and consensus is to have this.

> needed for YANG -- now, not someday in the future.

I agree.  Probably in a separate document though.  We don't really
need more pages in the yang doc...

/martin

---

From: Andy Bierman <[email-masked]>
Date: Wed, 21 Jan 2009 14:18:37 -0800

> revision, and consensus is to have this.

The one problem still left is the random nature
of an import statement without a revision.

I remember the interim meeting differently than you,
but there were 1000 details in 3 days so that is not surprising.
I remember Lada was very concerned about the import
revision being optional because of this very issue.
I was concerned as well, but I was under the impression
that if not specified, the most recent would be used
instead.

So the change I am requesting is that if module
revisions are available within the imported
module or included submodule, then the YANG tool
MUST use the newest version 'available' to the tool.
(That part is always going to be implementation-specific.)

If the revision-stmt is not used in the imported
module or included module, then it is implementation
dependent as to which 'version' is used.
By definition, such a module does not have a version,
so there cannot be any version match.

> need more pages in the yang doc...

agreed

Andy

---

From: WashamFan <[email-masked]>
Date: Thu, 22 Jan 2009 10:20:10 +0800

In sec 5.1, para 5, is a bullet missing? 
You enumerate module 'import' module, module 'include' submodule, submodule 'include' submodule, but what about submodule 'import' module? 
Am I missing something?

washam

---

From: WashamFan <[email-masked]>
Date: Thu, 22 Jan 2009 10:31:19 +0800

In sec 5.1.1, last para says:
When the author of "a" publishes a new revision, the changes may not
be acceptable to the author of "b".  If the new revision is
acceptable, the author of "a" can republish with an updated revision
in the import statement.

The last occurrence of "a" should be "b", is that right?

washam
>  You enumerate module 'import' module, module 'include' submodule, 
> submodule 'include' submodule, but what about submodule 'import' 
> module? 
>  Am I missing something?
>  
>  washam

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 22 Jan 2009 07:52:23 +0100 (CET)

> You enumerate module 'import' module, module 'include' submodule, submodule
> 'include' submodule, but what about submodule 'import' module?
> Am I missing something?

No, you're right.  I modified the first bullet to

- For a module or submodule to reference definitions in an external
  module, the external module MUST be imported.

Thanks.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 22 Jan 2009 07:53:05 +0100 (CET)

> The last occurrence of "a" should be "b", is that right?

Yes.  Now fixed.  Thanks!

/martin