Subject: Re: [netmod] include

From: Jernej Tuljak <[email-masked]>
Date: Fri, 29 Mar 2013 16:26:32 +0100

Dne 22.2.2013 18:12, pis(e Ladislav Lhotka:
> On Feb 22, 2013, at 6:03 PM, Andy Bierman <[email-masked]> wrote:
>
>>
>> On Fri, Feb 22, 2013 at 8:50 AM, Ladislav Lhotka <[email-masked]> wrote:
>>
>> On Feb 22, 2013, at 5:33 PM, Andy Bierman <[email-masked]> wrote:
>>
>>>
>>> On Fri, Feb 22, 2013 at 8:22 AM, Martin Bjorklund <[email-masked]> wrote:
>>> Ladislav Lhotka <[email-masked]> wrote:
>>>> On Feb 22, 2013, at 4:19 PM, Martin Bjorklund <[email-masked]> wrote:
>>>>> But what about collisions?  Compare w/ #ifndef.
>>>> With transitive inclusions, collisions have to be avoided in any
>>>> case.
>>> Exactly.  But with the current text, collisions are avoided, since it
>>> is only the module that "incorporates" the contents of the
>>> submodules.
>>>
>>>
>>>
>>> IMO, for export purposes, submodules are conceptually flat -- they all contribute
>>> to the module namespace.  The compiler has to detect any naming collisions for the
>>> entire module namespace.
>>>
>>> With your interpretation, how would the following corner-case be handled?
>>>
>>> module A {
>>>    include B;
>>>    typedef X { type string; }
>>> }
>>>
>>> submodule B {
>>>    include C;
>>>    leaf a { type a:X; }
>>> }
>>>
>>> submodule C {
>>>    typedef X { type int32; }
>>> }
>>>
>> With scenario (3), leaf "a" would be int32, and for a module importing A, the X type would be an alias for string. With scenario (1), it is an error because B includes C but A doesn't.
>>
>>
>> IMO, any compiler should report an error because it has to parse module A
>> in order to get to submodule C.  It must include B and then C and then
>> must parse the duplicate typedef.
>>
>> But my example seems to be legal YANG if submodules are treated as separate namespaces.
>> I don't think there is any text that says A MUST include C.
>> If it does there will be a conflict, but if C is only visible to B,
>> and the only typedef X visible to B is int32, the compiler must accept it,
>> and the typedef "X" has a hidden different version within the module.
> Yes, and that is, I think, the essence of (3). It can be useful, for instance, if one wants to share some definitions among multiple submodules but doesn't want to expose them to external modules. This is actually how I stumbled upon this whole issue.

Sorry for reheating this up, but no matter how many times I read this 
entire discussion, I still have no idea what the proper way of 
implementing this is. Will there be modifications to RFC6020 text or not?

I don't think that the current text supports option (3) the way Andy and 
Ladislav describe it at all. I don't like the "what's visible to 
external modules" part when a module that consists of submodules is 
imported in another module. The text for import statement says:

When a module is imported, the importing module may:
    o  use any grouping and typedef defined at the top level in the
       imported module or*its submodules*.

    o  use any extension, feature, and identity defined in the imported
       module or*its submodules*.

    o  use any node in the imported module's schema tree in "must",
       "path", and "when" statements, or as the target node in "augment"
       and "deviation" statements.

This seems to suggest that the importing module sees MORE than may be 
visible to the imported module which starts the submodule inclusion 
chain. That is: the importing module sees both "X" typedefs, but the 
including module only sees the string version --> instant ambiguity upon 
importing. Isn't this just utterly broken? (2) would at least detect the 
error, which is correct IMO.

The above text should say "in the imported module or *it's included 
submodules*" for this to work as described by Andy and Ladislav (?). 
Should be, since you decided that there are now two types of submodules 
- the ones included by the main module and the ones that are not - and 
this text refers to both of them (and any other type of submodule you 
can possibly think of, like dangling bugged ones - which aren't that 
hard to create - think multiple submodule revisions because someone 
forgot to clean up :D).

I think this text makes hiding top level definitions impossible (except 
schema trees defined in submodules not included by main module, but why 
would you want to hide that?).

Jernej

> Lada
>
>>   
>> Lada
>>
>>
>> Andy
>>   
>>
>>> In Yuma this would be a duplicate name error for 'X'.
>>> Is that correct or is leaf a an int32 and no problem?
>>> (Looks like a problem to me.)
>>>
>>>
>>>
>>> /martin
>>>
>>> Andy
>>>
>> --
>> Ladislav Lhotka, CZ.NIC Labs
>> PGP Key ID: E74E8C0C
>>
>>
>>
>>
>>
> --
> Ladislav Lhotka, CZ.NIC Labs
> PGP Key ID: E74E8C0C
>
>
>
>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Fri, 29 Mar 2013 09:41:22 -0700

On Fri, Mar 29, 2013 at 8:26 AM, Jernej Tuljak <[email-masked]> wrote:
...
>       imported module or its submodules.
>
>       module or its submodules.
>
> visible to the imported module which starts the submodule inclusion chain.
> That is: the importing module sees both "X" typedefs, but the including
> module only sees the string version --> instant ambiguity upon importing.
> Isn't this just utterly broken? (2) would at least detect the error, which
> is correct IMO.

IMO the above text is correct.

The include-stmt text not shown is related to the task of resolving
prefixes when they are used within a sub-module, not related to
exporting symbols out of the submodule.

The above text says to me that all top-level symbols from the
main module and all its submodules are available to the importing
module.

>
> The above text should say "in the imported module or it's included
> submodules" for this to work as described by Andy and Ladislav (?). Should
> be, since you decided that there are now two types of submodules - the ones
> included by the main module and the ones that are not - and this text refers
> to both of them (and any other type of submodule you can possibly think of,
> like dangling bugged ones - which aren't that hard to create - think
> multiple submodule revisions because someone forgot to clean up :D).
>

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 30 Mar 2013 11:24:32 +0100

Hi,

for the time being, an easy fix seems to be to require that all submodules be explicitly included from the main module. This may even qualify as an erratum.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sat, 30 Mar 2013 07:54:44 -0700

> for the time being, an easy fix seems to be to require that all submodules be explicitly included from the main module. This may even qualify as an erratum.

I don't agree tat any fix is needed.
I don't agree include-stmt and import-stmt are coupled in this way.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 30 Mar 2013 19:03:02 +0100

> I don't agree include-stmt and import-stmt are coupled in this way.

Coupled? So what's your definition of "its submodules" that appears in several places, in particular sec. 7.1.5?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sat, 30 Mar 2013 11:25:56 -0700

> Coupled? So what's your definition of "its submodules" that appears in several places, in particular sec. 7.1.5?

The "belongs-to" statement defines "its submodules":

7.1.6.  The include Statement

   [para 1].....  Modules are only allowed to
   include submodules that belong to that module, as defined by the
   "belongs-to" statement (see Section 7.2.2).

7.2.2.  The belongs-to Statement

   The "belongs-to" statement specifies the module to which the
   submodule belongs.  The argument is an identifier that is the name of
   the module.

There is only one XML namespace for the entire module.
QNames within that namespace must be unique.
There is no namespace per submodule in YANG.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 31 Mar 2013 10:17:05 +0200

> The "belongs-to" statement defines "its submodules":

So somebody writes a submodule with 
   
    belongs-to ietf-ip { prefix ip; }

and it becomes a submodule of "ietf-ip" without ever being included anywhere?

This can't be, so something must be missing in this definition of "its modules".

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 31 Mar 2013 06:13:51 -0700

> This can't be, so something must be missing in this definition of "its modules".

The server supports and advertises a specific set of modules.
The netconf-state schema list can identify the submodules.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sun, 31 Mar 2013 06:30:58 -0700

> This can't be, so something must be missing in this definition of "its modules".

You seem to keep ignoring the fact that a module has 1 namespace
and there cannot be multiple symbols resolving to the same QName.
I can't find any text that says a single module revision can export
multiple versions
of the same QName to another module.

The include-stmt just says how to resolve prefixes within the module
that is being parsed. The import-stmt says how to resolve prefixes
from other modules.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 31 Mar 2013 19:16:48 +0200

> The netconf-state schema list can identify the submodules.

Huh, are you saying that YANG-based models cannot be used without supporting ietf-netconf-monitoring?

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 31 Mar 2013 19:38:52 +0200

> of the same QName to another module.

It is not about namespaces or QNames, just about figuring out the (complete) schema, because submodules may contribute to it. I strongly disagree with involving netconf-state schema, the client must be able to learn the schema from hello and the text of YANG modules and submodules it already has. For one, how could the client learn whether there are any submodules of ietf-netconf-monitoring?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 31 Mar 2013 10:43:18 -0700

> Huh, are you saying that YANG-based models cannot be used without supporting ietf-netconf-monitoring?

Any protocol that uses YANG has to let each peer know the YANG
module implementation details of the other peer in order to automatically
derive the exact set of YANG files used.

There is only 1 module namespace exported to other modules.

Consider this simple example that shows why:

module A {
   ...
   import B { prefix b; }

   augment /b:top {
      leaf a { type b:b-type; }
   }
}

module B {
  include B1;
  include B2;
}

module B1 {
  belongs-to B { prefix b; }
  include B1-1;
}

module B1-1 {
  belongs-to B { prefix b; }

  list top {
     key i;
     leaf i { type int8; }
  }
  typedef b-type { type string; }
}

module B2 {
  belongs-to B { prefix b; }
  include B2-1;
}

module B2-1 {
  belongs-to B { prefix b; }

  container top;
  typedef b-type { type int32; }
}

Is this legal YANG according to your interpretation?
IMO, no because this would cause duplicate QNames in
the module B namespace.

Nothing in the syntax in module A violates any text related to
importing symbols.

So which object is 'top'? A container or a list?
Which type is 'b-type'? A string or an int32?

IMO the correct "bug fix" is to clarify that top-level symbols in the same
namespace (typedefs, groupings, nodes, etc.) MUST NOT be defined
more than once in any module, even if submodules are used.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sun, 31 Mar 2013 11:08:51 -0700

> It is not about namespaces or QNames, just about figuring out the (complete) schema, because submodules may contribute to it. I strongly disagree with involving netconf-state schema, the client mus…

The fact is, you cannot rely the information in the YANG modules to
determine the set of files that were used to construct the schema tree
in use by the server.

1)  module namspaces are assumed to be globally unique, not module names.
     Import "foo" does not reliably identify anything. The <capability> URI
    for the YANG module is needed to map namespace to module-name.

 2) Since submodules are not advertised as YANG module capabilities,
     the <hello> cannot be used to map sub-module names to the
     namespace-stmt value.

 3) Since import-by-revision and include-by-revision are optional to use,
     the import or include statements cannot be used to reliably determine
     which revision of the specified module or submodule to use.
     The <capability> URI can usually be used to resolve import, but
     nothing except the 'schema' list contains this information for submodules.
     (Making them mandatory to use would illuminate how fragile they are.)

Andy