Subject: [netmod] YANG 1.1 Key Flexibility

From: "Peyman Owladi -X (powladi - Ensoft Ltd at Cisco)" <[email-masked]>
Date: Wed, 7 May 2014 20:16:19 +0000

This is related to the restrictions on how keys can be specified for lists in YANG 1.0. I've listed two examples we've run up against where we hit limitations, and the extensions that could be made to support these use cases (which have some overlap).

Issue 1:

The elements of a subcontainer cannot be used as part of a key. In particular, take the case where a container with two elements is used as a data element in some cases, but needs to be used as a key in others.

grouping lacp-system-id-group {

  container lacp-system-id {

    leaf id  { type uint16; mandatory true; }

    leaf mac { type yang:mac-address; mandatory true; }

  }

}

This can be done in Yang 1.0 is to re-define the contents of the grouping directly in the list (or to use a grouping-within-the grouping).

list mc-lag-rgs {

  key "system-id system-mac";

 leaf id  { type uint16; mandatory true; }

  leaf mac { type yang:mac-address; mandatory true; }

  [...]

}

Issue 2:

A list entry may have a key which is a union; i.e. it is indexed by a single value which is one of multiple types. However, a YANG union is not always sufficient to model a choice between a set of types.

Take the following case for example:

choice maintenance-domain-identifier {

  leaf host { type inet:host; }

  leaf id   { type hex-string; }

  leaf name { type string; }

}

These strings can be overlapping, so this cannot be represented as a union.

There doesn't seem to be a way to use this as a key in Yang 1.0, without losing some typing information or inventing explicit invalid values (although Y09 could help with this to some extent -- see below).

Solution A:

Support keys within subcontainers. This brings the "key" statement in line with the "unique" statement, but doesn't necessarily mean there's any need to change the properties that are required of keys (e.g. they could still be mandatory within the list; though if Y09 is accepted this would be more flexible -- see Solution C).

This solve the example in "Issue 1", which becomes:

list mc-lag-rgs {

  key "lacp-system-id/id lacp-system-id/mac";

  uses lacp-system-id-group;

  [...]

}

Solution B:

Support non-leaf nodes (in particular container/choice) as keys. The complexity of the key type could be restricted (e.g. requiring that all leaves (or all leaves in all case statements) are mandatory, and that there there is only one level of hierarchy below the node selected as a key), or it could simply be taken as shorthand for Solution A for all the leaves (potentially in conjunction with Y09 to allow non-mandatory leaves).

Supporting the container as a key would solve issue 1, and supporting the choice as a key would solve issue 2.

Relationship with Y09:

Y09<http://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/issues.html#sec-9> tracks support for non-mandatory keys -- this has particular uses in combination with Solution A above, as it allows more flexibility in the leaves that can be used as part of the key.

For example, Issue 2 can be modelled as follows using just Y09 if the leaves are made entries directly in the list:

list maintenance-domains {

  key "host id name";

  leaf host { type inet:host;  mandatory false; }

  leaf id   { type hex-string; mandatory false; }

  leaf name { type string;     mandatory false; }

  // Rule to ensure exactly one of the above is specified

  must "count(host) + count(id) + count(name) = 1";

}

However, in conjunction with solution A, instead this could be modelled as follows (which allows the choice statement to be re-used):

list maintenance-domains {

  key "maintenance-domain-identifier/host/host "

    + "maintenance-domain-identifier/id/id "

    + "maintenance-domain-identifier/string/string";

  choice maintenance-domain-identifier {

    mandatory true;

    leaf host { type inet:host; }

    leaf id   { type hex-string; }

    leaf name { type string; }

  }

}

Further notes on Y09:

-    Is there a reason that even a single key has to be mandatory, rather than considering its absence to be an extension of its value space?

-    Presumably keys should be mandatory by default for compatibility with YANG 1.0? (i.e. using "mandatory false;" to override.)

-    The restriction on having an "empty" leaf as part of a key should presumably also be lifted if Y09 is accepted?

---

From: Andy Bierman <[email-masked]>
Date: Wed, 7 May 2014 17:36:25 -0700

Hi,

These topics were debated at length during the development of RFC 6020.
It was determined that (a) not only should keys be simple leafs that are
children of the list, but (b) the key leafs must be encoded on the wire
before any other child nodes, in order to improve server performance.

Nested leafs, containers full of whatever as keys, and variable numbers
of keys via choice-stmt would make YANG too complicated to be implemented
efficiently in servers.  N/Y is already very heavy-weight to implement.
This would make it much worse.

Andy

On Wed, May 7, 2014 at 1:16 PM, Peyman Owladi -X (powladi - Ensoft Ltd at
Cisco) <[email-masked]> wrote:

>  This is related to the restrictions on how keys can be specified for
> lists in YANG 1.0. I've listed two examples we've run up against where we
> hit limitations, and the extensions that could be made to support these use
> cases (which have some overlap).
>
>
>
> *Issue 1:*
>
>
>
> The elements of a subcontainer cannot be used as part of a key. In
> particular, take the case where a container with two elements is used as a
> data element in some cases, but needs to be used as a key in others.
>
>
>
> This can be done in Yang 1.0 is to re-define the contents of the grouping
> directly in the list (or to use a grouping-within-the grouping).
>
>
>
> list mc-lag-rgs {
>
> *Issue 2:*
> A list entry may have a key which is a union; i.e. it is indexed by a
> single value which is one of multiple types. However, a YANG union is not
> always sufficient to model a choice between a set of types.
>
>
>
> There doesn't seem to be a way to use this as a key in Yang 1.0, without
> losing some typing information or inventing explicit invalid values
> (although Y09 could help with this to some extent -- see below).
>
>
>
> *Solution A:*
>
>
>
> Support keys within subcontainers. This brings the "key" statement in line
> with the "unique" statement, but doesn't necessarily mean there's any need
> to change the properties that are required of keys (e.g. they could still
> be mandatory within the list; though if Y09 is accepted this would be more
> flexible -- see Solution C).
>
>
>
> *Solution B:*
> Support non-leaf nodes (in particular container/choice) as keys. The
> complexity of the key type could be restricted (e.g. requiring that all
> leaves (or all leaves in all case statements) are mandatory, and that there
> there is only one level of hierarchy below the node selected as a key), or
> it could simply be taken as shorthand for Solution A for all the leaves
> (potentially in conjunction with Y09 to allow non-mandatory leaves).
>
>
>
> *Relationship with Y09:*
> Y09 <http://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/issues.html#sec-9>tracks support for non-mandatory keys -- this has particular uses in
> combination with Solution A above, as it allows more flexibility in the
> leaves that can be used as part of the key.
>
>
>
> For example, Issue 2 can be modelled as follows using just Y09 if the
> leaves are made entries directly in the list:
>
>
>
> However, in conjunction with solution A, instead this could be modelled as
> follows (which allows the choice statement to be re-used):
>
>
>
> *Further notes on Y09:*
> -    Is there a reason that even a single key has to be mandatory, rather
> than considering its absence to be an extension of its value space?
>
> -    Presumably keys should be mandatory by default for compatibility
> with YANG 1.0? (i.e. using "mandatory false;" to override.)
>
> -    The restriction on having an "empty" leaf as part of a key should
> presumably also be lifted if Y09 is accepted?
>
>
>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>
>

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 08 May 2014 12:00:31 +0200 (CEST)

Hi,

Added as Y53.

/martin

"Peyman Owladi -X (powladi - Ensoft Ltd at Cisco)" <[email-masked]> wrote:
> lists in YANG 1.0. I've listed two examples we've run up against where
> we hit limitations, and the extensions that could be made to support
> these use cases (which have some overlap).
> 
> 
> 
> Issue 1:
> 
> 
> 
> particular, take the case where a container with two elements is used
> as a data element in some cases, but needs to be used as a key in
> others.
> 
> 
> 
> This can be done in Yang 1.0 is to re-define the contents of the
> grouping directly in the list (or to use a grouping-within-the
> grouping).
> 
> 
> 
> list mc-lag-rgs {
> 
> single value which is one of multiple types. However, a YANG union is
> not always sufficient to model a choice between a set of types.
> 
> 
> 
> There doesn't seem to be a way to use this as a key in Yang 1.0,
> without losing some typing information or inventing explicit invalid
> values (although Y09 could help with this to some extent -- see
> below).
> 
> 
> 
> Solution A:
> 
> 
> 
> Support keys within subcontainers. This brings the "key" statement in
> line with the "unique" statement, but doesn't necessarily mean there's
> any need to change the properties that are required of keys (e.g. they
> could still be mandatory within the list; though if Y09 is accepted
> this would be more flexible -- see Solution C).
> 
> 
> 
> complexity of the key type could be restricted (e.g. requiring that
> all leaves (or all leaves in all case statements) are mandatory, and
> that there there is only one level of hierarchy below the node
> selected as a key), or it could simply be taken as shorthand for
> Solution A for all the leaves (potentially in conjunction with Y09 to
> allow non-mandatory leaves).
> 
> 
> 
> Y09<http://svn.tools.ietf.org/svn/wg/netmod/yang-1.1/issues.html#sec-9>
> tracks support for non-mandatory keys -- this has particular uses in
> combination with Solution A above, as it allows more flexibility in
> the leaves that can be used as part of the key.
> 
> 
> 
> However, in conjunction with solution A, instead this could be
> modelled as follows (which allows the choice statement to be re-used):
> 
> 
> 
> -    than considering its absence to be an extension of its value space?
> 
> -    Presumably keys should be mandatory by default for compatibility with
> -    YANG 1.0? (i.e. using "mandatory false;" to override.)
> 
> -    presumably also be lifted if Y09 is accepted?
> 
>

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 08 May 2014 12:01:43 +0200 (CEST)

> This would make it much worse.

I fully agree.

/martin

---

From: "Peyman Owladi -X (powladi - Ensoft Ltd at Cisco)" <[email-masked]>
Date: Thu, 8 May 2014 13:49:34 +0000

> -----Original Message-----
> From: Martin Bjorklund [mailto:[email-masked]]
> Sent: 08 May 2014 11:02
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: Y53: YANG 1.1 Key Flexibility
> 

Fair enough; thanks for the explanation.

It would still be useful to get some thoughts on what the solution should be for issue 2 in the original mail (copied here for reference):
======
A list entry may have a key which is a union; i.e. it is indexed by a single value which is one of multiple types. However, a YANG union is not always sufficient to model a choice between a set of types.

======

Do you have thoughts on how this should be modelled (and if extensions are needed)?

In particular, it seems there is a solution using Y09-01, though to do this we effectively have to reinvent the choice/union using a "must" statement (and I'm not sure whether this is even compatible with the "CLI" comment in Y09-01):
======

======

Is that the best approach, or is there a better answer? (For example, introducing a union type with a discriminator would perhaps work as it would allow the above to be modelled as a union and therefore used as a key.)

Thanks,
Peyman

---

From: Andy Bierman <[email-masked]>
Date: Thu, 8 May 2014 07:04:43 -0700

On Thu, May 8, 2014 at 6:49 AM, Peyman Owladi -X (powladi - Ensoft Ltd at
Cisco) <[email-masked]> wrote:

> It would still be useful to get some thoughts on what the solution should
> be for issue 2 in the original mail (copied here for reference):
> ======

typedef maintenance-domain-identifier {
  type union {
    type inet:host;
    type hex-string;
    type string;
  }
}

This is legal YANG.
Figuring out which type-stmt matched is an implementation detail.

Andy

> In particular, it seems there is a solution using Y09-01, though to do
> this we effectively have to reinvent the choice/union using a "must"
> statement (and I'm not sure whether this is even compatible with the "CLI"
> comment in Y09-01):
> ======
> Is that the best approach, or is there a better answer? (For example,
> introducing a union type with a discriminator would perhaps work as it
> would allow the above to be modelled as a union and therefore used as a
> key.)
>
> Thanks,
> Peyman
>
>

---

From: "Peyman Owladi -X (powladi - Ensoft Ltd at Cisco)" <[email-masked]>
Date: Thu, 8 May 2014 15:12:29 +0000

From: Andy Bierman [mailto:[email-masked]]
Sent: 08 May 2014 15:05
To: Peyman Owladi -X (powladi - Ensoft Ltd at Cisco)
Cc: Martin Bjorklund; [email-masked]
Subject: Re: Y53: YANG 1.1 Key Flexibility

On Thu, May 8, 2014 at 6:49 AM, Peyman Owladi -X (powladi - Ensoft Ltd at Cisco) <[email-masked]<mailto:[email-masked]>> wrote:
> From: Martin Bjorklund [mailto:[email-masked]<mailto:[email-masked]>]
> Sent: 08 May 2014 11:02
> To: [email-masked]<mailto:[email-masked]>
> Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
> Andy Bierman <[email-masked]<mailto:[email-masked]>> wrote:
> > Hi,
> >

This is legal YANG.

Andy

To check we're on the same page, my understanding is that in that union, if a specifier matches the domain-name pattern, it would always resolve to the inet:host type (based on RFC 6020 9.2: "When a string representing a union data type is validated, the string is validated against each member type, in the order they are specified in the "type" statement, until a match is found.") -- though I could be misunderstanding what "validated" means here.

In any case, this question arises because there can be overlap in the value space of the different elements; so as well as the value itself, we need the information on which of the options is being represented to interpret the data correctly.

Based on that, it looks like the data needs to be represented as a choice or as independent leaves rather than a union, right?

Thanks,

Peyman

Thanks,
Peyman

---

From: Andy Bierman <[email-masked]>
Date: Thu, 8 May 2014 08:33:21 -0700

On Thu, May 8, 2014 at 8:12 AM, Peyman Owladi -X (powladi - Ensoft Ltd at
Cisco) <[email-masked]> wrote:

>
>
> *From:* Andy Bierman [mailto:[email-masked]]
> *Sent:* 08 May 2014 15:05
> *To:* Peyman Owladi -X (powladi - Ensoft Ltd at Cisco)
> *Cc:* Martin Bjorklund; [email-masked]
> *Subject:* Re: Y53: YANG 1.1 Key Flexibility
>
>
>
>
>
>
>
> In any case, this question arises because there can be overlap in the
> value space of the different elements; so as well as the value itself, we
> need the information on which of the options is being represented to
> interpret the data correctly.
>

A union type is like a logical-OR expression.
The YANG validation tool goes through each 'type' in order and checks
if the supplied value is valid for that type.  If so, then the leaf is
valid.
If the parser gets to end of the list and none of the types have validated
for the supplied value, then the leaf is rejected with an invalid-value
error-tag.

So you want to place the most selective patterns first, and the catch-all
'string' last.
It is an implementation detail how the union type is represented within the
server.

   leaf maintenance-domain {
       type maintenance-domain-identifier;
   }

>
> Based on that, it looks like the data needs to be represented as a choice
> or as independent leaves rather than a union, right?
>
>
>
>
No

Andy

---

From: "Peyman Owladi -X (powladi - Ensoft Ltd at Cisco)" <[email-masked]>
Date: Fri, 9 May 2014 14:01:41 +0000

[...snip...]

A union type is like a logical-OR expression.
The YANG validation tool goes through each 'type' in order and checks
if the supplied value is valid for that type.  If so, then the leaf is valid.
If the parser gets to end of the list and none of the types have validated
for the supplied value, then the leaf is rejected with an invalid-value error-tag.

So you want to place the most selective patterns first, and the catch-all 'string' last.
It is an implementation detail how the union type is represented within the server.

Even taking into account those clarifications, a union is still not sufficient for this scenario.  In particular, it's possible for "example.com" to be either a domain name or a string, and it represents a distinct identifier in each case. i.e.  It's possible to have two different domains, one identified by string "example.com" and one identified by domain name "example.com".

So the "choice/case" construct (or the Y09-based construct below) allows us to distinguish between these two cases, whereas the "union" construct does not.

If I've understood your comments about the type inference correctly (i.e. that there is no defined pattern for it), then there is of course an alternative here to have a separate enumeration that indicates the type being specified (though in that case I'm not sure there's a whole lot of value in the union any more):

  leaf identifier-type {
    type enumeration {
      enum host;
      enum id;
      enum name;
    }
  }
  leaf identifier { type string; mandatory false; }

But if that's an expected usage pattern, then perhaps having an explicit "discriminated union" type would have value, by allowing the enumeration values and corresponding types to be linked programmatically? (Again, this would be particularly add value when defining mappings from Yang to binary representations.)

  leaf identifier {
    type discriminated-union {
      option host { type inet:host; }
      option id   { type hex-string; }
      option name { type string; }
    }
  }

Cheers,
Peyman

No

Thanks,

Peyman

Andy

Thanks,
Peyman

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 May 2014 07:17:25 -0700

On Fri, May 9, 2014 at 7:01 AM, Peyman Owladi -X (powladi - Ensoft Ltd at
Cisco) <[email-masked]> wrote:

> Even taking into account those clarifications, a union is still not
> sufficient for this scenario.  In particular, it's possible for "
> example.com" to be either a domain name or a string, and it represents a
> distinct identifier in each case. i.e.  It's possible to have two different
> domains, one identified by string "example.com" and one identified by
> domain name "example.com".
>
>
>
> So the "choice/case" construct (or the Y09-based construct below) allows
> us to distinguish between these two cases, whereas the "union" construct
> does not.
>
>
>
> If I've understood your comments about the type inference correctly (i.e.
> that there is no defined pattern for it), then there is of course an
> alternative here to have a separate enumeration that indicates the type
> being specified (though in that case I'm not sure there's a whole lot of
> value in the union any more):
>
>
>
>   leaf identifier-type {
>
>     type enumeration {
>
>       enum host;
>
>       enum id;
>
>       enum name;
>
>     }
>
>   }
>
>  But if that's an expected usage pattern, then perhaps having an explicit
> "discriminated union" type would have value, by allowing the enumeration
> values and corresponding types to be linked programmatically? (Again, this
> would be particularly add value when defining mappings from Yang to binary
> representations.)
>
>
>
>   leaf identifier {
>

This does not work because the leaf local-name is "identifier".
That is not a virtual node that can be changed to  "host", "id", or "name"
in an instance document,

You can always add the discriminator leaf (e.g. identifier-type) to the
data model.
That is how YANG solves this problem now.

 Cheers,

Andy