Subject: [netmod] SMIv2 to YANG mapping: OBJECT-IDENTITY

From: David Reid <[email-masked]>
Date: Wed, 01 Apr 2009 14:10:36 -0400

I'd like to start some discussion on SMIv2 to YANG mapping. Several of 
us have implementations which are very similar, but not exactly the 
same. There are also several unresolved issues in my implementation 
(and probably everyone else). Juergen published a document on the subject 
(draft-schoenw-netmod-smi-yang-00.txt), which is a good place to start 
the discussion. The first topic in the Open Issues section of Juergen's 
document is:

>   o  Need to add support for the translation of OBJECT-IDENTITY macro
>      invocations to YANG identity statements.

The biggest problem I'm having with OBJECT-IDENTITY is knowing when to map
a SYNTAX clause of a MIB object with a SYNTAX of OBJECT IDENTIFIER to
a YANG identityref.

For example, RFC 3413 defines:   

    snmpTargetAddrTDomain OBJECT-TYPE
       SYNTAX      TDomain  <--- note: this resolves to an OBJECT IDENTIFIER
       MAX-ACCESS  read-create
       STATUS      current
       DESCRIPTION
           "This object indicates the transport type of the address
            contained in the snmpTargetAddrTAddress object."
       ::= { snmpTargetAddrEntry 2 }

Possible values for snmpTargetAddrTDomain are things like snmpUDPDomain from
RFC 3417:

   snmpUDPDomain  OBJECT-IDENTITY
       STATUS     current
       DESCRIPTION
               "The SNMP over UDP over IPv4 transport domain.
               The corresponding transport address is of type
               SnmpUDPAddress."
       ::= { snmpDomains 1 }

I want to map snmpUDPDomain to a YANG identity and map snmpTargetAddrTDomain
to a YANG leaf with type identityref. In this case, I could treat the TDomain
textual convention special, but in general, the syntax could have been
OBJECT IDENTIFIER or a vendor defined textual convention. So I don't know when
to map OBJECT IDENTIFIER to identityref and when to map it to something else,
like object-identifier. Of course, if I read the description clause I can 
figure it out. But I want my tool to work automatically.

-David Reid

---

From: Andy Bierman <[email-masked]>
Date: Wed, 01 Apr 2009 13:00:18 -0700

>>      invocations to YANG identity statements.

Translate type OBJECT IDENTIFIER and all the possible registration
OIDs as identities.  It is not exact, because SMIv2 OID is not
restricted like YANG identityref.

But you cannot automatically determine the 'base' can you?
So you are stuck making one base for all SMIv2 OIDs, e.g.:

typedef RegistrationOid {
    type identityref { base SmiV2OidBase; }
}

identity SmiV2OidBase {
    description "Base for all SMIv2 OBJECT-IDENTITY definitions.";
}

identity snmpUDPDomain {
    base SmiV2OidBase;
}

leaf snmpTargetAddrTDomain {
    type RegistrationOid;
}

Andy

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
> 
>

---

From: David Reid <[email-masked]>
Date: Wed, 01 Apr 2009 16:17:09 -0400

> leaf snmpTargetAddrTDomain {

How do I know that snmpTargetAddrTDomain should be of type RegistrationOid,
but snmpNotifyFilterSubtree (from the same module, RFC 3413), which also has
SYNTAX OBJECT IDENTIFIER, should be a different type? It is clear from the
description clause, but how does my tool know.

-David Reid

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 1 Apr 2009 23:42:38 +0200

> a SYNTAX clause of a MIB object with a SYNTAX of OBJECT IDENTIFIER to

There is likely no good answer since the SMIv2 does not distinguish
how OBJECT IDENTIFIERs are used. We could perhaps special case some
well known TCs but a generic solution that does the right thing on all
cases likely does not exist.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: David Reid <[email-masked]>
Date: Fri, 03 Apr 2009 17:22:08 -0400

> how OBJECT IDENTIFIERs are used. 

I was afraid of that. I can't come up with a good answer. Here is a 
possibility: use a yang extension to carry the SMIv2 OID information
from the SMIv2 OBJECT-IDENTITY to the YANG identity and then just convert
the SMIv2 OBJECT IDENTIFIER SYNTAX to a YANG object-identifier type in 
all cases.

identity snmpUDPDomain {
    smiv2:oid "1.3.6.1.6.1.1";
    base "smiv2-global-base";
    description "The SNMP over UDP over IPv4 transport...";
}   
    
leaf snmpTargetAddrTDomain {
    type object-identifier;
    config true;
    description "This object indicates the transport type of the address...";
}   

That doesn't seem very YANG-like, but I think it could work.

-David Reid

---

From: David Spakes <[email-masked]>
Date: Wed, 8 Apr 2009 17:15:26 -0400 (EDT)

It hasn't been that long ago that the working group added identityref
to the Yang spec.

While others were working on identityref, I was trying to figure out how
to map conformance statements from the SNMP world to the Yang world.  As a
result of that work, I began to believe that Yang really needed to have a
more generic ref type, but other duties as assigned took my attention away
from the matter.

Now, as I read the posts about mapping SNMP's OBJECT-IDENTITY/ OBJECT
IDENTIFIER to Yang's identity/identityref, I have the same thought that I
did before about the need for a more generic ref type.

We started out with a keyref, which was too restrictive.  Andy proposed
the leafref in July, 2008.

The leafref was too restrictive to handle OBJECT-IDENTITY mapping, so
identityref was added.

What I propose is that we drop identityref and leafref and replace them
with a generic 'ref' that can point to:

  - leafs
  - identities
  - lists (and other constructs)
  - modules
  - groupings
  - containers
  - notifications

Under this proposal, a MIB object of type OBJECT IDENTIFIER would always
be converted to a leaf having a type that is the generic ref type.

If the working group were to choose to adopt this proposal, I don't see
any reason why Yang would need to define the built-in object-identifier
and object-identifier-128 types.

Regards,

David Spakes

On Tue, 8 Jul 2008 [email-masked] wrote:

> I have some concerns about the keyref builtin data type.
>
> 8.8.  The keyref Built-in Type
> ------------------------------
>
>     The keyref type is used to reference a particular list entry in the
>     data tree.  Its value is constrained to be the same as the key of an
>     existing list entry.

...

> It is also too restrictive to limit a keyref to a leaf used as a key.
> A new list can be created with key leafs which happen to already be
> defined as non-key leafs in other data structure (does not have to
> be in a list).  Perhaps a separate 'leafref' would be better for this
> purpose.  IMO, this is also needed to define notification content
> properly.

-------------------------------------------------------------
 David Spakes                       email:   [email-masked]
 SNMP Research                      voice:   +1 865 573 1434
 3001 Kimberlin Heights Road          fax:   +1 865 573 9197
 Knoxville, TN  37920-9716  USA      http://www.snmp.com
-------------------------------------------------------------

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 08 Apr 2009 23:57:34 +0200 (CEST)

Hi David,

> be converted to a leaf having a type that is the generic ref type.

I must admit I don't quite understand how this would work.  Maybe you
could examplify your proposal by showing how a leaf/leafref,
identity/identityref and OID would be done with the current syntax,
and with your proposal?

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 9 Apr 2009 00:36:50 +0200

> and object-identifier-128 types.

An SMIv2 object of type OBJECT IDENTIFIER can hold arbitrary OID
values; restricting this to something that is defined somewhere won't
work. Please also note that there are other protocols using OIDs and
hence these types will be useful even if there is a way to eliminate
OIDs from translated SMIv2 modules (which I doubt will be possible).

/js

---

From: WashamFan <[email-masked]>
Date: Thu, 09 Apr 2009 14:48:24 +0800

Hi,

current interface definition:

     list interface {
         key "name";
         leaf name {
             type string;
         }
         leaf ifIndex {
             type uint32;
         }
         list address {
             key "ip";
             leaf ip {
                 type yang:ip-address;
             }
         }
     }

And corresponding default-address definition is:

     container default-address {
         leaf ifname {
             type leafref {
                 path "../../interface/name";
             }
         }
         leaf address {
             type leafref {
                 path "../../interface[name = current()/../ifname]"
                    + "/address/ip";
             }
         }
     }

If I changed the nested keys to ones placing in parallel:

     list interface {
         key "name" "ip";
         leaf name {
             type string;
         }
         leaf ifIndex {
             type uint32;
         }         
         leaf ip {
             type yang:ip-address;
         }
     }

Is the corresponding default-address defined as:

     container default-address {
         leaf ifname {
             type leafref {
                 path "../../interface/name";
             }
         }
         leaf address {
             type leafref {
                 path "../../interface[name = current()/../ifname]"
                    + "/ip";
             }
         }
     }

or

     container default-address {
         leaf ifname {
             type leafref {
                 path "../../interface/name";
             }
         }
         leaf address {
             type leafref {
                 path "../../interface/ip";
             }
         }
     }

Predicates are not used in the latter case, but I do not
think it is clear in current text that predicates MUST be 
used when refering a list entry if the list has multiplex keys.

PS:
line 5, para1, sec 9.10.2:
s/defined the module/defined in the module/

washam

---

From: WashamFan <[email-masked]>
Date: Thu, 09 Apr 2009 15:01:31 +0800

Hi,

given server:

     list server {
         key "name";
         unique "ip port";
         leaf name {
             type string;
         }
         leaf ip {
             type inet:ip-address;
         }
         leaf port {
             type inet:port-number;
         }
     }

The following configuration is valid:

     <server>
       <name>smtp</name>
       <ip>192.0.2.1</ip>
       <port>25</port>
     </server>

     <server>
       <name>http</name>
       <ip>192.0.2.1</ip>
     </server>

     <server>
       <name>ftp</name>
       <ip>192.0.2.1</ip>
     </server>

If I changed the server definition to:

     list server {
         key "name";
         unique "ip port";
         leaf name {
             type string;
         }
         leaf ip {
             type inet:ip-address;
         }
         leaf port {
             default 22;             <- set default
             type inet:port-number;
         }
     }

then, the same configuration should be invalid, right?

PS:
the title, sec 7.8.3
s/lists's/list's/

washam

---

From: Phil Shafer <[email-masked]>
Date: Thu, 9 Apr 2009 09:59:55 -0400

>then, the same configuration should be invalid, right?

This config should be invalid both with and without
the default.  A missing leaf should be considered
as part of uniqueness.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 09 Apr 2009 07:26:48 -0700

> the default.  A missing leaf should be considered

I don't think they are.
Incomplete tuples are skipped.
In this case, the default-stmt adds the <port> leaf,
but if it didn't, the rows without a complete
unique tuple get skipped in the test.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 09 Apr 2009 18:11:29 +0200 (CEST)

Hi,

> If I changed the nested keys to ones placing in parallel:
               ^^^^^^^^^^^

This should be "name ip".

>                  path "../../interface/ip";

Both are valid, but they mean different things.  The first one is the
one you're probably looking for.  It makes sure you give values for
one list entry instance.  The second one has to unrelated leafrefs,
each one points to one or many instances.

For example, suppose the db is:

   <interface>
      <name>eth0</name>   # 1.a
      <ip>10.0.0.1</ip>   # 1.b
   </interface>

   <interface>
      <name>eth0</name>   # 2.a
      <ip>10.0.0.2</ip>   # 2.b
   </interface>

   <interface>
      <name>eth1</name>   # 3.a
      <ip>10.0.0.2</ip>   # 3.b
   </interface>

   <default-address>
     <ifname>eth0</ifname>         # X
     <address>10.0.0.2</address>   # Y
   </default-address>

With your first alternative, the XPath expression for 'ifname' evaluates
to a nodeset with 1.a and 2.a, and the the XPath expression for
'address' evaluates to a nodeset with 2.b.

With your second alternative, the XPath expression for 'ifname' evaluates
to a nodeset with 1.a and 2.a, and the the XPath expression for
'address' evaluates to a nodeset with 2.b and 3.b.

> s/defined the module/defined in the module/

Thanks, fixed.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 09 Apr 2009 18:12:14 +0200 (CEST)

> then, the same configuration should be invalid, right?

Yes, see Andy's reply.

Thanks, fixed.

/martin

---

From: David Spakes <[email-masked]>
Date: Thu, 9 Apr 2009 16:42:40 -0400 (EDT)

> identity/identityref and OID would be done with the current syntax,

A MIB object of type OBJECT IDENTIFIER contains information about the
[MIB] schema implemented by the [SNMP] agent.  It's value is a path
to a position in the MIB tree.

A leaf having a type that is the proposed generic ref type contains
information about the [Yang] schema implemented by the [Netconf] agent.
It's value should also be a path to a position in the Yang module.

Below I have provided three examples.

1. Example 1 shows the YANG representation of the object definition for
   sysObjectID from RFC 3418 followed by the NETCONF XML Encoding in
   which the value of the object is taken from draft-ietf-netmod-yang-03
   page 18.

   YANG Example:

    leaf sysObjectID {
      type ref;
      config false;
      description
              "The vendor's authoritative identification of the
               network management subsystem contained in the entity.
               This value is allocated within the SMI enterprises
               subtree (1.3.6.1.4.1) and provides an easy and
               unambiguous means for determining `what kind of box' is
               being managed.  For example, if vendor `Flintstones,
               Inc.' was assigned the subtree 1.3.6.1.4.1.424242,
               it could assign the identifier 1.3.6.1.4.1.424242.1.1
               to its `Fred Router'.";
    }

   NETCONF XML Encoding (to refer to the module):

     <sysObjectID>acme-system</sysObjectID>

   NETCONF XML Encoding (to refer to a container in the module):

     <sysObjectID>acme:/system</sysObjectID>

2. Example 2 shows the YANG representation of exerpts from RFC 4293,
   where the leaf ipAddressPrefix points to a row in the
   ipAddressPrefixTable.  In the original MIB document, the object is
   defined to be of type "RowPointer" with semantics described in
   RFC 2579.

   YANG Example:

    container ipAddressPrefixTable {
      list ipAddressPrefixEntry {
        key   "ipAddressPrefixIfIndex, ipAddressPrefixType,
               ipAddressPrefixPrefix, ipAddressPrefixLength";
        description
           "An entry in the ipAddressPrefixTable.";

        leaf ipAddressPrefixIfIndex {
          type if-mib:InterfaceIndex;
          config false;
          description
           "The index value that uniquely identifies the interface on
            which this prefix is configured.  The interface identified
            by a particular value of this index is the same interface as
            identified by the same value of the IF-MIB's ifIndex.";
        }

        leaf ipAddressPrefixType {
          type inet-address-mib:InetAddressType;
          config false;
          description
           "The address type of ipAddressPrefix.";
        }

        leaf ipAddressPrefixPrefix {
          type inet:host;
          config false;
          description
           "The address prefix.  The address type of this object is
            specified in ipAddressPrefixType.  The length of this object
            is the standard length for objects of that type (4 or 16
            bytes).  Any bits after ipAddressPrefixLength must be zero.

            Implementors need to be aware that, if the size of
            ipAddressPrefixPrefix exceeds 114 octets, then OIDS of
            instances of columns in this row will have more than 128
            sub-identifiers and cannot be accessed using SNMPv1,
            SNMPv2c, or SNMPv3.";
        }

        leaf ipAddressPrefixLength {
          type inet-address-mib:InetAddressPrefixLength;
          config false;
          description
           "The prefix length associated with this prefix.

            The value 0 has no special meaning for this object.  It
            simply refers to address '::/0'.";
        }

        leaf ipAddressPrefixOrigin {
          type ip-mib:IpAddressPrefixOriginTC;
          config false;
          description
           "The origin of this prefix.";
        }

        /* ... detail omitted here ... */
      }
    }

    container ipAddressTable {
      list ipAddressEntry {
        key   "ipAddressAddrType, ipAddressAddr";
        description
           "An address mapping for a particular interface.";

        /* ... detail omitted here ... */

        leaf ipAddressPrefix {
          type ref;
          config false;
          description
           "A pointer to the row in the prefix table to which this
            address belongs.  May be { 0 0 } if there is no such row.";
        }

        /* ... detail omitted here ... */
      }
    }

   NETCONF XML Encoding:

     <ipAddressPrefix>"../../../ipAddressPrefixTable/ipAddressPrefixEntry/ipAddressPrefixOrigin","1","ipv4","192.168.1.0","24"</ipAddressPrefix>

   For the NETCONF client to correctly interpret the value of
   ipAddressPrefix, it may need to issue a <get-schema> query to
   the device.  In this case, the position is an instance of the
   columnar leaf object ipAddressPrefixOrigin.  The instance is:

      ipAddressPrefixIfIndex = 1
      ipAddressPrefixType    = ipv4
      ipAddressPrefixPrefix  = 192.168.1.0
      ipAddressPrefixLength  = 24

3. Example 3 shows the YANG representation of exerpts from RFC 2790,
   where the leaf hrStorageType points to any of several identities
   defined in the hrStoragetypes container in the HOST-RESOURCES-TYPES
   module.

   YANG Example:

    module HOST-RESOURCES-TYPES {
      namespace "urn:ietf:params:xml:ns:yang:smiv2:HOST-RESOURCES-TYPES";
      prefix "host-resources-types";

      container hrStorageTypes {
        identity hrStorageOther {
          description
              "The storage type identifier used when no other defined
               type is appropriate.";
        }

        identity hrStorageRam {
          description
              "The storage type identifier used for RAM.";
        }

        /* ... detail omitted here ... */
      }
    }

    module HOST-RESOURCES-MIB {
      namespace "urn:ietf:params:xml:ns:yang:smiv2:HOST-RESOURCES-MIB";
      prefix "host-resources-mib";

      /* ... detail omitted here ... */

      container hrStorage {
        container hrStorageTypes {
        }

        leaf hrMemorySize {
          type host-resources-mib:KBytes;
          config false;
          description
              "The amount of physical read-write main memory,
               typically RAM, contained by the host.";
        }

        container hrStorageTable {
          list hrStorageEntry {
            key   "hrStorageIndex";
            description
              "A (conceptual) entry for one logical storage area on
               the host.  As an example, an instance of the
               hrStorageType object might be named hrStorageType.3";

            leaf hrStorageIndex {
              type int32 {
                range "1..2147483647";
              }
              config false;
              description
              "A unique value for each logical storage area
               contained by the host.";
            }

            leaf hrStorageType {
              type ref;
              config false;
              description
              "The type of storage represented by this entry.";
            }

            /* ... detail omitted here ... */
          }
        }
      }
    }

   NETCONF XML Encoding:

     <hrStorageType>host-resources-types:/hrStorageTypes/hrStorageRam</hrStorageType>

Regards,

David

---

From: David Spakes <[email-masked]>
Date: Thu, 9 Apr 2009 16:45:30 -0400 (EDT)

Juergen,

I would not argue against keeping the object-identifier and
object-identifier-128 as built-in types in YANG for other purposes.

Regards,

David

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 9 Apr 2009 23:37:50 +0200

> to a position in the MIB tree.

The value of an OBJECT IDENTIFIER is well an OBJECT IDENTIFIER value.
The value can be anything and it does not have to be defined in any
MIB module.

Turning all OBJECT IDENTIFIERs into a new YANG ref type which requires
to introduce new syntatic constructs to refer to arbitrary elements in
a data tree won't solve the ambiguity mentioned above. 

Here is an example: The vacmViewTreeFamilySubtree object of the
SNMP-VIEW-BASED-ACM-MIB carried an OBJECT IDENTIFIER value that is
combined with a bitmask to identifier to identify a family of view
subtrees. There are two things worth to note here:

a) The vacmViewTreeFamilySubtree can be any OID - it does not have to
   point to anything ever defined in the SMIv2 because subidentifier
   can be masked (and as a consequence these masked subidentifier can
   be arbitrary without changing the semantics).

b) A mask is applied to the vacmViewTreeFamilySubtree OID. If you
   translate the OID to a ref, how to you then apply a bitmask to the
   ref value with a completely different syntax?

[ examples deleted ]

I believe YANG has the mechanisms in place to write proper YANG models
for all these examples that actually express more clearly what is
being referenced.

Automated translations from SMIv2 to "natural" YANG is difficult (or
requires additional annotations), but this is mainly a shortcoming of
the SMIv2. But note that automated translations to "workable" YANG is
possible - but this will expose SMIv2 concepts such as OIDs in the
YANG data models generated by a translator.

/js

---

From: WashamFan <[email-masked]>
Date: Fri, 10 Apr 2009 13:40:22 +0800

Hi,

para2 says:

   If "mandatory" is "true", the behavior of the constraint depends on
   the type of the leaf's closest ancestor node in the schema tree which
   is not a presence container (see Section 7.5.1):
            ^^^^^^^^^^^^^
   [ two bullets omitted ]

Could we just say "a container" instead?

Imagine that a mandatory leaf L is under a non-presence container C, then
bullet 2 applies:
L's existence depends on C's existence, since C is the closest ancestor which
is not a presence container.
but, ITOH,
C's existence depends on L's existence in return, since C is a container without
presence.

Does that make sense?

washam

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 14 Apr 2009 12:21:04 +0200 (CEST)

>    is not a presence container (see Section 7.5.1):

It should be "a non-presence container".   The idea is that
non-presence containers are ignored.

Now fixed, thanks.

/martin