Subject: [netmod] canonical order

From: Andy Bierman <[email-masked]>
Date: Sat, 03 Jan 2009 10:45:09 -0800

Hi,

When implementing XPath for YANG, the relative order
of nodes within the 'current document' is significant.

For the object tree (during YANG file validation),
the 'following' and 'preceding' axis are determined
by the document order of the YANG definitions.

For the value tree, these axis, plus 'following-sibling'
and 'preceding-sibling', will be determined by the actual
database tree instance order.

No problems, so far, except for key leafs.
The document order for list contents does not
have to match the PDU order, which MUST have the
key leafs first (only because of the CLR in YANG
that says so).

But that CLR only applies to PDU order, not
NETCONF database order.  There is no defined order
for that, for sibling objects or sibling instances
of an object.

In order for must-stmt, when-stmt, XPath and subtree
filtering to work the same way on different agents
(the point of a standard, I am told), then the
canonical order of a NETMOD database needs to
be defined in the draft.

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Jan 2009 10:12:10 +0100 (CET)

Andy Bierman <[email-masked]> wrote:

> the

I think there are two simple solutions:

  1. State that the order that currently is the defined PDU order
     instead is the canonical datastore order.

  2. State that any order in the datastore is implementation
     specific.  Consequently, XPath functions related to this order
     should not be used.

     (note that even the XPath spec states that the order among some
     specific nodes are implementation dependent)

Related to ordering, the draft currently has this text about how
augmented nodes are encoded:

  When a node is augmented, the augmented child nodes are encoded
  after all normal child nodes.  If the node is augmented more than
  once, the blocks of augmented child nodes are sorted (in
  alphanumeric order) according to their namespace URI and name of the
  first child node in each block.

I was thinking this should be removed, but unless we want to do (2)
above, maybe it should stay.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 05 Jan 2009 10:33:28 +0100

> I think there are two simple solutions:

Another solution would be to make the order arbitrary both in the
datastore and in PDUs. Are there any use cases where the order is
important?

We have to be careful with (leaf-)lists though, it would probably not be
a good idea to interleave list items with other sibling nodes.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Mon, 5 Jan 2009 08:02:57 -0500

+1 to both option 2 and removing the current text re: sorting.
Both should be implementation details and encouraging their
use will increase the cost for NETCONF server implementations
and will give no real benefit.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 05 Jan 2009 14:23:52 +0100

> and will give no real benefit.

If the sorting rule is removed, what will then be the order of elements
in PDUs if a module is augmented at the same spot by two or more
modules? Is it supposed to be arbitrary as well?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Jan 2009 14:28:00 +0100 (CET)

> modules? Is it supposed to be arbitrary as well?

Yes.  This is also OK wrt the XSD mapping, since all augmentation are
done using a substitutionGroup.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Mon, 5 Jan 2009 08:29:46 -0500

>modules? Is it supposed to be arbitrary as well?

The order in which augmentations appear should be implementation
dependent.  The only ordering requirement should be that all keys
proceed non-keys, to allow servers to avoid buffering.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 06:42:17 -0800

>      specific nodes are implementation dependent)

So the solution is to ignore my original issue,
which is that must-stmt and when-stmt expressions
appearing in a standard YANG data model will not evaluate
to the same result, for a significant subset of the
XPath language.  Any 'foo[3]' type of expression
(e.g., context order within a predicate) is going
to evaluate randomly on every single agent?

I fail to see the standards value in this approach.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Jan 2009 16:07:49 +0100 (CET)

> I fail to see the standards value in this approach.

Using positional functions will be error prone anyway.  You'd have to
calculate the offset depending on when expressions, if-feature etc.
For example:

   leaf a { ... }
   leaf b {
      when "../a > 10";
   }
   leaf c { ... }

The "following-sibling" of "a" is "b" or "c", depending on the value
of "a".  Why would you use this in "must" or "when" expressions?

I do not think that this is "a significant subset" of XPath.  I think
it's more clear to refer to other nodes by name than by position.  Do
you have some particular use case in mind where using the name does
not work?

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 07:20:57 -0800

> you have some particular use case in mind where using the name does

Of course there are better ways to write a must-stmt
than to use the relative position within an instance
document, or use the preceding, preceding-sibling,
following, or following-sibling axis.

If that is not important, then it needs to be clear
in the draft, so people who write XPath expressions
can avoid ones for which the YANG standard will produce
different results, depending on the implementation.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 11:29:10 -0800

>      specific nodes are implementation dependent)

Option 3:

State that any order in the datastore is implementation-specific,
however, an agent MUST NOT change the order of any nodes,
such that, if an application retrieves the entire datastore
at time T0, and an XPath expression is evaluated at time T1:

  - if no changes occurred between T0 and T1, then the document
    order in the retrieved datastore PDU MUST be valid.

  - if any changes did occur between T0 and T1, then the
    retrieved document order may no longer be valid.

In either case, position-dependent expressions SHOULD NOT be used.

What about leaf-lists and lists without keys?
They are intrinsically indexed by their position
in XML, regardless of what YANG does.

use-case:

   container multi-dns-options {
     when "/acme:dns/acme:server[2]";

     description
         "Used when more than one DNS server is configured
          on this host.";

     //...
   }

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 5 Jan 2009 14:29:18 -0500

>     when "/acme:dns/acme:server[2]";

This is more directly expressed as:

  when "count(/acme:dns/acme:server) >= 2";

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Jan 2009 12:24:36 -0800

>   when "count(/acme:dns/acme:server) >= 2";

That really isn't the point.

If both of these expressions do not yield the
same result on the same datastore, then the
XPath implementation is broken.  Both are valid XPath.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 5 Jan 2009 16:15:03 -0500

>XPath implementation is broken.  Both are valid XPath.

Sorry, too terse.  I was giving an alternative for your use case
which doesn't use positioning.  We should declare positioning as
implementation dependent and move on.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 06 Jan 2009 12:48:12 +0100 (CET)

>          "Used when more than one DNS server is configured

But this expression evaluates to the same (boolean) on different
agents even if the order is different.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Tue, 06 Jan 2009 05:11:02 -0800

> agents even if the order is different.

This expression would work fine, just like count(foo).
It's like C shorthand (!foo) or (foo != NULL).
It will be true if there are two or more entries.

If YANG is going to change this part of XPath,
then it needs to go in with the other changes
to XPath 1.0 that YANG mandates.

According to XPath, 2 identical XML instance documents
have the same forward or reverse order.  It doesn't say
the order is arbitrarily different on every implementation.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 06 Jan 2009 07:57:51 -0800

> implementation dependent and move on.

Do we at least agree that the order must be consistent within
a single implementation?

If the affected portion of the database has not
changed at all, and if a manager retrieves /foo[3] or
runs a must-test on /foo[3] at T0 and then T1, will
the result be the same each time?

IMO, MUST be yes.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 6 Jan 2009 11:11:43 -0500

>Do we at least agree that the order must be consistent within

Sounded like you and Martin disagree on whether XPath allows this
sort of order issues.   Within a single element type, I don't see
how para[3] could differ between implementations, but I certainly
see that *[3] or node()[3] would be implementation specific.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 06 Jan 2009 08:50:31 -0800

> see that *[3] or node()[3] would be implementation specific.

We are just talking about 1 manager issuing a <commit>
or a <get> with a an XPath filter to 1 particular agent.

If the manager issues the same command 2 times in a row
with the same XPath (I don't care what it is), and no nodes
have been added or removed from the database,
and no values in the database have changed at all:

Will the manager get the same answer or not?

If not, why not?

I understand that results from different agents
may not be the same, even for the same (*)
database contents.

* configurations can only be compared outside the database,
   represented as XML documents.

Andy