Subject: [netmod] yang-03, 8.3.x, stepping on NETCONF turf

From: Andy Bierman <[email-masked]>
Date: Mon, 02 Feb 2009 07:05:23 -0800

Hi,

This section on 'payload parsing' (8.3.1) contains some normative
restrictions of NETCONF protocol usage which do not belong in this spec.

    o  Leaf data values MUST match the type constraints for those leafs,
       including those defined in the type's range, length, and pattern
       properties.

    o  Key data MUST be present for all list instances.

    o  Only data for a single case MUST be present for any choices.

Already covered by NETCONF invalid-value.
No value added by NETMOD here.

    o  Data for nodes tagged with "if-feature" MUST NOT be present if the
       feature is not supported by the device.

Why?
What if the agent supports some, but not all of
the feature objects, so it cannot advertise it?
Why forbid a manager from sending any request it wants?
It is up to each individual agent to answer, and
existing NETCONF error codes are sufficient to address
the problem.  BTW, just because an agent advertises
a feature does not mean the manager can assume
anything about the success of any individual <rpc>.

    o  Data for nodes tagged with "when" MUST NOT be present if the
       condition does not evaluate to "true".

Again, why?
The agent will send the appropriate error for
any PDU it does not support.

    o  For insert handling, the value for the attributes "before" and
       "after" MUST be valid for the type of the appropriate key leafs.

    o  The attributes "before" and "after" MUST NOT appear for lists
       whose "ordered-by" property is "system".

This is backwards.  The YANG spec just needs to specify
what an agent does if these conditions occur
(invalid-value or operation-failed?)

Section 8.3.2 is completely redundant (covered by NETCONF),
except for bullet 3 which is under-specified.
Which error-tag is returned?  Better yet, which sub-section (13.5?)
contains the normative error handling behavior?

Also bullet 4 is a problem:

    o  If the NETCONF operation modifies a data node such that any node's
       "when" expression becomes false, then that node is deleted by the
       server.

I strongly object to this AUTOMATIC DELETION
'server behavior' slipped into this section.
This issue needs more discussion and WG consensus.
The idea that changing the value of knob X is going to
silently and automatically delete portions of the config
is going to (appropriately) be a concern to operators.

All it takes is 1 mistake, setting knob X to a 'wrong value',
and big chunks of config just vanish?  That's not a feature, IMO.
All deletes (except 'choice' subtrees) MUST be explicitly requested
by the operator.

IMO, the 'when' statement is just a hint to applications.
The server just needs to implement it correctly (like 'must').

Therefore, the first paragraph of 8.2 is a problem:

8.2.  Hierarchy of Constraints

    Conditions on parent nodes affect constraints on child nodes as a
    natural consequence of the hierarchy of nodes. "must" and "mandatory"
    constraints are not enforced if the parent node has a "when" or "if-
    feature" property that is not satisfied on the current device.

This should say that 'must' and 'mandatory' are enforced if the
parent exists.  The 'when' and 'if-feature' statements should not
be coupled to these other constraints.  If implemented correctly,
and the XPath is correct, then these nodes will not exist.
But if the node does exist, then 'must' and 'mandatory'
need to be enforced.

Andy

_______________________________________________
netmod mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netmod

---

From: WashamFan <[email-masked]>
Date: Tue, 03 Feb 2009 10:19:00 +0800

Hi,
>  any PDU it does not support.

I think we should spell out what agents should react if operators have sent such *bad* requests,  an error returned or session closed? What does "MUST NOT" here exactly mean?

>  But if the node does exist, then 'must' and 'mandatory'

Yes. That makes more sense.

washam

---

From: Phil Shafer <[email-masked]>
Date: Wed, 11 Feb 2009 16:44:37 -0500

>No value added by NETMOD here.

No, NETCONF makes no restriction on either what values may be
transmitted in elements, nor when those errors are reported.  YANG
adds these restrictions for YANG-based models.

>the feature objects, so it cannot advertise it?

Then it can either roll its own model or use deviations to report
its underimplementation of the standard model.

Either way, the nodes should not be present of the feature is not
supported.

>Why forbid a manager from sending any request it wants?

This seems like an odd question.  Why define a protocol at all?
We define it to allow clients to know what they can send and what
they will receive in reply, as well as allowing servers to know
what they will receive and what they need to send in reply.

>the problem.

It is up to the YANG standard to define how and when
NETCONF content is restricted and what error are sent,
even if the error is defined in the NETCONF standard.

>BTW, just because an agent advertises

No, but it means the client and server understand the
semantic and syntactic rules the model defines.

>any PDU it does not support.

Yes, the server will fail RPCs that are not valid, but the YANG
standard is defining that is and is not valid.  If you send
data the server does not understand, you will get an error.
The sentence above says that the server will not understand
nodes that are tagged with "when" when the "when" isn't true.

>(invalid-value or operation-failed?)

I think both are true here.  We need to say "don't to that" as
well as defining standard behavior when the client does not
behave correctly.

>'server behavior' slipped into this section.

This behavior is used in other operations, like setting a value
(the old value is deleted), changing a choice (the other choices
are deleted), etc.  These rules help keep the data aligned with the
known rules and avoid scenarios where invalid data is allowed to
remain in the database.  This is normal, real world behavior, as
implemented in many devices.

>The server just needs to implement it correctly (like 'must').

If the modeler wanted "must", they would have used it.  "when" gives
different functionality.

>But if the node does exist, then 'must' and 'mandatory'

Um, I think you argued it the other way previously.  If a non-presence
container has a "when", you wanted "must" contraints enforced even
if the n-p container does not exist.  This section means that if
the "when" is false, these are not enforced.  I would be happy to
remove the enforcement of "must"s when the parent does not exist,
if that is your new position.

All this language was added based on the concensus reached at the
interim.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Feb 2009 14:21:22 -0800

> adds these restrictions for YANG-based models.

I disagree.
The NETCONF protocol defines what agent MUST, SHOULD, MAY accept.
That is sufficient.  There is no point in mandating what NETCONF
managers can send inside the YANG spec.

> what they will receive and what they need to send in reply.

The NETCONF RFC already defines the protocol operations.
The agent should respond appropriately to any request.
If any additional requirements are needed, then 4741-bis
should define them, not the YANG RFC.

> nodes that are tagged with "when" when the "when" isn't true.

There is already an unexpected-element error for YANG nodes
that the agent does not understand.  IMO it is confusing to
have multiple flavors of the NETCONF protocol, each defined
separately in an ad-hoc fashion, each for only the portion of the
database associated with that document.  I prefer that 4741-bis
be the only document that defines how edit-config works.

> well as defining standard behavior when the client does not

You could go through the entire NETCONF RFC and write an inverse
manager rule for every agent rule found.  I prefer this entire section
be moved to a non-normative guidelines appendix at the end.
A set of usage tips would be helpful, if they were all grouped together.

> All this language was added based on the concensus reached at the

I do not remember any discussion of delete-all-false-when-stmt objects,
at the interim or on the mailing list.  IMO, consensus is TBD.
But if this 'feature' stays, there should be a standard 'config-changed'
notification defined ASAP to let managers know what happened to
chunks of YANG config when they changed that knob with the
complicated 'when-stmt' XPath expression they ignored.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 12 Feb 2009 08:30:50 -0500

>managers can send inside the YANG spec.

NETCONF defines an RPC-based protocol for transmitting data.  The
content and nature of that data is not specified.  The data model
has complete control over the organization and content of the data.
The model determines when content violates those rules and what
errors should be reported.  NETCONF defines some errors, but the
model defines when a value is in error.

>be the only document that defines how edit-config works.

Andy, I don't think I or anyone else is suggesting "multiple flavors
of the NETCONF protocol".  This is a red herring.

>I prefer this entire section

I don't think a non-normative appendix was the concensus.

>A set of usage tips would be helpful, if they were all grouped together.

These aren't helpful tips.  They are rules that govern how servers
that implement YANG-based models must behave.

>at the interim or on the mailing list.  IMO, consensus is TBD.

This was a lengthy discussion at the interim.

>complicated 'when-stmt' XPath expression they ignored.

A draft defining a standard 'config-changed' notification is a topic
that should addressed in the NETCONF WG, since the YANG WG doesn't
want to step on NETCONF's turf.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Feb 2009 07:45:21 -0800

> that implement YANG-based models must behave.

Rules for servers to follow as they process NETCONF PDUs
for YANG data models are fine.  (If they do not simply
repeat what RFC 4741 says to do.)

Rules for managers that send NETCONF PDUs are not fine.
This implies that every NETCONF application that sends
an <rpc-request> MUST:

   1) be aware of all the YANG data models that the agent
      supports.  (Even though you and Martin argued earlier
      that we need 'deviations=' in the capability URI,
      because a manager is free to ignore any capability URI
      it wants.)

   2) be aware of the full current contents of the target database
      and have XPath tools to process every when-stmt in affect
      on the agent, before invoking any database operations.

Our experience with 'mib-walker' and other common denominator
applications strongly suggests that there will be NETCONF
applications that do not choose to understand every YANG
capability URI sent to them (maybe none of them), and
therefore will send requests for data they do not know has
some if-feature-stmt and when-stmt false conditions.

Since the agent will always answer with an 'unexpected-element'
error for all these cases, it doesn't matter how much
or how little effort the application does on pre-filtering the
requests. No extra work for the agent, but a ton of work
for the manager, and no change in protocol outcome for
any of the application developer's efforts.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 12 Feb 2009 13:14:44 -0500

>Rules for managers that send NETCONF PDUs are not fine.

Sorry, but I don't see this.  YANG defines the orgnaization of the
model's elements, which turn into rules for the client.  If the
model list "name" as a key, the name element must appear when
needed.

>      because a manager is free to ignore any capability URI

Not true.  Applications need only pay attention to the models
which they care about.  An application doing OSPF need not
know anything about ISIS data models.

>      on the agent, before invoking any database operations.

No, they need to understand their own data models.  If there
is a "when" constraint in the /protocols/isis hierachy, the
OSPF application need not know it.

>any of the application developer's efforts.

So you are wanting to allow any garbage from the client, as long
as the server is allowed to give errors?   I don't see the point.
We want to give both the client and server strong constraints on
what they can send and what they will receive in order to increase
interoperability and help folks do the right think.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Feb 2009 11:12:04 -0800

....
> interoperability and help folks do the right think.

The server is 'required' to give errors, not 'allowed' to give errors.
It seems strange to mandate with MUST-type normative text
that the manager MUST NOT send certain requests that
the agent will answer with an 'unknown-element' error.

This is based on certain YANG clause values that need to be
validated with the proper YANG modules, XPath tools,
and fresh copy of the configuration database.

IMO, this is not important and not required in the text.
There is no difference between a manager sending a
request to edit '/foo/garbage' or '/feature-not-loaded-into-agent'.
than a request to edit '/foo/false-when-clause-object'
or '/foo/false-if-feature-object'.

All of these requests are just errors for non-existent nodes.
Why are YANG if-feature and when-stmt missing nodes different
from a generic 'unknown-element' error?
You might as well say the manager MUST NOT send any request
that the agent does not fully understand.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Thu, 12 Feb 2009 14:24:18 -0500

>the agent will answer with an 'unknown-element' error.

So you don't want the spec to say what is and isn't allowed,
you prefer to just say "if you send this, the agent MUST
send an error"?

>or '/foo/false-if-feature-object'.

Is there a difference between saying "don't request an
edit on non-existent nodes" and "if you send an edit
on non-existent node, the server MUST respond with
blah-blah error"?

>from a generic 'unknown-element' error?

You are targeting "when" and "feature", but this is a broader content
issue.  Consider the following text from the draft:

   - Key data MUST be present for all list instances.

Should this be rephrased as an error-emitting opportunity?

   - If key data is not present in all list instances, the
   server MUST generate an appropriate error.

Would you prefer a remark at the top or bottom of this list
(under the "Payload Parsing" section) saying something like:

   If the payload does not comply with any of these rules,
   the server MUST generate an appropriate error.

>that the agent does not fully understand.

We're trying to define rules that allow clients to know what
is and isn't valid, so they can always send requests that the
server fully understands.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 12 Feb 2009 20:51:00 +0100 (CET)

> the agent will answer with an 'unknown-element' error.

I agree with you.  What if rephrase the text we're discussing so that
it instead specifies which error the server MUST send?

For example, instead of:

  Leaf data values MUST match the type constraints for those leafs,
  including those defined in the type's range, length, and pattern
  properties.

we could say:

  If a leaf data value does not match the type constraints for the
  leaf, including those defined in the type's range, length, and
  pattern properties, the server MUST reply with an 'invalid-value'
  error-tag in the rpc-error.

I'm not sure if the original text also tried to put requirements on
data sent by the server as replies to e.g. <get> requests.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Thu, 12 Feb 2009 12:18:34 -0800

>   error-tag in the rpc-error.

This is excellent.
The necessary and sufficient text in the YANG document
should be limited to how (and when) a NETCONF agent enforces the
behavior in particular YANG constructs.

> data sent by the server as replies to e.g. <get> requests.

The YANG draft (e.g., canonical XML representation) certainly
puts requirements on what an agent can send.  That is OK.

Andy