Subject: Re: Interop issue: Proposal for fixing lock token provision

From: Jason Crawford <[email-masked]>
Date: Wed, 2 Oct 2002 00:36:50 -0400

<<
In the light of this, is it useful to add another header where lock
token can also be supplied? What do client implementors say?
In light of that and everything else I said of course. :-)

Boy the discussion on this topic seems light....

Anyway, I'll continue...

Stephan, for me the issue is about good design, not ease for the
client.  We'll make sure it's easy for the client to do the basics,
but right the use of the If: header is not fully and clearly defined
in the spec.  Before 2518bis the spec suggests that the If: header can be
used
for three purposes: (1) Client driven verification checks, (2)
submission of lock tokens for server side locking checks and (3) lock
refresh token submission.  The spec only goes in to any detail and
explicitly mentions (1).  In regard to (3), 2518bis moved that function
in to a seperate header.   And (2) is not defined in any spec;... only
mentioned.  Now that (3) is largely
solved, it's (2) that concerns me.  It makes it impossible for the
server writers to *always* get it "right" because "right" isn't clearly
defined.  We need to come up  with a way to describe what activity in
the If: header constitutes token submission.   We could have a rule
like, "if the token is mentioned in the If: header, then the server
can assume that the client is submitting the token".  But what if
the client checks a resource that is locked by that lock but that is
not relevant to the operation?  What if the client checks that the lock
is not applied to a resource that indeed it is not, then should that
be considered submitting the lock?   And if our If: headers become
more sophisticated, then what?   If the client asserts that either
or both of two tokens are applied to a resource, is that a submission?
We really need to seperate and clearly define the mechanism for token
submission.

It's not just a matter of affecting our ability to submit lock tokens.
It also affects a client's ability to fully and confidently use the If:
header to check assertions. The mix of purposes is already is affecting
our current design.  See the following paragraph from 9.5.1.1

   When the If header is applied to a particular resource, the Tagged-
   list productions MUST be searched to determine if any of the listed
   resources match the operand resource(s) for the current method.  If
   none of the resource productions match the current resource then the
   header MUST be ignored.  If one of the resource productions does
   match the name of the resource under consideration then the list
   productions following the resource production MUST be applied to the
   resource in the manner specified in the previous section.

So I believe this is saying that the server should only check the If header
for resources that the server feels are relevant to the operation.  I'm
guessing
this was placed in the spec to support the checking of locks by the
server, but it impacts a client's ability to check assertions on other
resources
that the server might not think are relevant and are in scope for the If:
header.  It also makes the semantics of the If: header simply difficult to
understand.

We faced a similar problem when it came to refreshing locks in 2518.  One
had to use the If: header in that spec to submit locks for refreshing.
After some discussion we decided to use a Lock-Token header for the purpose
of submitting a token to be refreshed.  2518bis.2 reflects this
change.

I'm suggesting that we do the same thing for client driven assertion
checking.  This will give us one purpose for each header.  Then we can
clearly define the semantics of each header and only face design
constraints that are relevant to the purpose of that header.

J.

------------------------------------------
Phone: 914-784-7569

---

From: Stefan Eissing <[email-masked]>
Date: Wed, 2 Oct 2002 11:24:31 +0200

This was written after having made the comments below. To me it is 
becoming
unclear what exactly we are disucssing here. In

http://lists.w3.org/Archives/Public/w3c-dist-auth/2002JulSep/0275.html

I read three proposals:
1. more clearly define the If header and its scope.
2. Introduce an additional request header for relaxed lock
    checking. The client says: " this is the bag of lock tokens
   I have, server, take your pick."
3. Introduce an additional response header where the server
   can indicated which token from the bag are no longer valid.

I am all for (1).

Regarding (2) and (3) I argued that, as we still need the If header, is
it really worth it to add a new header with related semantics?

Especially in your original mail
(http://lists.w3.org/Archives/Public/w3c-dist-auth/2002JulSep/0398.html)
you said:

 > It's not really the length that's the problem in my mind.  It's
 > the fact that we have to define what it means for the If:
 > header to have *submitted* the token.  The If: header
 > can be quite sophisticated and we might choose to make
 > it even more sophisticated as we discover more requirements.
 > If we combine the functionality, then it constrains how much
 > we can extend the If: header.
 >
 > Yes, we could say that clients can support multiple If:
 > headers.  (I forget if we already do.)  But we'd end up
 > defining two types of If: headers for the two purposes.
 >
 > I'd prefer to seperate those explicitly.  WebDAV is only
 > in version 1 now and hopefully a design decision like
 > this will clarify the protocol now and avoid specification
 > headaches down the road.

If I understand you correctly, you seem to extend Lisa's original
proposal. Do you think 2518bis should remove lock tokens
from the If header completely?

//Stefan

Am Mittwoch, 02.10.02, um 06:36 Uhr (Europe/Berlin) schrieb Jason 
> In light of that and everything else I said of course. :-)

Must be the language barrier. I never meant to say that your
arguments should not be considered.

> mentioned.  Now that (3) is largely

Unless I do not fully understand your point, (2) is defined by me in
2518 ch. 7.6:
   "In order to prevent these collisions a lock token MUST be submitted
     by an authorized principal in the If header for all locked 
resources that
     a method may interact with or the method MUST fail."

Now, I fully agree to your point that the set of affected resources is
not clearly defined for all methods. Particularly it is not clearly 
defined
what the untagged production applies to and how deep locks are
handled. I'm all for clarifying in 2518bis how the If header should
be used and what it should look like, so clients and servers
implementations do not have to guess.

> be considered submitting the lock?   And if our If: headers become

Those are all good questions. Do you think this needs to be defined
in 2518bis or is this an argument for dropping lock-tokens from the
If header because it is impossible to get it right?

> We really need to seperate and clearly define the mechanism for token

Agreed.

> header.  It also makes the semantics of the If: header simply 

Again, I agree that it is not easy to understand. As you say, this is
partly due to the current wording in the spec. The other complexity
is due to the use cases it tries to solve.

---

From: "Lisa Dusseault" <[email-masked]>
Date: Wed, 2 Oct 2002 09:29:27 -0700

> it really worth it to add a new header with related semantics?
 
I am now convinced (I wasn't before the interop events) that it is
worthwhile to add a new and simple header.  Although we could spend a
lot of time defining the IF header more carefully (and may need to
anyway), and we could also document how to use the IF header to supply
lock tokens without causing the request to fail because of condition
matching, we have continued interoperability problems with every new
client implementation when they encounter odd situations and different
server implementations.  Because the IF header is so verbose, some of
the solutions are not feasible in practice.  A simple
(comma-delimitered) header for providing lock tokens would vastly
improve interoperability, and that's after all the goal for RFC2518 bis.

The original lock/if design had a lot of good ideas and definitely good
intentions, but with any complex design, the Law of Unintended
Consequences means there will be Unintended Consequences.  After
implementation experience, we know what those are.  Maybe we can fix
them.

That was the summary, here are the details.  Note that I do not mention
authentication below, in order to keep the discussion simple.  Any time
I discuss using a lock token to see if the write operation is allowed,
assume the server may also apply authentication checks to make sure the
user can use the lock.

A.  What are the continued interoperability problems?  

Basically, clients find their requests failing because of two major
cases:
 (1) they do not supply all the correct lock tokens (not knowing exactly
which lock tokens need to be supplied)
 (2) when they do supply all the correct lock tokens, the server applies
them to resources that are not locked with those lock tokens, and the
request fails because conditions must be met.  Clients find it difficult
to know what resources the server considers "are affected by" a request.

Much of this is due to the ambiguous sentence in section 9.4.1, "If a
method, due to the presence of a Depth or Destination header, is applied
to multiple resources then the No-tag-list production MUST be applied to
each resource the method is applied to."

Examples of (1): (I'm not asking for answers, I'm asking the questions
clients must ask themselves) 
 - Basically, what defines "resources the method is applied to"?  What
resource is a DELETE request applied to?
 - When you MOVE a resource into a locked collection, do you have to
supply the target collection's lock token?
 - When you MOVE a resource from a locked collection, do you have to
supply the source collection's lock token?  
 - When you COPY a resource from one locked collection to another, which
lock token do you have to supply, or both?
 - What happens when a collection is locked with depth 0?  What
operations are affected?  How is this handled differently than a depth
infinity lock on the same collection?  (Hint: depends on the server
implementation!)

Examples of (2):
 - When you DELETE a resource that's in a locked collection, can you
simply send the IF header with an untagged lock token?  No, on *some*
servers the token must be tagged with the parent collection (the root of
the lock).  
 - Any MOVE request where locks are involved cannot use untagged lock
tokens.
 - Some servers allow a lock token to be tagged with any resource-URL
that the lock scope includes.  Other servers require that the lock token
be tagged with the URL for the resource at the root of the lock. 

Basically, a careful client eventually learns to always tag every lock
token.  This results in a very large If header value already.

B. Why does the current situation guarantee additional roundtrips, but
not additional security?

A careful client (one that has learned the lessons from section A) that
is asked to do a write operation will attempt to include a tagged (with
the root-of-lock URL) lock token for every lock that could possibly be
considered to "affect" the operation.  So far so good -- as long as
those locks still exist, the request should succeed.

However, if any of those locks has been timed out or removed, the
request will fail because that lock token is no longer valid. 

Now the client must decide whether they want to try again to make the
request succeed, or not.  However, it's not necessarily clear from the
response which condition failed.  It's possible for the server to
respond simply with "precondition failed" and give no information which
one failed.  

An error to the user is typically undesirable.  Therefore, if the client
is a reasonably sophisticated client, they will typically try again.
Even if the user is consulted, the user will typically try again.  The
client will send the write operation again without the lock token.

A good (well-behaved) client doing a write operation that will overwrite
a regular resource should provide the ETAG to make sure that the
resource hasn't changed since the GET operation.  However, there's no
guarantee that a client MUST do that; a client can overwrite the
resource without checking the ETAG, simply by retrying the original
write request without the lock token that failed.

Thus, the current design requires an extra round-trip whenever the
client misunderstood the lock situation, but it does not guarantee any
additional protection because a poorly-behaved client can simply reissue
the request without the offending token and overwrite the resource.

Note in the case of large PUT requests the roundtrip may be particularly
time-consuming and resource-consuming.

C. Why does the current situation simply annoy a well-behaved client,
without particularly helping it?

A well-behaved client should always provide the ETAG when overwriting a
regular resource.  This is a good idea if the resource is locked or not.

When a resource is locked, and the client expects it has remained
locked, it may still be appropriate to overwrite the resource --
providing the ETAG has not changed.  It doesn't particularly matter if
the lock has gone away unexpectedly, as long as the resource is still
the same resource.

Thus, a PUT (or other write) request with an ETAG condition and a lock
token should be able to succeed if the ETAG is correct and the lock
token matches the current lock, BUT ALSO the request should succeed if
the ETAG is correct and the lock no longer exists.  

Since the well-behaved client should be using ETAGs anyway, the failure
of the request when the lock has gone away provides no help and instead
is simply annoying.

D. How could this situation be solved with the existing header, and why
is that solution poor?

Somebody suggested that the client can provide a IF header clause that
is guaranteed to succeed, but will also contain the lock token.  Here's
how it would work:
	IF  URL-A matches (lock-token-A OR NOT lock-token-A)
	
This header contains the lock token, so the request would succeed if the
resource is locked.  However, it has an OR clause, so the request would
succeed also if the resource is not locked.

PROBLEM #1: Servers may not support the OR, and the NOT, correctly,
because most clients don't currently use this.  This solution HAS NOT
been proven, it is only theoretical. It might not work.

PROBLEM #2: What if multiple locks are required (e.g. moving a
collection that has multiple locked resources?  What if the URLs are
long?  The IF header becomes very long and may be truncated by some
proxies.  E.g. 
  If: <http://www.ics.uci.edu/users/f/fielding/index.html> 
    (<opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf6>
     NOT <opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf6>)
     <http:// www.ics.uci.edu/users/f/fielding/anotherfile.html>
    (<opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf7>
     NOT <opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf7>)
     <http:// www.ics.uci.edu/users/f/fielding/thirdfile.html>
    (<opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf8>
     NOT <opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf8>)

PROBLEM #3: Uck, this is really complicated. While it's useful to know
that a client can do this with some existing servers, provided they
handle it correctly, surely we can do this in a simpler mechanism that
is less prone to interoperability problems.  A simple header that allows
the client to supply lock tokens to use is semantically equivalent to
the above example, but shorter, simpler, and easier to implement.  Also
it can be split across multiple lines.
  Use-Lock-Tokens:
<opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf6>, 
    <opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf7>
  Use-Lock-Tokens:
<opaquelocktoken:f81d4fae-7dec-11d0-a765-00a0c91e6bf8>

E.  How could this work better?

Servers could do a much better job of helping clients use write
operations and locks, rather than get in the way. Some ideas:

  1.  Allow write operations to succeed if the correct lock token is
provided in a "use these lock tokens" (comma-separated) header.  The
client would be able to make the write operation
  2.  Tell clients exactly what lock tokens are no longer valid.  
  3.  Tell clients exactly what resources are locked, when the client
asks for a write operation that affects resources that are locked
without providing the lock token.

Much of this can be (and needs to be) better specified within the
existing framework.  However, it's my opinion the existing framework can
be simplified, and that simplification would provide greater
interoperability.

F.  Why is simplicity so important?

When a request/response protocol is needlessly complex, client
implementers have several problems:
 1. First of all, complexity is just hard for the client to implement,
let alone implement correctly.
 2. Not only must the client deal with complexity, it must explain
things to users. WebDAV clients have to make complex things simple in
order to handle the typical user of a productivity or browser
application.  How is the client supposed to explain that a lock that may
have been there before may not be there now, and that it may or may not
be OK to overwrite the resource anyway?  It's not acceptable to explain
that to most users.
 3. Servers may implement the complex stuff differently.  It becomes
increasingly difficult to handle multiple server implementations and
their vagaries, as the complexity goes up.  
 4. Clients have so much else to worry about (usability, backward
compatibility, installation, multiple OS, GUIs) it's no surprise to find
that the protocol implementation team in a product like Office is a tiny
part of the team (or maybe even a library provided by a separate tiny
team). The protocol is just another minor tool, not the main value of an
application like Word or Photoshop. The client must be able to take the
protocol working correctly for granted. 

WebDAV does a good job in most cases of allowing clients to deal with
simple stuff first.  E.g. it's quite possible for a client to implement
only some methods and not others.  That's a great benefit: it allows a
client to spread an extended WebDAV-support effort over several
releases.  However, since other clients can lock resources, all clients
that do write operations must deal with much of the complexity of
handling locks.

Now that WebDAV has many server implementations which are fairly
reliable, fully featured and interoperable, the continued and growing
success of WebDAV depends on good, usable and ubiquitous client
implementations.  If we wish large vendors with existing productivity
applications to support WebDAV, including locks of regular resources and
collections, then it would be a great benefit to simplify the things
that have been found to be overly complex.  To me, that's worthwhile.

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 3 Oct 2002 20:08:11 +0200

(Geoff has posted an initial draft at [1], here are some comments)

Geoff,

I see that you are approaching variant handling from a versioning point of
view -- at least you are re-using RFC3253 as framework (which I think makes
sense).

In my point of view, variant handling extensions to WebDAV need to be
compatible with RFC2616 (HTTP), so I was trying to approach the problem from
a lower level.

Let's take a look at a very simple use case that doesn't involve any
extensions of HTTP(1.1) at all:

- Discover a resource through GET or HEAD.

- Through the "vary" header, the server announces that the representation
(variant) it sent varies on one or several of the request headers, such as
"accept-language".

- It may also send a "content-location" header, giving the URI of the
possibly authorable variant of the requested resource.

- Client can then discover that variant, and if this does not vary on any
request header, PUT to it's URI.

How does this translate to WebDAV:

- clarify that PROPFIND acts regarding variants the same way as GET and HEAD

- the information from the vary header should be available as live property

- there should be a generic way to express HTTP headers as WebDAV properties
(RFC2518's approach is broken, because there's no obvious mapping from HTTP
header names to WebDAV properties)

The next step would be to consider

- copy and move behaviour

- authoring of the varying resource

- direct discovery of variants

- define how variants move with the varying resource (I think this is
something you're planning to handle with the variant controller resource,
right?)

Do we have agreement on the goals, or do you really have a different idea of
variant handling?

Julian

[1]
<http://www.webdav.org/deltav/protocol/draft-ietf-deltav-variants-xx.1.htm>
--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760