Subject: RE: Submitting lock tokens without a validity check

From: "Clemm, Geoff" <[email-masked]>
Date: Wed, 20 Nov 2002 23:14:35 -0500

Oops: "to avoid lock protection" should read "to avoid lost updates".

Cheers,
Geoff

-----Original Message-----
From: Clemm, Geoff [mailto:[email-masked]]
Sent: Wednesday, November 20, 2002 10:52 PM
To: 'Webdav WG'
Subject: Submitting lock tokens without a validity check

One of the topics discussed at this weeks WebDAV working group meeting 
was how to provide a mechanism that would allow a client to submit a 
set of lock tokens without a validity check (i.e. the request could 
succeed even if some or all of those lock tokens have expired). 
Note that a client needs to submit an If header with etags with such 
a request, to avoid lock protection. 

There are currently two alternative proposals for this (the semantics 
of these two proposals are identical, so this is a marshalling question): 

Proposal One: Extend the If header so that it can take a comma 
separated list of arguments (and therefore can be split into multiple 
If statements).  To submit a set of lock tokens without a validity 
check, the following pattern would be used: 

  If: urlA (tokenA [etagA]) (Not tokenA [etagA]) 
  If: urlB (tokenB [etagA]) (Not tokenB [etagB]) 
  ... 

Proposal Two: Add a new header for a comma separated list of lock 
tokens that indicate possession of the lock token but do not cause the 
request to fail if they are invalid (I neglected to write down the 
proposed name, so I'll just call it New-Header).  Since the etag list 
can be long when the client holds a large number of locks, the 
extension defined in alternative one is also required, to handle the 
possibly large number of etags.  The pattern of usage for this 
proposal would be: 
  New-Header: tokenA 
  If: urlA ([etagA]) 
  New-Header: tokenB 
  If: urlB ([etagB]) 
  ... 

Advantage of proposal 1: 
- It does not require defining an extra header. 

Advantage of proposal 2: 
- It requires 40% fewer strings per resource (3 non-constant strings 
instead of 5 non-constant strings).  Lisa: You calculated that 
proposal one requires four times as many non-constant strings ... how 
did you get that number? 

I believe that it is not appropriate to add a new header to the protocol 
just to decrease the header length for this particular use case by 40%. 

I am particularly disinclined to optimize this kind of request, 
because I believe that it is significantly simpler for a client to 
use a standard If header, and if locks have expired, the request 
fails, the client deletes from its state those expired locks, and then 
resubmits the request, replacing the expired locks with etags.  This 
allows the client to just issue very simple If header requests, 
i.e. if the lock token for urlA is still valid but the lock token for 
urlB has expired: 

If: urlA (tokenA) 
If: urlB ([etagB]) 

Cheers, 
Geoff

---

From: "Clemm, Geoff" <[email-masked]>
Date: Thu, 21 Nov 2002 13:37:15 -0500

The semantics of locks guarantee that a lock never "moves",
so the client can simply store and submit the URL and the lock-token
as a pair, so there is no compelling
reason to have a "*" operator for lock token URLs.  In addition,
the uniqueness of an etag is only guaranteed for a particular
URL, so the "*" operator cannot be used for etag clauses of an
If header.

Cheers,
Geoff  

-----Original Message-----
From: Lisa Dusseault [mailto:[email-masked]]
Sent: Thursday, November 21, 2002 9:34 AM
To: 'Clemm, Geoff'; 'Webdav WG'
Subject: RE: Submitting lock tokens without a validity check

If a client uses "If: urlA (tokenA) (Not tokenA)" then the client will
*not* find out that their lock is invalid.   So that's not a real
advantage of the existing header depending on how the client chooses to
use it.  Since the client could still choose to use the If header if a
new header were defined, the two proposals you outline are equivalent in
that regard.

There is, however, another way to solve this that does have the property
of consistently letting the client know when it has bad tokens: extend
the existing If header to use * to indicate "any resource".  This is
consistent with If-Match and If-None-Match (HTTP 1.1) which use * to
indicate "any token".

To provide a token in this manner the client would use the * for the URL
with the lock token they wanted to use:

    If: * (<lock-token-A>)

Or several in one header:
    If: * (<lock-token-A>) * (<lock-token-B>) <urlC> (<lock-token-C>)

The way the server evaluates this: if <lock-token-A> matches any
resource, AND <lock-token-B> matches any resource, AND <lock-token-C>
matches the resource <urlC>, then the request succeeds.

This gives the advantage (not found with the two proposals discussed
previously), of letting the server return an error if the lock token is
invalid.  The client can use the lock token more easily as long as it's
valid, and the client gets its state corrected if its token is invalid.

--- 

Side note on header length: this is an improvement over the If-OR-NOT-If
syntax in length terms (it has one fewer lock token and a
single-character URL).  However, we might still want to add commas to
the If header.

Side note on backward compatibility: Adding commas and a '*' syntax to
the if header (assuming no OPTIONS flag) would cause some
interoperability issues.  However, it shouldn't last too long (I think
servers can pick this up rather quickly in particular) and is fairly
easy to deal with.  If a client sends the */, syntax to a server that
doesn't yet support it, a 400 Bad Request is the most likely response,
so I expect for a few years clients that want to use the */, syntax will
see this and fail back to the older syntax.  Servers should be able to
parse the If header with or without tokens, and the addition of the '*'
URL is simply an extension of the syntax that doesn't affect how the
existing URL stuff works.  (And of course, we could also add a "I
support rfc2518 bis" flag to the OPTIONS response to prevent the error
roundtrip for the case of new client --> old server.)

Side note on how I came up with "four times" larger: Assuming the new
header requires one copy of the token and zero URLs, and the
If-OR-NOT-If solution requires two copies of the token and one URL (per
resource), I should have said "three times" larger.  Just an off by one
error.  But in my experience URLs are twice as long as lock tokens so
maybe in practice it is four times larger :)

Lisa

-----Original Message-----
From: Clemm, Geoff [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2002 7:52 PM
To: 'Webdav WG'
Subject: Submitting lock tokens without a validity check

of these two proposals are identical, so this is a marshalling
question):

Advantage of proposal 1:

---

From: "Clemm, Geoff" <[email-masked]>
Date: Sun, 24 Nov 2002 08:23:15 -0500

> > If a client uses "If: urlA (tokenA) (Not tokenA)" then the
   > > client will *not* find out that their lock is invalid.

   > Wow. You managed to completely puzzle me.
   > I thought that was exactly the point of this kind of request.

   Weird, eh?  By using the If-Not-If hack, the client asks the
   request to succeed if the locktoken is invalid, or the locktoken is
   valid.  So the client never finds out if the locktoken is invalid,
   because the if-not-if combination never fails.

And if the client uses the proposed new header, the exact same thing
occurs, i.e. the client never finds out if the locktoken is invalid,
because the new header never fails.  What puzzled Julian was not
the semantics that you were describing, but rather the fact that
you dislike the semantics when it is marshalled in an If header,
but like it when it is marshalled in the new header.

   The issue to be solved is that all the client developers who have
   spoken up about the if header say that it's too hard to make
   interoperable.

And we have two proposals for how to fix this.  We are now debating
the relative merits of those two proposals.

   Every time the client is tested against a new server, some operations on
   locked resources fail.  Usually they fail because 
    1. the server expects to see a lock token that the client hasn't
   provided,
    2. the server expects a lock token to apply to a specific URL and the
   client didn't use the correct URL, 
    3. the client provides a locktoken for a URL that isn't even affected
   in the request, and that lock token or URL is incorrect, or 
    4. the lock expired.

   The reason why I suggested the approach of defining '*' as a wildcard
   for the IF header is because 
    - it removes the need to figure out which URL to put the locktoken on
   (fixing #2)

Since the only safe way (i.e. that prevents lost updates) to avoid
checking the validity of the lock token is to also specify an If
header with an etag for the resource, the client must specify
the appropriate URL for the etag.  And if it can remember what etag
goes with a URL, it can remember what lock token goes with a URL.

This is one of my objections to this separate header.  It makes it too
easy for a client writer who finds the If header "too complicated" to
just use this new header, omit to specify an If header with
appropriate etags, and then overwrite changes made by other clients to
resources whose locks have expired.  Conversely, a client that can
properly keep track of an etag (which requires keeping track of what
URL it goes with), can use the same data structures to keep track of
what lock token goes with that URL (and therefore #2 would not be an
issue).

    - when multiple resources are under the same lock, and when the lock
   token appears with '*', it's clear that  it applies to any resource
   locked with that token (fixing #2 even more);

As above, * doesn't work with etags (which must be used if locks are
not going to be checked), and if a client is going to accurately track
the etag for a URL, it can use the same data structure to accurately
track the lock token for a URL.

    - it makes it easier to include lock tokens that aren't actually
   required (fixing #3 and making #1 less likely)

A debate of which method is "easier" is unlikely to be productive.
Those of us that prefer the If/Not method believe it is "easier"
because it is more regular and more accurately reflects the semantics
being used.

    - It's shorter than putting the full URL (although we may still need
   comma support) (making #1 easier to achieve by including more lock
   tokens before running up against length limitations)

We have agreed that the "new header" method has 25% fewer non-constant
strings per locked resource (3 instead of 4).  But several of us feel
that the 25% savings in header length for this use case does not merit
the introduction of a new header.

   Julian said: "Checking: the desired semantics are that the request
   succeeds independently of the lock still being present or not?"

   Geoff said: "Yes."

   My response: I don't agree with that narrow definition of what's
   desired.  I think the problem statement is that the if header is
   constantly causing interoperability problems. The desired semantics
   are "some semantics that make locks easier to use".

Julian was not asking for a problem statement.  He was asking for the
desired semantics for the proposed new header.

Cheers,
Geoff