Subject: Submitting lock tokens without a validity check

From: "Clemm, Geoff" <[email-masked]>
Date: Wed, 20 Nov 2002 22:51:31 -0500

One of the topics discussed at this weeks WebDAV working group meeting
was how to provide a mechanism that would allow a client to submit a
set of lock tokens without a validity check (i.e. the request could
succeed even if some or all of those lock tokens have expired).
Note that a client needs to submit an If header with etags with such
a request, to avoid lock protection.

There are currently two alternative proposals for this (the semantics
of these two proposals are identical, so this is a marshalling question):

Proposal One: Extend the If header so that it can take a comma
separated list of arguments (and therefore can be split into multiple
If statements).  To submit a set of lock tokens without a validity
check, the following pattern would be used:

  If: urlA (tokenA [etagA]) (Not tokenA [etagA])
  If: urlB (tokenB [etagA]) (Not tokenB [etagB])
  ...

Proposal Two: Add a new header for a comma separated list of lock
tokens that indicate possession of the lock token but do not cause the
request to fail if they are invalid (I neglected to write down the
proposed name, so I'll just call it New-Header).  Since the etag list
can be long when the client holds a large number of locks, the
extension defined in alternative one is also required, to handle the
possibly large number of etags.  The pattern of usage for this
proposal would be:

  New-Header: tokenA
  If: urlA ([etagA])
  New-Header: tokenB
  If: urlB ([etagB])
  ...

Advantage of proposal 1:

- It does not require defining an extra header.

Advantage of proposal 2:

- It requires 40% fewer strings per resource (3 non-constant strings
instead of 5 non-constant strings).  Lisa: You calculated that
proposal one requires four times as many non-constant strings ... how
did you get that number?

I believe that it is not appropriate to add a new header to the protocol
just to decrease the header length for this particular use case by 40%.

I am particularly disinclined to optimize this kind of request,
because I believe that it is significantly simpler for a client to
use a standard If header, and if locks have expired, the request
fails, the client deletes from its state those expired locks, and then
resubmits the request, replacing the expired locks with etags.  This
allows the client to just issue very simple If header requests,
i.e. if the lock token for urlA is still valid but the lock token for
urlB has expired:

If: urlA (tokenA)
If: urlB ([etagB])

Cheers,
Geoff

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 21 Nov 2002 09:52:33 +0100

Thanks for the summary, Geoff.

Some thoughts/questions:

> From: [email-masked] [mailto:[email-masked]]On
> Behalf Of Clemm, Geoff
> Sent: Thursday, November 21, 2002 4:52 AM
> To: 'Webdav WG'
> Subject: Submitting lock tokens without a validity check
>
>

Checking: the desired semantics are that the request succeeds independently
of the lock still being present or not?

>   If: urlB (tokenB [etagA]) (Not tokenB [etagB])

I think this can be minimized to:

  If: urlA (tokenA [etagA]) (Not (<DAV:no-lock>) [etagA])

(<DAV:no-lock> is the URI of a known not-to-be-present lock, so the second
List always evaluates to true).

> possibly large number of etags.  The pattern of usage for this

Is ordering relevant here? So would

  New-Header: tokenB
  If: urlA ([etagA])
  New-Header: tokenA
  If: urlB ([etagB])

mean the same thing?

> proposal one requires four times as many non-constant strings ... how

With my minimization above, I'm getting only one additional non-constant
string.

> just to decrease the header length for this particular use case by 40%.

Agreed.

> i.e. if the lock token for urlA is still valid but the lock token for

Maybe we should discuss enhancements for error reporting for precondition
failures on if headers? That would probably make it easier for a client to
recover.

Julian

--
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: "Lisa Dusseault" <[email-masked]>
Date: Thu, 21 Nov 2002 06:33:39 -0800

If a client uses "If: urlA (tokenA) (Not tokenA)" then the client will
*not* find out that their lock is invalid.   So that's not a real
advantage of the existing header depending on how the client chooses to
use it.  Since the client could still choose to use the If header if a
new header were defined, the two proposals you outline are equivalent in
that regard.

There is, however, another way to solve this that does have the property
of consistently letting the client know when it has bad tokens: extend
the existing If header to use * to indicate "any resource".  This is
consistent with If-Match and If-None-Match (HTTP 1.1) which use * to
indicate "any token".

To provide a token in this manner the client would use the * for the URL
with the lock token they wanted to use:

    If: * (<lock-token-A>)

Or several in one header:
    If: * (<lock-token-A>) * (<lock-token-B>) <urlC> (<lock-token-C>)

The way the server evaluates this: if <lock-token-A> matches any
resource, AND <lock-token-B> matches any resource, AND <lock-token-C>
matches the resource <urlC>, then the request succeeds.

This gives the advantage (not found with the two proposals discussed
previously), of letting the server return an error if the lock token is
invalid.  The client can use the lock token more easily as long as it's
valid, and the client gets its state corrected if its token is invalid.

--- 

Side note on header length: this is an improvement over the If-OR-NOT-If
syntax in length terms (it has one fewer lock token and a
single-character URL).  However, we might still want to add commas to
the If header.

Side note on backward compatibility: Adding commas and a '*' syntax to
the if header (assuming no OPTIONS flag) would cause some
interoperability issues.  However, it shouldn't last too long (I think
servers can pick this up rather quickly in particular) and is fairly
easy to deal with.  If a client sends the */, syntax to a server that
doesn't yet support it, a 400 Bad Request is the most likely response,
so I expect for a few years clients that want to use the */, syntax will
see this and fail back to the older syntax.  Servers should be able to
parse the If header with or without tokens, and the addition of the '*'
URL is simply an extension of the syntax that doesn't affect how the
existing URL stuff works.  (And of course, we could also add a "I
support rfc2518 bis" flag to the OPTIONS response to prevent the error
roundtrip for the case of new client --> old server.)

Side note on how I came up with "four times" larger: Assuming the new
header requires one copy of the token and zero URLs, and the
If-OR-NOT-If solution requires two copies of the token and one URL (per
resource), I should have said "three times" larger.  Just an off by one
error.  But in my experience URLs are twice as long as lock tokens so
maybe in practice it is four times larger :)

Lisa

-----Original Message-----
From: Clemm, Geoff [mailto:[email-masked]] 
Sent: Wednesday, November 20, 2002 7:52 PM
To: 'Webdav WG'
Subject: Submitting lock tokens without a validity check

of these two proposals are identical, so this is a marshalling
question):

Advantage of proposal 1:

---

From: "Julian Reschke" <[email-masked]>
Date: Thu, 21 Nov 2002 15:50:19 +0100

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Lisa Dusseault
> Sent: Thursday, November 21, 2002 3:34 PM
> To: 'Clemm, Geoff'; 'Webdav WG'
> Subject: RE: Submitting lock tokens without a validity check
>
>
>

Wow. You managed to completely puzzle me.

I thought that was exactly the point of this kind of request.

Can you please explain again what the issue to be solved is, or point me to
"the" mailing list message (in the archive) that explains it?

---

From: Stefan Eissing <[email-masked]>
Date: Thu, 21 Nov 2002 16:49:15 +0100

Lisa,

the use case which started the whole thread is "the client wants the
request to succeed irregardless of wether the lock is still there or 
not".
The addition of "*" to the If header does not help to solve this case.

The introduction of "*" addresses the problem where the client is not
sure for exactly which URIs it must provide lock-tokens in the IF 
header.

That is also a problem worth addressing. Jason mentioned some weeks
ago that 2518 does not clearly define when a lock-token is "submitted"
and how the methods should check for submitted tokens.

As I see it, the IF header is used for two purposes:
P1) resource state checking
P2) submitting "known" lock tokens

P1 is, IMHO, pretty good defined. A client puts the URI of the resource
to check in the IF header. If the URI is omitted, the request URI is 
silently
substituted.
All mentioned resource states are then verified by the server. Should
a state not match (e.g. a lock token no longer there), the server will
fail the request immediatly.

For P2 we should define that server implementations should regard all
lock-tokens supplied in the IF header as "submitted tokens".
When verifying locks on resources affected by methods (such as COPY
or MOVE), servers should for locked resources look into the set of
"submitted tokens". If the lock is there, the request can proceed.
Otherwise it will fail with a 423.

Defining this behaviour will increase interoperability and ease life for
clients. Clients just memorize the lock token together with the URI
the lock was created on. They can then simply place those tuples
in the IF headers of requests and everything should work fine.

More optimizing clients might more carefully chose the locks to
submit, but that is left up to their implementation.

//Stefan

Am Donnerstag, 21.11.02, um 15:33 Uhr (Europe/Berlin) schrieb Lisa

---

From: "Lisa Dusseault" <[email-masked]>
Date: Sat, 23 Nov 2002 14:13:07 -0800

> I thought that was exactly the point of this kind of request.

Weird, eh?  By using the If-Not-If hack, the client asks the request to
succeed if the locktoken is invalid, or the locktoken is valid.  So the
client never finds out if the locktoken is invalid, because the
if-not-if combination never fails.

> 
> Can you please explain again what the issue to be solved is, or point
me
> to

The issue to be solved is that all the client developers who have spoken
up about the if header say that it's too hard to make interoperable.
Every time the client is tested against a new server, some operations on
locked resources fail.  Usually they fail because 
 1. the server expects to see a lock token that the client hasn't
provided,
 2. the server expects a lock token to apply to a specific URL and the
client didn't use the correct URL, 
 3. the client provides a locktoken for a URL that isn't even affected
in the request, and that lock token or URL is incorrect, or 
 4. the lock expired.

The reason why I suggested the approach of defining '*' as a wildcard
for the IF header is because 
 - it removes the need to figure out which URL to put the locktoken on
(fixing #2)
 - when multiple resources are under the same lock, and when the lock
token appears with '*', it's clear that  it applies to any resource
locked with that token (fixing #2 even more);
 - it makes it easier to include lock tokens that aren't actually
required (fixing #3 and making #1 less likely)
 - It's shorter than putting the full URL (although we may still need
comma support) (making #1 easier to achieve by including more lock
tokens before running up against length limitations)

Lisa

---

From: "Julian Reschke" <[email-masked]>
Date: Sun, 24 Nov 2002 14:20:14 +0100

> Sent: Saturday, November 23, 2002 11:13 PM
> To: [email-masked]

Yes. Really weird. Why would it want to know then? It will certainly find
out when it finally does the UNLOCK. Where's the problem with that?

>  1. the server expects to see a lock token that the client hasn't

So we need to clarify which are required.

> client didn't use the correct URL,

Fix the client or the server.

>  3. the client provides a locktoken for a URL that isn't even affected

If the URL isn't affected, use If header syntax that makes the matching of
the lock optional (we have that),

> in the request, and that lock token or URL is incorrect, or

Fix the client.

>  4. the lock expired.

Yes. That's the point of locking. If it expired before it should, the server
may be broken. Fix it. On the other hand, somebody may have removed the lock
*on purpose*. We *want* the request to fail in this case.

>  - it removes the need to figure out which URL to put the locktoken on

But that's trivial to figure it. It's the URI that was used to create the
LOCK.

> locked with that token (fixing #2 even more);

See answer above.

> required (fixing #3 and making #1 less likely)

We already have a syntax for that.

> tokens before running up against length limitations)

I'm stricty against changing the *existing* protocol just to minimize header
sizes.