Subject: Re: Summary of ETag related issues in RFC2518bis

From: Geoffrey M Clemm <[email-masked]>
Date: Mon, 19 Dec 2005 22:47:17 -0500

Jim:

What about the point made by an earlier poster, namely that
a server is allowed to modify the content stored by a PUT,
so that a GET following the PUT might return different content
than was PUT (the earlier poster gave the example of a server
that expands RCS keywords on PUT).

In this case (i.e. the server modifies the content stored by
the PUT), if server returns the etag that would be returned
on a GET, and the client requests a GET with an If-None-Match
header with the etag returned by the PUT, the client will
incorrectly conclude that the text it sent with the PUT is
what would be retrieved by the GET.

So unless we are going to disallow servers from modifying the
content stored from a PUT (note that our server does not do this,
so I am speaking as a neutral party here :-), we pretty much
have to have PUT return the entity tag of the content that was
PUT, not what would be returned by the GET.

Then a client that wants to continue modifying a resource to
which it has just done a PUT, would need to do a GET with
an If-None-Match call following the PUT, to handle servers
that do this kind of rewriting on PUT.

Note that this is just a single GET, not to be confused with
the "polling" scenario described in "promotion from weak to
strong etag" thread.

Cheers,
Geoff

Jim wrote on 12/19/2005 09:11:02 PM:
>
> Julian,
> 
> Thanks for making this more clear -- you're right, there is a 
> significant issue here.
> 
> > The question here is whether an ETag returned upon PUT is for the 
> > entity the client sent (1), or for the entity the server would send 
> > upon a subsequent GET (2).
> >
> > There are cases where both will not be the same, so this needs to 
> > be clarified. In case of (2), a client will need a subsequent GET 
> > if it's planning to use the ETag for subsequent GET/Range requests.
> >
> 
> I think option #2 is the best one here (the Etag returned by PUT is 
> the one a subsequent GET would retrieve).

---

From: "Dan Brotsky" <[email-masked]>
Date: Mon, 19 Dec 2005 21:09:21 -0800

Geoff,

I don't follow your reasoning here when you say "the client will 
incorrectly conclude that the text it sent with the PUT is 
what would be retrieved by the GET."  It seems like there are three
cases:

1. The server modifies the value "on the way up", that is, before
returning from the PUT.  (This is typically how a version control system
would expand keywords, as part of the checkin.)  In this case the value
that would eventually be retrieved by GET is known and thus its etag can
be returned, even if that etag is a timestamp.

2. The server returns before modifying the value, but knows that it will
do so.  In this case a synthetic value for the etag can be generated and
returned, as long as the server takes steps to make sure that etag is
returned with the eventual GET and all GETs requested before the
modifications are complete are blocked (e.g., with "server busy").  This
etag can still be a timestamp, by the way, and can even be a timestamp
of the checkin, as long as the server associates that time with the
eventual result (which version control systems also typically do).

3. The server returns before modifying the value, and doesn't know that
a modification will take place.  (For example, the "type" of the file is
later changed so that the file undergoes keyword expansion later.)  In
this case, at the time the file is modified by the server, it should
assign a new etag, because indeed the etag returned at the time of the
PUT should not match what a client would eventually GET.  But before
that later modification is done, the etag is correct.

In no case does a client ever assume that "the text it sent with the PUT
is what would be retrieved by the GET."  That's not what the etag is
for.  The etag is to reassure the client that the value on the server
*has not changed since the PUT completed*.  No guarantees are issued
that the value doesn't change as part of the PUT; that would be a part
of the PUT semantics for that server and are outside the scope of
WebDAV.

    dan

________________________________

	From: [email-masked]
[mailto:[email-masked]] On Behalf Of Geoffrey M Clemm
	Sent: Monday, December 19, 2005 19:47
	To: [email-masked]
	Subject: Re: Summary of ETag related issues in RFC2518bis

	Jim: 

	
	content stored from a PUT (note that our server does not do
this, 

	
	Cheers, 
	Geoff

---

From: Julian Reschke <[email-masked]>
Date: Tue, 20 Dec 2005 10:12:24 +0100

Geoffrey M Clemm wrote:
> ...
> ...

As a meta comment:

I note here's another experienced person who interprets ETag-upon-PUT 
differently than people on the HTTP mailing list. *If* RFC2518bis wants 
to make normative statements about ETags in PUT, it MUST resolve this 
issue, and in a way that won't conflict with future revisions of RFC2616.

At this point I firmly believe that we can't deliver RFC2518bis in time, 
unless we drop all changes introducing new requirements that are 
non-controversial.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 20 Dec 2005 10:17:04 +0100

> of the PUT semantics for that server and are outside the scope of

Thanks, Dan :-)

So let's look at what clients are interested in again:

- they want to avoid fetching an ETag after PUT,

- in some cases, they want to be able to find out whether the server 
stored exactly what they sent,

- if they interleave PUT and PROPPATCH, they want their ETags to 
continue to work.

I believe all of these things can be accomplish by protocol extensions, 
and I'll be happy to spec them out. On the other hand, I don't think it 
would be a good idea to rush them into RFC2518bis, which was supposed to 
be finished around this time of year (if I may remember).

Best regards, Julian

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 20 Dec 2005 11:11:12 +0100

>> of the PUT semantics for that server and are outside the scope of

Yes.
> - they want to avoid fetching an ETag after PUT,

In my thinking, there is no such thing as an ETAG for the PUT body. For 
the HTTP client, the server's storage mechanism is a black box. With 
PUT it can insert bits into that box. With GET it gets bits out of that 
box. The internal wheels and springs of that box is not to be known by 
the client.

IFF a HTTP server sends an ETAG in the repsonse to a PUT, the contract 
should be as follows:
    The PUT bits changed the state of the resource to something where at 
least one varient of carries the following ETAG. If you make a 
subsequent PUT with this ETAG in an IF-* header, your PUT is likely to 
succeed during normal operations. Or the other way round: if any agent 
subsequently modifies the resource, the ETAG will change.

The last part is important for a WebDAV client, since it wants to use 
ETAG for optimistic  locking. It is not really interested in ETAG for 
caching purposes.

//Stefan

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 20 Dec 2005 09:29:49 -0500

I agree with Julian's meta-comment.  This is an interesting issue,
but it is clear that it is controversial (and the consensus
solution non-obvious), therefore I believe it is also clear that we
should not try to resolve it in 2518bis.

Cheers,
Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 20 Dec 2005 10:23:04 -0500

"Dan Brotsky" <[email-masked]> wrote on 12/20/2005 12:09:21 AM:

> is what would be retrieved by the GET."

I agree.

> That's not what the etag is for.

That is what is up for debate.  If a server has modified the text in a
way that the user needs to see (i.e., the modified text needs to be 
the basis for subsequent edits), then there needs to be an efficient way
for the client to find this out. 

One technique would be to define the etag returned by the PUT as
being the etag corresponding to PUT text.  Then if a client wants to 
perform
further edits on that text, it should issue a GET with an If-None-Match
header containing that etag.  If new text is returned, then it should
base its edits on that new text.  If no new text is returned, it knows
it can continue editing the text that was submitted earlier with the PUT.

> The etag is to reassure the client that the value on the server
> *has not changed since the PUT completed*.

How is that sufficient?  If the changes made by the server are
substantive (and should be the basis for subsequent edits), returning
the etag that corresponds to the text on the server will mislead the
client into thinking it can make changes to the text it sent up with
the PUT, when in fact it should download the server text with a GET first.

Cheers,
Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 20 Dec 2005 10:45:11 -0500

Stefan wrote on 12/20/2005 05:11:12 AM:

> IFF a HTTP server sends an ETAG in the response to a PUT, the contract 
> should be as follows:

> subsequently modifies the resource, the ETAG will change.

What about the situation where the server itself is the "agent that
modified the resource" (and did so during the PUT)?  If this modification
is substantive, the client cares both for GET (to see what is currently
on the server) and for PUT (to base subsequent edits on what is actually
on the server). 

> ETAG for optimistic  locking. It is not really interested in ETAG for 

Actually, I think a WebDAV client cares about both optimistic locking
and caching.  Also note that they are rather intimately linked, since
optimistic locking is based on knowing that the client's edits are based 
on the
text that is currently on the server, while caching is based on knowing
that what the client currently has is based on the text that is currently 
on
the server.

Cheers,
Geoff

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 20 Dec 2005 17:58:01 +0100

> on the server) and for PUT (to base subsequent edits on what is 

Yes, such servers are imaginable. No, there are not very useful:

Mathematically speaking if PUT(x) is the function in the server that 
takes the client input and modifies it, so that a subsequent 
GET(PUT(x)) defines the content a client gets back. We are talking 
about the situation where    x != GET(PUT(x)) because the other case is 
trivial.

So, the discussion is about: when is it safe for the client to continue 
editing x and when should it really be editing GET(PUT(x)) and what is 
the role of the ETAG in all of this.

So, let edit(x, z) be the content after an edit z of x, then we have 
basically two  cases:

   a) GET(PUT(edit(x,z)) == GET(PUT(edit(GET(PUT(x)),z))
      Read: the result of putting the edit of the original x and the 
putting of the edit of GET(PUT(x)) is the same. Keyword substitutions 
fall in this category. It is safe to submit old values since they are 
always overwritten by the new ones on PUT. Not a problem.

   b) GET(PUT(edit(x,z)) != GET(PUT(edit(GET(PUT(x)),z))
      In this case the server modifies the content of a PUT in a way 
which somewhat random or for example counting the number of 
modifications inside the content.

Case b) seems to be what you mean with "substantive" changes. My 
interpretation would be that the server shall either be silent with 
regards to ETAG on the PUT response. And that a client which does not 
see an ETAG in a PUT response, should make a GET afterwards. If it 
wants to avoid concurrent updates, it could lock the resource just for 
the PUT with subsequent GET.

//Stefan

> optimistic locking is based on knowing that the client's edits are 
> based on the
> that what the client currently has is based on the text that is 
> currently on
> the server.

One Counterexample is Keyword substitution. It is safe to edit and 
submit old substitution values.

---

From: Jim Luther <[email-masked]>
Date: Tue, 20 Dec 2005 09:14:35 -0800

On Dec 20, 2005, at 8:58 AM, Stefan Eissing wrote:
> Mathematically speaking if PUT(x) is the function in the server  
> that takes the client input and modifies it, so that a subsequent  
> about the situation where x != GET(PUT(x)) because the other case  
> is trivial.

As the implementer of a WebDAV file system client, the whole idea of  
x != GET(PUT(x)) makes my head hurt. For file systems to work, they  
require x == GET(PUT(x)) if there have been no PUTs from other  
clients between the PUT and the GET.

- Jim

---

From: Yves Lafon <[email-masked]>
Date: Tue, 20 Dec 2005 18:17:11 +0100 (MET)

On Tue, 20 Dec 2005, Stefan Eissing wrote:

> Mathematically speaking if PUT(x) is the function in the server that takes 
> the client input and modifies it, so that a subsequent GET(PUT(x)) defines 
> the content a client gets back. We are talking about the situation where    x 
> != GET(PUT(x)) because the other case is trivial.
>
> editing x and when should it really be editing GET(PUT(x)) and what is the 
> role of the ETAG in all of this.
>
>     Read: the result of putting the edit of the original x and the putting 
> of the edit of GET(PUT(x)) is the same. Keyword substitutions fall in this 
> category. It is safe to submit old values since they are always overwritten 
> by the new ones on PUT. Not a problem.
>
>     In this case the server modifies the content of a PUT in a way which 
> somewhat random or for example counting the number of modifications inside 
> the content.

If the server marginally modified the resource (like adding a cvs commit
timestamp), editing the current "x" copy should not generate errors (case 
a).
If changes may be more drastic in the document after the PUT so that
continuing to edit "x" client side is an issue, then it would be safe for
the server to reply to the PUT(x) with the HTTP status code 205 (Reset
Content), even if "The server has fulfilled the request and the user agent
SHOULD reset the document view which caused the request to be sent." could
be clarified in this case as "the client should issue a GET to retreive
the latest version fo the document to edit"

In any case, I don't see what's preventing the server to generate the ETag 
of the document you might get... with a GET (so, not the ETag of what has 
been put)

-- 
Yves Lafon - W3C
"Baroula que barouleras, au tiéu toujou t'entourneras."

---

From: Yves Lafon <[email-masked]>
Date: Tue, 20 Dec 2005 18:28:59 +0100 (MET)

On Tue, 20 Dec 2005, Jim Luther wrote:

> As the implementer of a WebDAV file system client, the whole idea of x != 
> GET(PUT(x)) makes my head hurt. For file systems to work, they require x == 
> GET(PUT(x)) if there have been no PUTs from other clients between the PUT and 
> the GET.

Well, as long as it is carefully hidden by the server, the client should 
care at all if there is a discrepancy between the edited document and 
what's on the server. You can have the same issue with caching.

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 20 Dec 2005 10:01:22 -0800

So when is it OK for a client doing multiple edits to do several PUTs 
in a row without intermediate GET requests?

I'll take the example of a source code file which is being edited, and 
the source control server expands keywords in comments in the file.  
The client software could issue a PUT and get back an ETag, then when 
later changes are made, issue another PUT with If-Match with the ETag 
just received.   Each time, the server could expand the keywords, 
without harm done from the client "losing" the server's changes.

In this case, multiple PUT without intermediate GET is OK to do -- the 
server is prepared to make the same changes on each PUT and doesn't 
really need the client to re-synchronize their changes.  There are 
other cases like some CalDAV cases where the server adds an internal 
event identifier or alternate address to the event.  I'd also bet that 
there are clients that already do multiple PUT requests without 
intermediate GETs especially if the client holds a lock.  But if in 
some cases the server needs the client to do a GET between two 
subsequent PUTs because the changes are important to preserve, how can 
the server accomplish that?

I believe there is a way without any additional mechanisms:

  - If the server is making changes that can be overwritten without 
harm, or if the server is making no changes, it can return an ETag in 
response to PUT and the client doesn't have to do a GET unless it later 
sees a different ETag.

  - If the server is making changes that must be preserved, then the 
server can respond to the initial PUT with a throwaway ETag, then 
immediately update the ETag of the resource to a new and more permanent 
value.  Now the client will be forced to recognize that there are new 
changes to be synched -- just as if another client had made the change 
in that period of time.  Most clients would already be compliant with 
this.

If we decided to make this kind of recommendation, we'd also have to 
specify whether it's OK to do this while the client is holding a LOCK.

Lisa

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 20 Dec 2005 15:02:58 -0500

If WebDAV were just for file systems, then life would be simpler,
but it's not, so we do have to deal with the head-hurting scenarios (:-).

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 20 Dec 2005 15:20:36 -0500

Lisa Dusseault <[email-masked]> wrote on 12/20/2005 01:01:22 PM:

> ... if in 
> in that period of time.

I'd word it somewhat differently, but this is basically what I meant by
saying that "the etag should correspond to the content specified for the 
PUT".
If the content specified by the PUT is equivalent to what will be 
retrieved
by a GET, then the PUT etag will match the GET etag.  If the content 
specified
by the PUT is not equivalent to what will be retrieved by a GET (the 
second 
case), I wouldn't call this a "throwaway" etag, but 

> Most clients would already be compliant with this.

Yes, that is why I prefer this interpretation.

> specify whether it's OK to do this while the client is holding a LOCK.

I think it is clear that the answer to this should be "yes", but stating 
it
explicitly is fine with me.

Cheers,
Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 20 Dec 2005 15:30:42 -0500

> the latest version fo the document to edit"

Making returning a 205 a MUST (or at least a SHOULD) in this case
sounds good to me.

> of the document you might get... with a GET (so, not the ETag of what 
has 

A server can of course do what it wants since the spec currently doesn't 
deal
with this issue, but doing so will cause failures in clients that use
a GET with an If-None-Match header with that etag, since the server 
will not return the new content (because the etag matches the server 
value).

Cheers,
Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 20 Dec 2005 15:42:35 -0500

Stefan Eissing <[email-masked]> wrote on 12/20/2005 11:58:01 

> the PUT with subsequent GET.

If the server includes the etag corresponding to the PUT content, 
standard If-Match and If-None-Match processing works properly. 
Isn't supporting the standard semantics better than hoping clients will
interpret the absence of an etag header in a certain way? 

Cheers,
Geoff

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Tue, 20 Dec 2005 14:21:16 -0800

And x == GET(PUT(x)) is true for DAV servers which are usable (by  
your client, anyway) as file systems.

   I don't think it's reasonable to require that all DAV servers  
cater to file system clients.  DAV wasn't designed to be an NFS  
replacement, and is, for the most part a pretty suboptimal protocol  
for building network file servers.

	-wsv

On Dec 20, 2005, at 9:14 AM, Jim Luther wrote:

> As the implementer of a WebDAV file system client, the whole idea  
> of x != GET(PUT(x)) makes my head hurt. For file systems to work,  
> they require x == GET(PUT(x)) if there have been no PUTs from other

---

From: "Dan Brotsky" <[email-masked]>
Date: Tue, 20 Dec 2005 22:08:34 -0800

We're getting closer... :) 

Yes.

> 
> - in some cases, they want to be able to find out whether the 
> server stored exactly what they sent,

No.  I claim no client wants to know this.  Ever.  If they really want
to know that the content matches what they have, then they use a server
that supports MD5 hashes and they rely on those.

If they are worried about network corruption then TCP/IP has things for
that.

> - if they interleave PUT and PROPPATCH, they want their ETags 
> to continue to work.

Yes.

> 
> I believe all of these things can be accomplish by protocol 
> extensions, and I'll be happy to spec them out. On the other 
> hand, I don't think it would be a good idea to rush them into 
> RFC2518bis, which was supposed to be finished around this 
> time of year (if I may remember).

I believe all of these things that clients want are accomplished with
what we have, as long as we have servers hand back strong etags on PUT.

    dan

---

From: "Dan Brotsky" <[email-masked]>
Date: Tue, 20 Dec 2005 22:17:16 -0800

Ah, now I begin to understand what I believe is a big misimpression on
the part of the server authors.  (Sadly, I see almost no client authors
in these discussions anymore, which has been a big part of my problem
with the WebDAV process forever and why I've been quiet for so long
now.)
 
Folks, clients are perfectly well aware that servers munge data when
they store it.  It's *way* outside the scope of WebDAV to try and match
up the server-munging semantics with clients that rely on those
semantics.  If you are a client that's looking for a faithful dumb store
and you stumble across a CVS-backed webdav server that does keyword
expansion to your graphics files, you will figure this out quickly and
stop using that server.  That doesn't mean it's not a WebDAV-compliant
server, just not one with the store semantics you were expecting.
 
Similarly, if you are a client expecting a CVS server and you use a
file-backed Apache server, it also won't meet your needs.  But it's
still WebDAV-compliant.
 
The point of this effort is to define a greatest common denominator that
can be divided evenly into the semantics of any server that's called
WebDAV compliant.  But believe me, every client in the world understands
that there may be extra factors in that server's semantics that are
relatively prime with the client's intent!
 
That's a good thing.
 
    dan

	From: [email-masked]
[mailto:[email-masked]] On Behalf Of Geoffrey M Clemm
	Sent: Tuesday, December 20, 2005 07:23
	To: webdav
	Subject: RE: Summary of ETag related issues in RFC2518bis

	"Dan Brotsky" <[email-masked]> wrote on 12/20/2005 12:09:21

	I agree. 
	
	> That's not what the etag is for. 
	
	That is what is up for debate.  If a server has modified the
text in a 
	way that the user needs to see (i.e., the modified text needs to
be 
	the basis for subsequent edits), then there needs to be an
efficient way 
	for the client to find this out. 

	being the etag corresponding to PUT text.  Then if a client
wants to perform 
	further edits on that text, it should issue a GET with an
If-None-Match 
	header containing that etag.  If new text is returned, then it
should 
	base its edits on that new text.  If no new text is returned, it
knows 
	it can continue editing the text that was submitted earlier with
the PUT. 

	How is that sufficient?  If the changes made by the server are 
	substantive (and should be the basis for subsequent edits),
returning 
	the etag that corresponds to the text on the server will mislead
the 
	client into thinking it can make changes to the text it sent up
with 
	the PUT, when in fact it should download the server text with a
GET first. 
	
	Cheers,
	Geoff

---

From: "Dan Brotsky" <[email-masked]>
Date: Tue, 20 Dec 2005 22:22:17 -0800

and caching.   
 
I strongly disagree.  I think the first author's statement is correct.
 
 Also note that they are rather intimately linked, since 
optimistic locking is based on knowing that the client's edits are based
on the 
text that is currently on the server, while caching is based on knowing 
that what the client currently has is based on the text that is
currently on 
the server.  
 
No.  Optimistic locking is based on knowing that the client's edits are
being applied to the content the client last sent to the server, and are
not overwriting other client's edits.  Caching is based on knowing that
the content you *last received* is the same as what you *would receive*
on your next get.
 
HTTP clients, whether webdav or not, know they must have *received* the
content associated with an etag in order to avoid fetching it again.
That's caching.
 
WebDAV clients only need to know that they *sent* the content associated
with an etag in order to avoid overwriting someone else's edits.  That's
optimistic locking.
 
    dan

---

From: "Dan Brotsky" <[email-masked]>
Date: Tue, 20 Dec 2005 22:28:50 -0800

> As the implementer of a WebDAV file system client, the whole 
> idea of x != GET(PUT(x)) makes my head hurt. For file systems 
> to work, they require x == GET(PUT(x)) if there have been no 
> PUTs from other clients between the PUT and the GET.

Yes, but there are many other webdav servers out there that are *not*
file systems.  I strongly believe that WebDAV was not intended just for
file-system access.

By the way, everyone, please note that GET(PUT(x)) == x does *not* imply
PUT(x) == x.

    dan

---

From: "Dan Brotsky" <[email-masked]>
Date: Tue, 20 Dec 2005 22:33:37 -0800

As to your last question: Yes it's OK and no the server needs to break
the lock if it does this (because it's indistinguishable from another
client's edit).  Not all clients will work efficiently against servers
that unexpectedly munge data after PUTs are complete but  that's life.

    dan

> -----Original Message-----
> From: Lisa Dusseault [mailto:[email-masked]] 
> Sent: Tuesday, December 20, 2005 10:01
> To: Dan Brotsky
> Cc: [email-masked]; Geoffrey M Clemm
> So when is it OK for a client doing multiple edits to do 
> several PUTs in a row without intermediate GET requests?
> 
> I'll take the example of a source code file which is being 
> edited, and the source control server expands keywords in 
> comments in the file.  
> The client software could issue a PUT and get back an ETag, 
> then when later changes are made, issue another PUT with 
> If-Match with the ETag 
> In this case, multiple PUT without intermediate GET is OK to 
> do -- the server is prepared to make the same changes on each 
> PUT and doesn't really need the client to re-synchronize 
> their changes.  There are other cases like some CalDAV cases 
> where the server adds an internal event identifier or 
> alternate address to the event.  I'd also bet that there are 
> clients that already do multiple PUT requests without 
> intermediate GETs especially if the client holds a lock.  But 
> if in some cases the server needs the client to do a GET 
> between two subsequent PUTs because the changes are important 
> to preserve, how can the server accomplish that?
> 
>   - If the server is making changes that can be overwritten 
> without harm, or if the server is making no changes, it can 
> return an ETag in response to PUT and the client doesn't have 
> to do a GET unless it later sees a different ETag.
> 
>   - If the server is making changes that must be preserved, 
> then the server can respond to the initial PUT with a 
> throwaway ETag, then immediately update the ETag of the 
> resource to a new and more permanent value.  Now the client 
> will be forced to recognize that there are new changes to be 
> synched -- just as if another client had made the change in 
> that period of time.  Most clients would already be compliant 
> with this.
> 
> If we decided to make this kind of recommendation, we'd also 
> have to specify whether it's OK to do this while the client 
> is holding a LOCK.
> 
> Lisa
> 
> what would 
> > be retrieved by the GET."  It seems like there are three
> > cases:
> >
> > returning from the PUT.  (This is typically how a version control 
> > system would expand keywords, as part of the checkin.)  In 
> this case 
> > the value that would eventually be retrieved by GET is 
> known and thus 
> > its etag can be returned, even if that etag is a timestamp.
> >
> > 2. The server returns before modifying the value, but knows that it 
> > will do so.  In this case a synthetic value for the etag can be 
> > generated and returned, as long as the server takes steps 
> to make sure 
> > that etag is returned with the eventual GET and all GETs requested 
> > before the modifications are complete are blocked (e.g., 
> with "server 
> > busy").
> > This
> > 3. The server returns before modifying the value, and doesn't know 
> > that a modification will take place.  (For example, the 
> "type" of the 
> > file is later changed so that the file undergoes keyword expansion 
> > later.)  In this case, at the time the file is modified by 
> the server, 
> > it should assign a new etag, because indeed the etag 
> returned at the 
> > time of the PUT should not match what a client would 
> eventually GET.  
> > But before that later modification is done, the etag is correct.
> >
> > In no case does a client ever assume that "the text it sent 
> with the 
> > PUT is what would be retrieved by the GET."  That's not 
> what the etag 
> > is for.  The etag is to reassure the client that the value on the 
> > server *has not changed since the PUT completed*.  No 
> guarantees are 
> > issued that the value doesn't change as part of the PUT; 
> that would be 
> > a part of the PUT semantics for that server and are outside 
> the scope 
> > of WebDAV.
> >
> >     dan
> >
> >
> >
> > 	> > entity the client sent (1), or for the entity the 
> server would 
> > send

---

From: Yves Lafon <[email-masked]>
Date: Wed, 21 Dec 2005 11:08:56 +0100 (MET)

On Tue, 20 Dec 2005, Lisa Dusseault wrote:

> In this case, multiple PUT without intermediate GET is OK to do -- the server 
> is prepared to make the same changes on each PUT and doesn't really need the 
> client to re-synchronize their changes.  There are other cases like some 
> CalDAV cases where the server adds an internal event identifier or alternate 
> address to the event.  I'd also bet that there are clients that already do 
> multiple PUT requests without intermediate GETs especially if the client 
> holds a lock.  But if in some cases the server needs the client to do a GET 
> between two subsequent PUTs because the changes are important to preserve, 
> how can the server accomplish that?

How about reusing 205 (Reset Content) when the server knows a subsequent 
PUT would be harmful?

---

From: Stefan Eissing <[email-masked]>
Date: Wed, 21 Dec 2005 12:11:45 +0100

Am 20.12.2005 um 21:42 schrieb Geoffrey M Clemm:
> Stefan Eissing <[email-masked]> wrote on 12/20/2005 
> 11:58:01 AM:

I was looking for a way to tell the client that it has to refetch the 
content. If the server just answers with a temp ETAG, the client will 
find out on the next PUT that "someone" changed the content and 
probably needs to ask its user what to do about it.

Yves proposed 205 answer however sounds like a much better approach.

//Stefan

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 21 Dec 2005 09:17:22 -0500

I agree that returning a 205 is a good way to signal this.

But I also maintain that returning a 205 *and* returning an
ETAG for the PUT content is even better, since existing clients
that use If-Match and If-None-Match (and don't know about the
new 205 convention) will work properly.

Was there some reason why you thought returning a 205 and
not returning an ETAG was preferable?

Cheers,
Geoff

Stefan Eissing <[email-masked]> wrote on 12/21/2005 06:11:45

---

From: Julian Reschke <[email-masked]>
Date: Wed, 21 Dec 2005 15:54:11 +0100

> that unexpectedly munge data after PUTs are complete but  that's life.

For the record: I think that linking the ETag behavior for PUT to the 
fact whether the resource was locked or not would be a really bad idea.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 21 Dec 2005 15:56:05 +0100

Dan Brotsky wrote:
> with the WebDAV process forever and why I've been quiet for so long now.)

Dan, it's good that you're back here. At least Apple and ourselves 
(remember, we also have a client) are following this as well. Of course 
it would be good if Microsoft was around here as well :-).

> relatively prime with the client's intent!

+1 on that summary.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 21 Dec 2005 16:02:23 +0100

> that supports MD5 hashes and they rely on those.

It seems to me that many client implementors want this, because they may 
want to do Get/Range requests.

> what we have, as long as we have servers hand back strong etags on PUT.

1) Servers may not be able to return strong ETags upon PUT. Again, let's 
consider adding an indicator to PUT that let's the server know the 
client really needs a strong ETag, so that the server can optimize it's 
behavior for that case.

2) It seems to me that we can't rule out that servers touch the ETag 
upon PROPPATCH (for instance, because they are indeed updating metadata 
in the file content, such as with XMP). In which case telling the server 
to return the new ETag upon PUT seems to be a very good idea.

Best regards, Julian

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 21 Dec 2005 11:02:43 -0500

I agree.

> to return the new ETag upon PUT seems to be a very good idea.

Did you mean, return the new ETag upon PROPPATCH?

Cheers,
Geoff

---

From: Julian Reschke <[email-masked]>
Date: Wed, 21 Dec 2005 17:07:00 +0100

> Did you mean, return the new ETag upon PROPPATCH?

Yep. Thanks for the correction :-)

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 21 Dec 2005 11:14:35 -0500

> Yep. Thanks for the correction :-)

In that case, yes, I agree.

Also note that there is no issue wrt PUT-content vs. GET-content for
PROPPATCH, so that happily that issue does not arise for a PROPPATCH
etag (:-).

Cheers,
Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 21 Dec 2005 11:20:44 -0500

I like Dan's answer better than mine, so I switch my vote to what Dan
says below.

Cheers,
Geoff

> > ... we'd also

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 21 Dec 2005 11:26:51 -0500

> fact whether the resource was locked or not would be a really bad idea.

Julian: I agree with you, but did you think Dan was suggesting otherwise,
or were you just agreeing with Dan's statement (or at least, with the
"yes, it's OK" part)?   I am assuming that you were not disagreeing
with Dan, since I don't believe he suggests anything that would make ETag
behavior depend on whether the resource was locked or not.

Cheers,
Geoff

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 21 Dec 2005 08:29:59 -0800

Hey Dan, I'm a client developer now and Chandler definitely cares about 
both optimistic locking and caching :)  We want to be able to share 
collections of arbitrary files between users, but those collections 
should also be accessible offline.  The client can be responsible for 
merging content in case there's already a change to a cached resource 
changed while offline.

Lisa

On Dec 20, 2005, at 10:22 PM, Dan Brotsky wrote:

> No.  Optimistic locking is based on knowing that the client's edits 
> are being applied to the content the client last sent to the server, 
> and are not overwriting other client's edits.  Caching is based on 
> knowing that the content you *last received* is the same as what you 
> *would receive* on your next get.
>  
> HTTP clients, whether webdav or not, know they must have *received* 
> the content associated with an etag in order to avoid fetching it 
> again.  That's caching.
>  
> WebDAV clients only need to know that they *sent* the content 
> associated with an etag in order to avoid overwriting someone else's 
> edits.  That's optimistic locking.
>  
>     dan
>

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 21 Dec 2005 13:30:33 -0500

"Dan Brotsky" <[email-masked]> wrote on 12/21/2005 01:22:17 AM:

>> I think a WebDAV client cares about both optimistic locking 
>> and caching. 
> 
> I strongly disagree.

Like Lisa, our client writers care about both optimistic locking
and caching.  And I don't think we are the only ones that feel that way.

currently on 
 
> *would receive* on your next get.

Our client writers care about all three, i.e. optimistic locking (whose
definition we appear to agree on), GET caching (i.e., what you define as
caching), and re-use of the content submitted for a PUT (included in my
more general definition of caching).

> the content associated with an etag in order to avoid fetching it 

I am happy to use "caching" to only mean "GET caching"
if that's what you prefer, but our clients also care about the more 
general
form of caching that I described, whatever name we use for it.
One of the points of this discussion is to determine whether clients 
should
be able to use etags returned by PUT for this other kind of caching.

> edits.  That's optimistic locking.

Our WebDAV clients also care about performance, so they also care about
caching, not just about optimistic locking.

Cheers,
Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Wed, 21 Dec 2005 13:40:18 -0500

I agree that there may be a big misimpression on the part of server
authors (but probably we differ on who we think those server authors
are :-).

In particular, I don't think anyone is suggesting/expecting that
we match up server munging semantics ... all that some of us
are suggesting/expecting is that there be a well-defined way in
which a server can let a client know that it has significantly modified
the submitted text, preferably in a way that will automatically work
for (at least some) existing clients.  That is the reason for
focusing on etags, since an existing client that does an If-Match
or If-None-Match with the etag returned by PUT will work properly
on server-munged content if we return an etag for the PUT content
that is different from the etag for the subsequent GET content.

Using the 205 return code is also desireable, although this would
primarily be for the benefit of future clients, since I doubt many
existing clients have special handling for when 205 is returned by PUT.

Cheers,
Geoff

"Dan Brotsky" <[email-masked]> wrote on 12/21/2005 01:17:16 AM:

> Ah, now I begin to understand what I believe is a big misimpression 
> on the part of the server authors.  (Sadly, I see almost no client 
> authors in these discussions anymore, which has been a big part of 
> my problem with the WebDAV process forever and why I've been quiet 
> for so long now.)
> 
> they store it.  It's *way* outside the scope of WebDAV to try and 
> match up the server-munging semantics with clients that rely on 
> those semantics.  If you are a client that's looking for a faithful 
> dumb store and you stumble across a CVS-backed webdav server that 
> does keyword expansion to your graphics files, you will figure this 
> out quickly and stop using that server.  That doesn't mean it's not 
> a WebDAV-compliant server, just not one with the store semantics you
> were expecting.
> 
> The point of this effort is to define a greatest common denominator 
> that can be divided evenly into the semantics of any server that's 
> called WebDAV compliant.  But believe me, every client in the world 
> understands that there may be extra factors in that server's 
> semantics that are relatively prime with the client's intent!
> 
> That's a good thing.
> 
>     dan
> 
> From: [email-masked] [mailto:[email-masked]] 

> On Behalf Of Geoffrey M Clemm

perform

---

From: Julian Reschke <[email-masked]>
Date: Thu, 22 Dec 2005 07:32:22 +0100

> behavior depend on whether the resource was locked or not.

I agree that servers can rewrite the content upon PUT, even if the 
resource is locked. I do not agree that they need to break the lock to 
do that.

Best regards, Julian

---

From: Yves Lafon <[email-masked]>
Date: Thu, 22 Dec 2005 09:18:03 +0100 (MET)

On Thu, 22 Dec 2005, Julian Reschke wrote:

> I agree that servers can rewrite the content upon PUT, even if the resource 
> is locked. I do not agree that they need to break the lock to do that.

Agreed, lock is there to ensure other clients won't edit the resource. The 
server can do what it wants.

---

From: "Dan Brotsky" <[email-masked]>
Date: Thu, 22 Dec 2005 00:20:58 -0800

Geoff, Lisa,
 
Thanks for the clarification.
 
I think cacheing clients such as yours really care whether the server
munges the received content or not.  My perspective is that you need to
actually do a get, then, after your put.  I don't think you should rely
on the received etag;  you will need the MD5 to avoid the get.
 
    dan

	From: [email-masked]
[mailto:[email-masked]] On Behalf Of Geoffrey M Clemm
	Sent: Wednesday, December 21, 2005 10:31
	To: webdav
	Subject: RE: Summary of ETag related issues in RFC2518bis

	"Dan Brotsky" <[email-masked]> wrote on 12/21/2005 01:22:17

	and caching.  And I don't think we are the only ones that feel
that way. 
	
	>> that what the client currently has is based on the text that
is currently on 
	  
	> *would receive* on your next get. 
	
	Our client writers care about all three, i.e. optimistic locking
(whose 
	definition we appear to agree on), GET caching (i.e., what you
define as 
	caching), and re-use of the content submitted for a PUT
(included in my 
	more general definition of caching). 

	I am happy to use "caching" to only mean "GET caching" 
	if that's what you prefer, but our clients also care about the
more general 
	form of caching that I described, whatever name we use for it. 
	One of the points of this discussion is to determine whether
clients should 
	be able to use etags returned by PUT for this other kind of
caching. 
	
	> edits.  That's optimistic locking. 
	
	Our WebDAV clients also care about performance, so they also
care about 
	caching, not just about optimistic locking. 
	
	Cheers, 
	Geoff

---

From: Geoffrey M Clemm <[email-masked]>
Date: Thu, 22 Dec 2005 08:54:01 -0500

I believe that you are basing your conclusion on the assumption
that the authoring client can ignore the rewriting that is being
done by the server.  In that case, I (and I'm reasonably sure, Dan)
would agree that there is no need for the server to cancel the LOCK.

But the case being discussed here is where the server has
rewritten the text and wants the client to base future edits on the 
rewritten text.  You may say "my server would never do that" or even
"I would never use a server that did that" (in which case you might
not want to spend any more time on this thread :-), but in case 
the server did do that (and I have scenarios in which I can imagine
a server wanting to do that), I believe that automatically breaking
the lock is a good way of getting this point across to existing clients
(who don't know about the 205 convention).

Cheers,
Geoff

Yves wrote on 12/22/2005 03:18:03 AM:
> > I agree that servers can rewrite the content upon PUT, even if 
theresource

---

From: Julian Reschke <[email-masked]>
Date: Thu, 22 Dec 2005 15:01:21 +0100

Geoff,

could you post an example scenario? I really have a hard time 
understanding why it would make a difference in practice...

Best regards, Julian

---

From: "Dan Brotsky" <[email-masked]>
Date: Thu, 22 Dec 2005 18:46:53 -0800

Couldn't agree more with Geoff.
 
    dan

	From: [email-masked]
[mailto:[email-masked]] On Behalf Of Geoffrey M Clemm
	Sent: Thursday, December 22, 2005 05:54
	To: webdav
	Subject: Re: Summary of ETag related issues in RFC2518bis

	done by the server.  In that case, I (and I'm reasonably sure,
Dan) 
	would agree that there is no need for the server to cancel the
LOCK. 
	
	But the case being discussed here is where the server has 
	rewritten the text and wants the client to base future edits on
the   
	rewritten text.  You may say "my server would never do that" or
even 
	"I would never use a server that did that" (in which case you
might 
	not want to spend any more time on this thread :-), but in case 
	the server did do that (and I have scenarios in which I can
imagine 
	a server wanting to do that), I believe that automatically
breaking 
	the lock is a good way of getting this point across to existing
clients 
	(who don't know about the 205 convention). 
	
	Cheers, 
	Geoff 
	
	Yves wrote on 12/22/2005 03:18:03 AM:
	> > I agree that servers can rewrite the content upon PUT, even
if theresource