Subject: Re: [Fwd: Re: PUT vs strong ETags]

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Sat, 3 Dec 2005 15:11:56 -0800

On Nov 29, 2005, at 1:43 PM, Lisa Dusseault wrote:

> On Nov 29, 2005, at 1:12 PM, Julian Reschke wrote:
>>
>> Wilfredo Sánchez Vega wrote:
>>>   Question for you.
>>>   Apache HTTPD emits a weak etag for a file resource if the  
>>> timestamp on the file is the same as the current time (one second  
>>> accuracy) and a strong etag otherwise.
>>>   The reason for that logic, as I understand it, is that because  
>>> the etag generation depends on the timestamp and that the file  
>>> may change multiple times within a one-second period, an etag  
>>> generated from a timestamp matching the current time isn't reliable.
>>
>> Finally a good explanation.
>
> Do other people consider this behavior to be incorrect according to  
> the spec?  (I'm assuming the explanation is correct and it aligns  
> with what I've seen).  But it seems that within-second changes do  
> not guarantee "semantic equivalence" as required by HTTP.
>
> Lisa

   My best read of the spec leads me to believe that this is not  
correct as per the spec.

   A more correct solution may be to append something to the strong  
ETag if it is in the same timespan as now instead of using a weak  
ETag in that case.

   However, this doesn't address the problem of correctly changing  
the ETag if multiple changes occur within a subsecond timespan.  But  
then, neither does the weak ETag trick.

   If people agree that Apache HTTPd is incorrect here, I'll be happy  
to implement, lobby for and commit an appropriate change for future  
versions.

	-wsv

---

From: Lisa Dusseault <[email-masked]>
Date: Wed, 7 Dec 2005 16:52:19 -0800

On Dec 3, 2005, at 3:11 PM, Wilfredo Sánchez Vega wrote:

>> with what I've seen).  But it seems that within-second changes do not 
>> guarantee "semantic equivalence" as required by HTTP.
>>
>> Lisa
>
> ETag if it is in the same timespan as now instead of using a weak ETag 
> in that case.
>
>   However, this doesn't address the problem of correctly changing the 
> ETag if multiple changes occur within a subsecond timespan.  But then, 
> neither does the weak ETag trick.
>

Wow, that would be fantastic.  But I bet you'd get a strong reply if 
you asked such a question on the Apache list.

Lisa

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Wed, 7 Dec 2005 16:59:14 -0800

On Dec 7, 2005, at 4:52 PM, Lisa Dusseault wrote:

> Wow, that would be fantastic.  But I bet you'd get a strong reply  
> if you asked such a question on the Apache list.

   I was hoping for an answer here first.  :-)

   That is, I'd like a could of voices other than me to agree that  
Apache HTTPd's behaviour is out-of-spec.

	-wsv

---

From: Jim Whitehead <[email-masked]>
Date: Wed, 7 Dec 2005 17:49:22 -0800

I don't have a firm read on whether the Apache HTTPd behavior is non- 
compliant. However, I strongly feel that its current behavior is very  
bad for clients.

What clients really need is for the ETag to not change unless the  
content changes.

So, if an ETag starts out weak, it should stay weak. If it starts  
strong, it should stay strong. The weak->strong transition is a  
problem. I don't think clients care particularly about weak vs strong  
etags. In fact, if the server does some amount of background  
processing on the content, but returns a semantically equivalent  
representation then the server should be using weak etags  
consistently, AFAIK. For example, if a CalDAV server receives an  
event, bursts it out into its database, then reconstructs a slightly  
different XML representation that is semantically equivalent, it  
should only be using weak etags in this case to represent the fact  
that there are minor tweaks to the representation of the resource.

- Jim

On Dec 3, 2005, at 3:11 PM, Wilfredo Sánchez Vega wrote:

>>>> timestamp on the file is the same as the current time (one  
>>>> second accuracy) and a strong etag otherwise.
>> Do other people consider this behavior to be incorrect according  
>> to the spec?  (I'm assuming the explanation is correct and it  
>> aligns with what I've seen).  But it seems that within-second  
>> changes do not guarantee "semantic equivalence" as required by HTTP.
>>
>> Lisa
>
> the ETag if multiple changes occur within a subsecond timespan.   
> But then, neither does the weak ETag trick.
>
>   If people agree that Apache HTTPd is incorrect here, I'll be  
> happy to implement, lobby for and commit an appropriate change for  
> future versions.
>
> 	-wsv
>
>

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Wed, 7 Dec 2005 18:32:04 -0800

On Dec 7, 2005, at 5:49 PM, Jim Whitehead wrote:

> I don't have a firm read on whether the Apache HTTPd behavior is  
> non-compliant. However, I strongly feel that its current behavior  
> is very bad for clients.
>

   Well, I'd argue that what they *really* need is for the ETag to  
change if the content changes.

   While I agree than changing the ETag for otherwise-same content is  
inconvenient and best avoided, unless it's happening a lot (it this  
case it happens at most once), I don't see that as tragic.

   Note that this situation only arises during the one-second span of  
time that a resource is modified.  That does mean that it's almost  
always going to yield the "temporary" ETag on a PUT request, but  
rarely on a GET.

   As long as httpd is using a filesystem timestamp to compute the  
ETag, this is going to be unavoidable.  An MD5 hash would be a great  
ETag, but it, and anything that involves opening the file, is far  
more expensive to compute.  I think the trade-off here is a  
reasonable one, given the data we have available.  The other option  
is to punt and not emit an ETag, since we arguably don't have  
accurate enough information.  But I think that would be worse.

   Anyway, whether it changes or not is unrelated to the issue I'm  
angling for, which is whether the use of a weak etag is wrong.

> problem. I don't think clients care particularly about weak vs  
> strong etags. In fact, if the server does some amount of background  
> event, bursts it out into its database, then reconstructs a  
> slightly different XML representation that is semantically  
> equivalent, it should only be using weak etags in this case to  
> represent the fact that there are minor tweaks to the  
> representation of the resource.

   OK, that's a vote for "the weak etag is wrong".

	-wsv

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Dec 2005 14:02:32 +0100

Wilfredo Sánchez Vega wrote:
>> non-compliant. However, I strongly feel that its current behavior is 
>> very bad for clients.
>>
>   Well, I'd argue that what they *really* need is for the ETag to change 
> if the content changes.

+1

> inconvenient and best avoided, unless it's happening a lot (it this case 
> it happens at most once), I don't see that as tragic.

I note that I changed my own opinion several times now. One possible 
interpretation of Apache's behaviour is that in the case where content 
indeed changes several times within a second, a client may *want* not to 
have to re-sync until the resource is stable again. This is achieved by 
the current implementation.

> always going to yield the "temporary" ETag on a PUT request, but rarely 
> on a GET.
> 
>   As long as httpd is using a filesystem timestamp to compute the ETag, 
> this is going to be unavoidable.  An MD5 hash would be a great ETag, but 
> it, and anything that involves opening the file, is far more expensive 
> to compute.  I think the trade-off here is a reasonable one, given the 
> data we have available.  The other option is to punt and not emit an 
> ETag, since we arguably don't have accurate enough information.  But I 
> think that would be worse.
> 

I think I disagree here. A few days ago I asked about this on the HTTP 
mailing list, and the consensus seems to be:

- just because a server accepts a PUT and returns a (strong) ETag 
doesn't mean that it didn't rewrite the contents

- the ETag is *not* for the entity body returned with PUT, but for the 
entity you would get upon a subsequent GET/HEAD

- and yes, RFC2616 needs to be clarified

(see thread 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2005OctDec/thread.html#13>)

>   OK, that's a vote for "the weak etag is wrong".

My take is that the current drafts requirements for strong ETags and for 
ETags returned upon PUT are questionable. I think it has been 
demonstrated that

1) in some cases, weak ETags are just fine, and that

2) a requirement to return an ETag upon PUT will *need* to also clarify 
what that means

The latter optimally would be an erratum to RFC2616, which well require 
a discussion over on the HTTP mailing list, with proposed text changes 
and consensus among the readers of the list (I'm *not* volunteering to 
do this because of other priorities).

Best regards, JUlian

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Thu, 8 Dec 2005 11:06:56 -0800

On Dec 8, 2005, at 5:02 AM, Julian Reschke wrote:
>>   While I agree than changing the ETag for otherwise-same content  
>> is inconvenient and best avoided, unless it's happening a lot (it  
>> this case it happens at most once), I don't see that as tragic.
>
> I note that I changed my own opinion several times now. One  
> possible interpretation of Apache's behaviour is that in the case  
> where content indeed changes several times within a second, a  
> client may *want* not to have to re-sync until the resource is  
> stable again. This is achieved by the current implementation.

   Yes, but it would also be achieved if a strong ETag was delivered  
during the first second (including on PUT).  Use of a weak ETag  
doesn't help with that, as far as I can tell.

   My read is that the using a strong ETag in the first second would  
be more correct than the current implementation and that issuing no  
ETag at all in the first second would be the most correct.  But the  
latter may be more problematic in a practical sense than the former,  
despite being more correct.

>>> strong etags. In fact, if the server does some amount of  
>>> background processing on the content, but returns a semantically  
>>> equivalent representation then the server should be using weak  
>>> etags consistently, AFAIK. For example, if a CalDAV server  
>>> receives an event, bursts it out into its database, then  
>>> reconstructs a slightly different XML representation that is  
>>> semantically equivalent, it should only be using weak etags in  
>>> this case to represent the fact that there are minor tweaks to  
>>> the representation of the resource.
>
> I think I disagree here. A few days ago I asked about this on the  
> HTTP mailing list, and the consensus seems to be:
>
> - the ETag is *not* for the entity body returned with PUT, but for  
> the entity you would get upon a subsequent GET/HEAD
>
> (see thread <http://lists.w3.org/Archives/Public/ietf-http-wg/ 
> 2005OctDec/thread.html#13>)

   Consensus on that thread seems to be that a ETag on PUT should be  
the ETag you'd get on a subsequent GET.  I don't see any discussion  
about strong vs. weak and whether changing from strong to weak and  
back is a good idea, which is, I think the point that Jim was making.

> My take is that the current drafts requirements for strong ETags  
> and for ETags returned upon PUT are questionable. I think it has  
> been demonstrated that
>
> 2) a requirement to return an ETag upon PUT will *need* to also  
> clarify what that means
>
> The latter optimally would be an erratum to RFC2616, which well  
> require a discussion over on the HTTP mailing list, with proposed  
> text changes and consensus among the readers of the list (I'm *not*  
> volunteering to do this because of other priorities).

   I'm not sure why you think the requirements for strong ETags in  
the current HTTP spec is a problem.  I agree that clarification on  
the expected ETag on PUT would be useful.

	-wsv

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Dec 2005 21:50:24 +0100

Wilfredo Sánchez Vega wrote:
> ...
> during the first second (including on PUT).  Use of a weak ETag doesn't 
> help with that, as far as I can tell.

But with the current algorithm, the server can't return a strong ETag, 
unless it blocks updates of that content for one second. That may be 
unacceptable for some resources.

> ...
>   Consensus on that thread seems to be that a ETag on PUT should be the 
> ETag you'd get on a subsequent GET.  I don't see any discussion about 
> strong vs. weak and whether changing from strong to weak and back is a 
> good idea, which is, I think the point that Jim was making.

Yes, that question wasn't asked over there.

>> My take is that the current drafts requirements for strong ETags and 
>> for ETags returned upon PUT are questionable. I think it has been 
>> demonstrated that
>>
>> require a discussion over on the HTTP mailing list, with proposed text 
>> changes and consensus among the readers of the list (I'm *not* 
>   I'm not sure why you think the requirements for strong ETags in the 
> current HTTP spec is a problem.  I agree that clarification on the 

I was talking about the current version of RFC2518bis...

Best regards, Julian

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Thu, 8 Dec 2005 16:21:24 -0800

On Dec 8, 2005, at 12:50 PM, Julian Reschke wrote:

> But with the current algorithm, the server can't return a strong  
> ETag, unless it blocks updates of that content for one second. That  
> may be unacceptable for some resources.

   Why couldn't it?  Just append a suffix or prefix?  What HTTPd does  
right now is use the same ETag string but prefixes it with 'W/' to  
make it weak.  This accomplishes the goal of making the first-second  
etag different, but I think it should do that without making the tag  
weak, which implies something that isn't the case.

	-wsv

---

From: Julian Reschke <[email-masked]>
Date: Fri, 09 Dec 2005 10:50:49 +0100

>   Why couldn't it?  Just append a suffix or prefix?  What HTTPd does 

But where would it get the suffix from? If it had any way to store it, 
we wouldn't have the problem in the first place...

> right now is use the same ETag string but prefixes it with 'W/' to make 
> it weak.  This accomplishes the goal of making the first-second etag 
> different, but I think it should do that without making the tag weak, 
> which implies something that isn't the case.

It also accomplishes that clients will not be "forced" to sync multiple 
times within a second.

Consider a resource that changes 10 times per second, then is stable. A 
hypothetical client that does conditional GET requests will get an 
arbitrary version of the resource from within the first second, and then 
later the stable one. Seems to me that this is a good trade-off somehow.

Best regards, Julian

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Fri, 9 Dec 2005 13:16:27 -0800

Right.  I'd change it so that instead of using a weak etag, append  
the string "-potentially-spaztic" or something to the ETag during the  
first second and use the same ETag without the prefix after the first  
second.  Same result as the current implementation, but without  
implying the semantics of a weak ETag.

	-wsv

On Dec 9, 2005, at 1:50 AM, Julian Reschke wrote:

>>> ETag, unless it blocks updates of that content for one second.  
>>> That may be unacceptable for some resources.
> But where would it get the suffix from? If it had any way to store  
> it, we wouldn't have the problem in the first place...
>
>> make it weak.  This accomplishes the goal of making the first- 
>> second etag different, but I think it should do that without  
>> making the tag weak, which implies something that isn't the case.
>
> It also accomplishes that clients will not be "forced" to sync  
> multiple times within a second.
>
> Consider a resource that changes 10 times per second, then is  
> stable. A hypothetical client that does conditional GET requests  
> will get an arbitrary version of the resource from within the first  
> second, and then later the stable one. Seems to me that this is a  
> good trade-off somehow.
>
> Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 09 Dec 2005 22:31:50 +0100

Wilfredo Sánchez Vega wrote:
>   Right.  I'd change it so that instead of using a weak etag, append the 
> string "-potentially-spaztic" or something to the ETag during the first 
> second and use the same ETag without the prefix after the first second.  
> Same result as the current implementation, but without implying the 
> semantics of a weak ETag.

Hm, no.

A strong ETag allows you to do GET with Range headers, while a weak ETag 
doesn't.

Best regards, Julian

---

From: Wilfredo Sánchez Vega <[email-masked]>
Date: Fri, 9 Dec 2005 13:41:18 -0800

Ah, that's what I was looking for.  Thanks.

	-wsv

On Dec 9, 2005, at 1:31 PM, Julian Reschke wrote:

> A strong ETag allows you to do GET with Range headers, while a weak  
> ETag doesn't.