Subject: Comments on draft-dusseault-http-patch-00

From: Julian Reschke <[email-masked]>
Date: Thu, 20 Nov 2003 22:23:51 +0100

...see 
<http://www.sharemation.com/~milele/public/dav/draft-dusseault-http-patch-00.txt>

In general, I think this addresses a real performance issue, and I'd 
imagine that existing WebDAV clients and servers will soon take 
advantage of it.

Comments:

 >   2.  DeltaV extends WebDAV to do versioning.  In versioning
 >       environments, a large number of files may be updated with very
 >       small changes.  For example, a programmer may change the name of
 >       a function used in a hundred source files.  Versioning
 >       applications typically send deltas or 'diffs' to the server to
 >       modify these files, however DetaV does not yet have this
 >       functionality.

Note: it may make sense to talk to the Subversion guys about how they 
are handling this case.

 >   3.  The SIMPLE WG is devising a way to store and modify configuration
 >       information. The biggest feature missing from HTTP is the ability
 >       to modify information in a very lightweight manner, so that the
 >       client that decides to change its presence state from "free" to
 >       "busy" doesn't have to upload a large document. This can be
 >       accomplished through changes to a HTTP resource as well.

In general, this can also be done by choosing a greater granularity in 
authorable resources (URIs). That being said, changing part of the 
contents of an authorable resource is of course a useful feature in any 
case.

 >   This specification defines a new HTTP 1.1 method for patches [HTTP].
 >   A new method is necessary to improve interoperability and prevent
 >   errors. The PUT method is already defined to overwrite a resource
 >   with a complete new body, and MUST NOT be reused to do partial
 >   changes. Otherwise, proxies and caches and even clients and servers
 >   may get confused as to the result of the operation.

Note: an alternative would be to use the HTTP Extension Framework 
(RFC2774), use M-PUT and define a mandatory range extension.

 >   PATCH bodies are not cachable.  A cache MAY mark the resource
 >   identified in the Request-URI as stale if it sees a successful
 >   response to the PATCH request.

I think we need to talk about the cacheability of responses (in this 
case, MUST not). We may also need to define whether PATCH is a "safe" or
"idempotent" message (RFC2616, sectiom 9.1). I'd say that PATCH is
neither safe nor idempotent (I think it may be idempotent for a few
specific patch formats, but in general probably not).

 >   The PATCH request MUST have a body.  It MUST include the Content-Type
 >   header with a value indicating what the body type is.  It MUST be a

...what the mime type of the message body is...

 >   format that has the semantics of defining a change to an existing
 >   document (such as gdiff or vcdiff).   The PATCH request MUST also use
 >   one of the standard HTTP/1.1 mechanisms that let the server know when
 >   the request body is done. The PATCH request body length MUST NOT be
 >   indicated only by closing the connection when the body is complete,
 >   because an incomplete PATCH body could conceivably corrupt the target
 >   resource.

How would that work? If a client sends PATCH and closes the connection 
before reading the result, this wouldn't work anyway. Please elaborate.

 >   The PATCH request MUST only be used in a context which ensures that
 >   only one user may apply a patch at a time.  There are two reliable
 >   ways to do this. The first way is to find out the resource ETag at
 >   the time the body is downloaded, and use that Etag in the PATCH
 >   request to make sure the resource is still unchanged.  The second way
 >   to use WebDAV LOCK/UNLOCK [WEBDAV] to reserve the file (first LOCK,
 >   then GET, then PATCH, then UNLOCK).  PATCH collisions from multiple
 >   users are more dangerous than PUT collisions, because a PATCH that is
 >   not operating from a known base point may corrupt the resource.
 >   Therefore, if neither strong ETags nor LOCKS are available from the
 >   server, the client MUST use If-Last-Modified as a less-reliable
 >   safeguard.

In this case (:-), I'd argue for a stronger requirement (MUST only work 
for resources that support strong etags).

 >   The PATCH method can return the following errors. Please note that
 >   the notation "DAV:foobar" is merely short form for expressing "the
 >   'foobar' element in the 'DAV:' namespace".  It has meaning only in
 >   English, not on the wire.  Also note that the string error codes are
 >   not meant to be displayed but instead as machine parsable known error
 >   codes (thus there is no language code).

<default-comment>Use RFC3253 terminology, thus name the precondition,
not the error condition</default-comment>

 >   Set of defined delta formats

General comment: there should be only one (or two) MUST requirements, 
one for sending updates to arbitrary ranges in the resource, one for 
truncating the resource to a specific length. It would be optimal if 
both can be done using the same format.

 >   When the OPTIONS request addresses a specific modifiable resource,
 >   the Patch header in the response indicates which delta formats may be
 >   used for this specific resource.  When an OPTIONS request addresses
 >   the server as a whole (Request-URI = "*") the Delta header in the
 >   response indicates the union of all delta formats supported by the
 >   server.

Not implementable using the Java servlet API.

Regards, Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Julian Reschke <[email-masked]>
Date: Fri, 21 Nov 2003 09:36:20 +0100

> <http://www.sharemation.com/~milele/public/dav/draft-dusseault-http-patch-00.txt> 

Here's another thought. You could get rid of any changes to OPTIONS 
altogether by specifying:

A PATCH request with no or unknown content type must result in a 4xx 
status, and a "Patch" response header summarizing the supported content 
types. Then a client can simply submit a PATCH request with no 
Content-Type header (and no body) to get the list of supported types.

I think this would be a good simplification as it makes PATCH 
independant of all other HTTP methods.

Julian

---

From: Geoffrey M Clemm <[email-masked]>
Date: Fri, 21 Nov 2003 09:23:19 -0500

I agree that this would be an improvement over the OPTIONS technique.

Cheers,
Geoff

---

From: "Lisa Dusseault" <[email-masked]>
Date: Fri, 21 Nov 2003 16:08:39 -0800

The OPTIONS changes are as much to find out if the server supports PATCH, as
to 
indicate which patch formats are supported.  It's nice for clients to be
able
to discover all major optional features supported by the server in one
OPTIONS request, rather than trying features which might not work.

lisa

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Julian Reschke
> Sent: Friday, November 21, 2003 12:36 AM
> To: 'Webdav WG'
> Subject: Re: Comments on draft-dusseault-http-patch-00
> 
> 
> 
> Julian Reschke wrote:
> 
> > ...see 
> > 

Julian

---

From: Julian Reschke <[email-masked]>
Date: Sat, 22 Nov 2003 16:19:08 +0100

Lisa Dusseault wrote:

> The OPTIONS changes are as much to find out if the server supports PATCH, as to

That doesn't require additional functionality, there's already the 
"Allow" header.

> indicate which patch formats are supported.  It's nice for clients to be able

Is this doesn't seem to be time-critical, I'm not sure that I see the 
benefit. Not changing OPTIONS will create less dependencies and will 
possibly make it easier to add the feature to existing servers.

Note that the proposed "OPTIONS *" functionality will not work anyway. 
Is it worth keeping the remainder?

Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Mon, 24 Nov 2003 10:04:57 -0800

> Is it worth keeping the remainder?

OPTIONS * is an HTTP feature, not a WebDAV feature that we can
keep or throw away.  It's been there for years.  I haven't seen
much opposition to the feature, outside of a few people on the
WebDAV mailing list.  It's got useful semantics.

It's too bad, as Julian has pointed out in the past, that the
Java servlet design made it difficult to add stuff to OPTIONS *.
(It's not impossible, just difficult.  I can point to existence
proofs that it's possible, it just requires taking over the root
namespace with a servlet application, or doing something outside
the servlet framework.)  To me, that argues for fixes to the 
Java servlet functionality, not dropping an HTTP feature.  If
Microsoft "broke" OPTIONS * in its ISAPI design, the standards
community would not be so likely to quietly drop support for it.

Lisa

---

From: Geoffrey M Clemm <[email-masked]>
Date: Mon, 24 Nov 2003 13:31:25 -0500

One of the problem with "OPTIONS *" is that it is easy for
a client to misunderstand the scope of the "server" that is
answering the request.  Commonly, a client will assume that
it refers to "any resource under /", but this will not be
the case when different servers are handling different
resources under "/".
 
So "OPTIONS *" is reasonably well defined in simple cases
where there is one server handling the entire web site,
but we shouldn't be defining protocols that only work for the
simple cases.

Note: It doesn't particularly matter if only a "few people
on the WebDAV mailing list" make a point, if that point is
valid.  Most people building web servers only read the
WebDAV mailing list infrequently, if at all, and even fewer
of them feel comfortable or have the time to post.  So we should
make optimal use of those that are consistent readers and
posters.

Cheers,
Geoff

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Nov 2003 19:50:10 +0100

> keep or throw away.  It's been there for years.  I haven't seen

Lisa, I pointed out multiple times that your proposals (both RFC2518bis 
and the PATCH spec) indeed contradict RFC2616, which says:

"If the Request-URI is an asterisk ("*"), the OPTIONS request is 
intended to apply to the server in general rather than to a specific 
resource. Since a server's communication options typically depend on the 
resource, the "*" request is only useful as a "ping" or "no-op" type of 
method; it does nothing beyond allowing the client to test the 
capabilities of the server. For example, this can be used to test a 
proxy for HTTP/1.1 compliance (or lack thereof)."

> WebDAV mailing list.  It's got useful semantics.

Yes. Please stick to them.

> community would not be so likely to quietly drop support for it.

I agree that it would be nice if the servlet spec would allow us to do 
something here. But it doesn't. If you feel this is a big problem, raise 
it in the servlet community.

However I feel it entirely unacceptable to add new "must" level 
requirements (RFC2518bis, section 9.1) when

a) this contradicts RFC2616 and
b) it clearly is impossible to implement for a generic Java servlet.

If you feel strongly about adding this to WebDAV, please discuss it 
first. However just silently putting it in without any prior discussion 
IMHO really isn't constructive.

Regards,

Julian

---

From: Helge Hess <[email-masked]>
Date: Mon, 24 Nov 2003 20:03:32 +0100

> requirements (RFC2518bis, section 9.1) when
...
> b) it clearly is impossible to implement for a generic Java servlet.

Hm, this comment is *really* weird. A generic HTTP API is restricted 
because some API which isn't nearly as old as HTTP doesn't provide the 
required functionality?

best regards,
   Helge
-- 
OpenGroupware.org	http://www.opengroupware.org/

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Nov 2003 20:24:58 +0100

> because some API which isn't nearly as old as HTTP doesn't provide the 

No, it's not weird at all. When we discuss RFC2518bis, we are talking 
about progressing RFC2518 from "proposed" to "draft". Usually, this 
precludes adding any new requirements, unless there is already proven 
interoperability.

As a matter of fact, many of the currently existing WebDAV server 
implementations are based on the servlet API, and thus -- by defintion 
-- can't comply to the new requirement.

In addition, I have some very strong feelings (:-) about adding new 
"must" level requirements to a spec revision without

- any prior discussion (or even consensus)
- any clear benefit
- or even any notice to the working group (such as announcing the change 
after the Internet Draft was submitted)

Speaking about the PATCH method proposal - this is something new that 
doesn't need to be compatible with any existing implementation. However, 
I feel that putting additional requirements on OPTIONS is a bad idea

- because it will harm implementability in servlets and
- it introduces a dependency on other HTTP methods that doesn't seem to 
be necessary.

Finally, adding "must" level requirements to a spec that are likely to 
be ignored by many servers is really a disservice to everybody. Either 
people won't implement the spec at all, or they will and will claim 
conformance, although their implementation doesn't fully comply. In the 
latter case clients that rely on that behaviour (being a "must") will 
run into problems (in the best case, they'll not be able to use that 
feature although it's there).

Regards, Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Mon, 24 Nov 2003 12:03:35 -0800

I'm not so sure about clients assuming "any resource under /".  Do
you have actual cases there?  For example, even though OPTIONS *
would return "LOCK, UNLOCK" in the Allow header for a WebDAV
Level 2 server, WebDAV clients don't necessarily assume that all
collections really support LOCK.  They can't, because for example
IIS 5.0 collections don't.  Other situations might arise when the 
principal-URL space doesn't support LOCK even if the regular 
resource space does.  So I think it's already well understood, that
OPTIONS * means only that the server may in some sub-namespace
support a feature.
 
I didn't mean to denigrate anybody making any points, valid or otherwise.
Perhaps I should be more clear: if this is a HTTP problem, it needs to be
brought up in the context of HTTP design (e.g.
http://ftp.ics.uci.edu/pub/ietf/http/).
I haven't heard HTTP implementors (and there are many, many) outside
of the WebDAV WG complain about OPTIONS *.
 
Whenever WebDAV defines OPTIONS headers or bodies, WebDAV needs
to define their behavior in OPTIONS * as well as OPTIONS /specific/uri.
How could we not define this, when clients use OPTIONS * and servers
support it?
 
Lisa

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Geoffrey M Clemm
Sent: Monday, November 24, 2003 10:31 AM
To: 'Webdav WG'
Subject: Re: OPTIONS * (Was: RE: Comments on draft-dusseault-http-patch-00)

Cheers, 
Geoff

---

From: "Lisa Dusseault" <[email-masked]>
Date: Mon, 24 Nov 2003 12:13:36 -0800

> proxy for HTTP/1.1 compliance (or lack thereof)."

This paragraph specifically *does* say that the client can test
the capabilities of the server.  How are the RFC2518bis or PATCH 
proposals in opposition to this?

> IMHO really isn't constructive.

As you know by my disagreement, I don't see how this contradicts 
2616.  So I'm not trying to add this particular functionality to 
contradict 2616.   I've also pointed out that it's not impossible 
for an application that uses Java servlets, so I'm also not trying 
to break those applications.

As for discussion, we *are* discussing.  I-D documents *are* what
we discuss.  The IETF process is to submit proposals, that not everybody
may agree with, as I-Ds.  These are discussed and frequently change as
a result.  By the very act of publishing an Internet-Draft, I am being
anything but silent about suggested text, possible solutions, proposals,
and so on.

I appreciate your attempts to help gain consensus.  It is my opinion,
and supported by IETF practice in many working groups and by RFC 3160,
(The Tao of IETF) that publishing I-Ds with specific new proposals to 
discuss is a useful way to achieve consensus:

"Internet Drafts are tentative documents -- they're meant for readers 
to comment on, so authors can mull over those comments and decide which 
ones to incorporate in the draft."

Lisa

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Nov 2003 21:13:53 +0100

> of the WebDAV WG complain about OPTIONS *.

That's because

- RFC2518 doesn't require it and
- RFC2518bis has added it without any announcement whatsoever.

How would an HTTP implementor outside the WebDAV working group even 
*know* about that proposed change?

> to define their behavior in OPTIONS * as well as OPTIONS /specific/uri.

Fine. So please define that WebDAV doesn't put any additional semantics 
on OPTIONS *.

> How could we not define this, when clients use OPTIONS * and servers

Only some servers support it (in the way you are trying to mandate it). 
And so far, I haven't seen a client using it for anything WebDAV related 
(for the simple reason that it won't work with all servers).

Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Nov 2003 21:34:29 +0100

> proposals in opposition to this?

I guess it depends on the definition of the server. Fact is

- the servlet spec doesn't support that in general,
- it doesn't seem to be supported by Apache modules (OPTIONS * on 
http://test.webdav.org doesn't list any WebDAV methods) and
- it doesn't seem to work with IIS (just tried and I got a 404).

So as far as RFC2518 is concerned, there is demonstratibly *no* 
interoperability for this, and I never have heard of any client 
requiring it. Therefore it doesn't belong into RFC2518bis.

> to break those applications.

It's impossible for a web application that is deployed on a standard 
servlet engine, unless it's mapped to the root (and the WebDAV WG has 
already stated that it's perfectly legal to have a WebDAV server 
implementation that doesn't control the whole namespace of a particular 
HTTP host).

> ones to incorporate in the draft."

Yes. I just wish it would be easier to do that for RFC2518bis. 
Up-to-date issues/change lists would really help. The only way to spot 
this change is indeed to diff the TXT versions of the internet drafts. 
Does it really have to be that hard???

Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Mon, 24 Nov 2003 12:37:16 -0800

> - RFC2518bis has added it without any announcement whatsoever.

Ahh, here's our disconnect.  My assumption is that RFC2518 *does* 
require OPTIONS * to work, because RFC2518 requires that RFC2616
be implemented.  It's just one of many things that are part of 
WebDAV because they're part of HTTP/1.1.  It's my reading that support
of OPTIONS * is required for HTTP/1.1 therefore it was already 
required for WebDAV.  The addition to RFC2518bis was intended to
make obvious something that I thought implementors might not notice.

> *know* about that proposed change?

First, an HTTP implementor may not need to know about WebDAV changes.
Nothing in RFC2518 or RFC2518bis affects HTTP compliance or support.

Second, it's exactly by putting proposals into Internet-Drafts that
people outside a working group can most reliably learn about them.
Internet-Draft announcements are sent to a very broad list.  I myself
sometimes read Internet-Drafts for other working groups even if I'm 
not on the mailing list for that working group.

> Fine. So please define that WebDAV doesn't put any additional 

That's an interesting proposal.  How would you recommend doing that? 
Would the Allow header be incomplete?  Would the DAV: header be 
*missing* on a response to OPTIONS *?  Would DeltaV elements like
'DAV:version-history-collection-set' be illegal in the OPTIONS request
body if the Request-URI were '*'?  I fear this approach would break 
existing client implementations that do use OPTIONS * to detect 
server support.

It's a valid approach to say that WebDAV doesn't use, or doesn't
need, OPTIONS *.  However, I think that approach is less consistent, 
less compliant, and less interoperable with HTTP/1.1.  I still 
prefer for WebDAV to make clear that OPTIONS * support is required.
Then perhaps we'd eventually get out of our current situation of 
servers not supporting it properly, by having them add support.

Lisa

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Nov 2003 21:50:58 +0100

> make obvious something that I thought implementors might not notice.

Ok, let's assume HTTP *does* require this (IMHO that on the other hand 
should be clarified on the HTTP mailing list). In which case for 
RFC2518bis, it's our job to actually find out whether there's 
interoperability for this.

I just tried both Apache/mod_dav and our server. None of them supports 
it. On the other hand, I haven't seen any client failing because of 
this. Thus, if any clarification is required, then it's to say that 
clients can not rely on anything WebDAV specific being returned by 
OPTIONS *.

>>>to define their behavior in OPTIONS * as well as OPTIONS 
>>
>>/specific/uri.
>>

It is with many existing implementations.

> *missing* on a response to OPTIONS *?  Would DeltaV elements like

Same.

> body if the Request-URI were '*'?  I fear this approach would break 

Yes. That's a known defect in DeltaV, therefore the marshalling through 
OPTIONS has been deprecated, and live properties have been added instead 
(see issues 5.5_OPTIONS_BODY and 5.5_USE_PROPERTIES on 
<http://www.webdav.org/deltav/protocol/rfc3253-issues-list.htm>).

> existing client implementations that do use OPTIONS * to detect 

Show me one.

> servers not supporting it properly, by having them add support.

Well. Servers not supporting it right now doesn't seem to be any 
*practical* problem. Could you please elaborate?

Regards, Julian

---

From: "Lisa Dusseault" <[email-masked]>
Date: Mon, 24 Nov 2003 13:09:05 -0800

Well, obviously I've been reluctant to encourage implementors
to diverge from HTTP, given my assumptions that OPTIONS * is
required in HTTP/1.1.  But if that's the general consensus, 
we need to know.  

Everybody, please consider, reply, and discuss: 

1. Does HTTP/1.1 require support for OPTIONS *?  Would a HTTP
server that considered OPTIONS * to be a "bad request" be a 
compliant HTTP/1.1 server?

2. If the answer to 1 is YES, then should WebDAV servers get 
special dispensation to leave OPTIONS * unimplemented? 

3. If the answer to 2 is NO, then should WebDAV servers be 
exempt from showing WebDAV support in OPTIONS *?  

Bonus question: Can you report on any client or server HTTP or
WebDAV implementations that do, or do not, support or use 
OPTIONS *?

It may be premature to ask these questions, if the questions are
poorly formed, but at least by asking the questions we may get
feedback about how to properly ask the questions.

Implementation report: Xythos WebFile Server *does* support OPTIONS *
(and it is java servlet based, surprise surprise).  Xythos WebFile
Client does not use OPTIONS *, but it does use "OPTIONS /".

Lisa

> From: Julian Reschke [mailto:[email-masked]] 
> Sent: Monday, November 24, 2003 12:51 PM
> To: Lisa Dusseault
> Cc: 'Geoffrey M Clemm'; 'Webdav WG'

---

From: Julian Reschke <[email-masked]>
Date: Mon, 24 Nov 2003 22:19:29 +0100

> compliant HTTP/1.1 server?

1b) Would a server that does not list PROPFIND in the Allow: header for 
OPTIONS * although it does support WebDAV resources in some parts of 
it's namespace compliant?

> special dispensation to leave OPTIONS * unimplemented? 

I guess this depends of what support for "OPTIONS *" actually means, and 
this is something the HTTP mailing list can hopefully clarify.

> WebDAV implementations that do, or do not, support or use 

Servers:

- Apache/mod_dav doesn't list PROPFIND in the Allow header
- Nor does Apache/Tomcat (with the default webdav servlet installation)
- Nor does SAP Enterprise Portal Server

Clients:

- none of the clients I'm aware of relies on "OPTIONS *" returning 
anything WebDAV-specific (this includes Microsoft clients, various Adobe 
clients, MacOsX, our own and others).

> Client does not use OPTIONS *, but it does use "OPTIONS /".

Re: "OPTIONS /"? What does it use it for? Seems to be nothing essential, 
otherwise it wouldn't work with our server (which it does).

Julian

---

From: Geoffrey M Clemm <[email-masked]>
Date: Mon, 24 Nov 2003 18:23:27 -0500

> Does it really have to be that hard???

I vigorously agree with Julian here.  Each significant change (addition,
deletion, modification) to RFC2518bis should be preceded by an
email with that proposed change, and then should be documented in the 
issues
list with a pointer to that email message(and marked as OPEN, until
the change has been accepted by consensus, or at least, if no objections
have been raised in a reasonable period of time).

A new revision of the I-D is a good way to see the proposed change in
context, but just publishing the draft is not an effective way to
announce or achieve consensus on a set of changes.

Cheers,
Geoff

---

From: Larry Masinter <[email-masked]>
Date: Mon, 24 Nov 2003 16:01:26 -0800

> compliant HTTP/1.1 server?

I think the wording in RFC 2616 is unclear about requiring
that servers implement OPTIONS *. There's no explicit
language to that effect, but it does say (9.2 para 4):

   If the Request-URI is an asterisk ("*"), the OPTIONS request is
   intended to apply to the server in general rather than to a specific
   resource. Since a server's communication options typically depend on
   the resource, the "*" request is only useful as a "ping" or "no-op"
   type of method; it does nothing beyond allowing the client to test
   the capabilities of the server. For example, this can be used to test
   a proxy for HTTP/1.1 compliance (or lack thereof).

So there seems to be some assumption that HTTP/1.1 compliance
has something to do with implementing OPTIONS (otherwise how
could it be used as a test for HTTP/1.1 compliance?).

For a response, '501 Not Implemented' seems better than
'400 Bad Request'.

> special dispensation to leave OPTIONS * unimplemented? 

I'm not in favor of giving 'WebDAV servers' special dispensation.

> exempt from showing WebDAV support in OPTIONS *?  

Yes, for the reason of the above paragraph "a server's communication
 options typically depend on the resource".

---

From: Geoffrey M Clemm <[email-masked]>
Date: Tue, 25 Nov 2003 08:42:39 -0500

Larry wrote on 11/24/2003 07:01:26 PM:

> > 3. ... should WebDAV servers be 

I think Larry has identified the key point here.  We should not be
using OPTIONS to communicate resource-dependent information.  So if
the support for a method depends on what resource it
is being applied to (which it usually does), then we should not be 
marshalling
this information via "OPTIONS *", but rather by applying the query
to the appropriate resource.

Cheers,
Geoff