Subject: I-D for WebDAV methods - APPEND and PATCH

From: [email-masked]
Date: Tue, 1 Aug 2006 14:05:55 -0700 (PDT)

Hello,

My name is Suma Potluri, a graduate student at UCSC. I am working with Jim
Whitehead to define the WebDAV methods - APPEND and PATCH. We've
submitted an Internet-Draft and would like to solicit your comments on it.

Here is the URL for this I-D:

http://www.ietf.org/internet-drafts/draft-suma-append-patch-00.txt

These two methods would be very useful for WebDAV clients for efficiently
transferring partial contents of a file. And since earlier efforts to
define such methods have been unsuccessful, I think it is worth
considering these new methods and hopefully get something useful out of
it.

Thanks,
-Suma

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 01 Aug 2006 14:47:01 -0700

Hi Suma,

> considering these new methods and hopefully get something useful out of

Some initial comments:

1) You should define how this interacts with WebDAV ACL. The obvious 
solution to that is to say that APPEND and PATCH are controlled in the same 
way as PUT via ACL (i.e. write-content privilege is needed).

2) You might want to (informatively) reference 
<http://www.ietf.org/internet-drafts/draft-ietf-simple-xml-patch-ops-02.txt> 
as way of doing XML diffs.

3) Which 'format' of a resource is being patched? With the Accept header a 
client can get resource data from the server in several different formats 
if supported (e.g. text/plain or text/html). It needs to be clear that 
APPEND and PATCH are operating on the 'native' content type of the resource 
(i.e. the one indicated by the 'getcontenttype' WebDAV property) and not on 
one of the 'derived' types that the client may have retrieved.

4) Also please explain any differences from 
<http://greenbytes.de/tech/webdav/draft-dusseault-http-patch-06.html>.

5) You should defined some pre/post condition error codes for each method. 
One error to include in there is that APPEND and PATCH cannot be applied to 
a collection.

6) Should Patch-Type values be registered (IANA)?

7) BTW Is APPEND really needed? Surely a PATCH can do the same thing 
(except that PATCH cannot create a new resource)?

8) Mention that ETag/If-Match/If-None-Match can be used to prevent 
'lost-updates'.

9) Discuss interaction with LOCK.

10) Its worth pointing out somewhere that the PATCH body can in some cases 
be larger than the entire modified resource, so in that case a PUT is 
better than PATCH. i.e. a client should be smart enough to figure out which 
is better and use that.

-- 
Cyrus Daboo

---

From: Julian Reschke <[email-masked]>
Date: Wed, 02 Aug 2006 11:59:41 +0200

Hi Suma,

there's quite a lot of previous discussion on the HTTP WG mailing list 
that you should review, for instance around the thread 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2004OctDec/thread.html#msg4>).

So I would approach the topic this way:

- discuss on the HTTP WG mailing list, not here. This is not specific to 
WebDAV,

- realize that APPEND is a special case of PATCH,

- also realize that PATCH itself doesn't need a lot of work. By 
submitting a PATCH request a client asks the server to modify the 
existing resource based on the instructions in the message body. Full stop.

How the message body is to be interpreted depends on it's Content-Type 
(so you don't need "Patch-Type" at all). All the spec really needs to 
specify is at least one very simply diff format, and a MIME type for it.

Speaking of which I see use cases for at least two patch formats (one 
for textual diffs such as in the Unix diff command output), and one for 
modifying parts on binaries (seek/write/etc).

<http://greenbytes.de/tech/webdav/draft-dusseault-http-patch-04.html> of 
the PATCH draft actually used that model, and IMHO it was a mistake to 
move away from it. Quite logically, draft 05 it good serious pushback 
from Roy Fielding 
(<http://lists.w3.org/Archives/Public/ietf-http-wg/2004OctDec/0011.html>).

Best regards, Julian

---

From: Jim Whitehead <[email-masked]>
Date: Wed, 2 Aug 2006 11:29:10 -0700

>
> - discuss on the HTTP WG mailing list, not here. This is not  
> specific to WebDAV,
>

IMO, APPEND and PATCH are authoring related methods, and hence are  
quite within the scope of this mailing list. That said, discussing  
them on the HTTP list makes sense too. More eyes are better.

- Jim

---

From: "Suma Potluri" <[email-masked]>
Date: Fri, 4 Aug 2006 00:05:52 -0700 (PDT)

Hi Julian,

Thanks for your response. I wasn't aware of the earlier draft on PATCH by
Lisa Dusseault when I submitted this latest draft. I admit that I should
have done a little more research into it before submitting the new I-D. In
any case, I noticed that there hasn't been much activity in this area
since the past two years. I would be happy to collaboratively work with
Lisa Dusseault or anyone else that would be interested in these methods,
so that we could get something useful out of this.

As for the earlier discussion on the PATCH method, I have a few
questions/comments.

1) There seemed to be an extensive debate about using a new header field
for the patch format. The way I thought it was that there could
potentially be many patch-types with each patch-type working on a set of
content-types (one indicated by the 'getcontenttype' property). So each
patch-type is mapped to a set of content-types that it would work on.
Using the content-type header to indicate the patch-type that would map
against a set of content-types seemed confusing. But as I was reading the
earlier discussion, there were some convincing objections against using a
new header field. If the Content-Type header is the preferred way to
define the patch format, we could proceed with that.

2) I also noticed that one of the earlier suggestions was to use the
normal diff or diff -e as the required patch format for text files which
was what I proposed in the I-D. It is a simple and widely used diff format
that could be easily generated by the clients. For binary files we could
explore the option of using gdiff as the required diff format.

3) APPEND is a simple and efficient method, since there is no need to be
concerned about the patch formats and content-types. For clients that do
mostly appends, this could be very useful. I'd be interested to hear
comments from the client's perspective with regards to using this method
as opposed to the PATCH method.

Also I have been working on the mod_dav server and was able to get a
working implementation for the two methods - APPEND and PATCH according to
the I-D specification. I am planning to do some performance testing to see
how these methods compare to the PUT method. If we could reach an
agreement on the specification, I could change the prototype accordingly
before testing these methods.

Again thanks for your comments,
-Suma

WebDAV,
the PATCH draft actually used that model, and IMHO it was a mistake to
move away from it. Quite logically, draft 05 it good serious pushback from
Roy Fielding

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Aug 2006 09:37:31 +0200

> since the past two years. I would be happy to collaboratively work with

That's correct, and it's completely OK to start a new take on getting 
this method defined. It's just that there is a lot of useful discussion 
in these email threads.

> content-types (one indicated by the 'getcontenttype' property). So each

General remark: I think it's not a good idea to base this on WebDAV. 
This is a generic HTTP problem, and if the solution depends on WebDAV in 
any way, that's problematic. On the other hand, it would be ok to have 
appendices discussing on how these messages relate to WebDAV, versioning 
and ACLs.

> define the patch format, we could proceed with that.

You simply have to do that. RFC2616 *mandates* that when you send an 
HTTP message, the Content-Type header defines the type of the content. 
That's what it's for.

> explore the option of using gdiff as the required diff format.

True. Again, the method definition should just state that the path 
format is defined in the Content-Type header. This gives you all the 
extensibility we need (text diff, binary patches...).

The single issue with this spec (that eventually brought the previous 
attempt to a halt) is that there seems to be agreement that the spec 
should require at least one simple (!) patch format, and that needs to 
have a MIME type (either already defined or defined by this spec), and 
that needs to satisfy the IPR requirements of the IETF.

> as opposed to the PATCH method.

OK, here's an example:

Instead of saying:

         APPEND /~suma/index.txt HTTP/1.1
         Host: dav.cse.ucsc.edu
         Content-type: text/plain

         Testing Append
         Hello World Again!!!

Just use:

         PATCH /~suma/index.txt HTTP/1.1
         Host: dav.cse.ucsc.edu
         Content-type: text/entity-append

         Testing Append
         Hello World Again!!!

And then let the spec define the MIME types for "text/entity-append" 
(appending to a resource with text/* MIME type) and possibly 
"application/entity-append" (binary).

> before testing these methods.

I understand that the prototype implementation is an essential part of 
your project, and it's definitively useful to test it in practice, but 
of course proper specification is what the people over here are more 
concerned with...

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 04 Aug 2006 09:44:29 +0200

Suma Potluri schrieb:
> I agree...I assumed that the behaviour of APPEND and PATCH with regards to
> the WebDAV ACL, Etag, LOCK and the error messages related to a collection
> are the same as the PUT method (Your comments 1, 5, 8 and 9). I will
> clarify this in the I-D.

I wouldn't think it's essential, but a very short appendix probably 
doesn't hurt.

> It is a good idea to add this reference while defining the Patch-Type for
> text/xml resources.

In particular, now would be a good moment to remind the author of that 
draft that a MIME type should be defined :-).

>> one of the 'derived' types that the client may have retrieved.

Authoring content-negotiated resources is underspecified in HTTP. Common 
wisdom seems to be that the right way to do that is for the server to 
expose the Content-Location header for the variant (in HEAD or GET 
responses), and for clients always to address these in write operations. 
Everything else gets messy.

Anyway, if people want to address this (interesting) problem, please do 
not restrict it to a specific method. It deserves a generic solution, 
working with PUT, DELETE, PROPPATCH and so on...

> Yes.. the 'native' content-type of the resource is to be considered while
> patching and not one of the 'derived' types. I developed a prototype
> implementation for these two methods in the apache mod_dav module for the
> mod_dav_fs repository. And this is exactly what I am doing - checking if
> the content-type returned by the 'getcontenttype' property is supported by
> the corresponding Patch-Type.

I think this is making things more complex than necessary.

> It is true that PATCH can do the job of APPEND as well. But the APPEND
> method is much simpler and can be applied to any resource without being
> concerned about the resource type and the Patch-Type. Assuming that a
> client need to do mostly appends, this method comes in handy without the
> 'baggage' associated with the PATCH method. Also it is more efficient to
> implement at the server and may have a quicker response time.

I don't see how it's easier to specify or more efficient to execute than 
the format I proposed in my other mail.

> ...

Best regards, Julian

---

From: "Suma Potluri" <[email-masked]>
Date: Fri, 4 Aug 2006 00:04:46 -0700 (PDT)

Hi Cyrus,

Thanks for the quick feedback.

> is better and use that.

Yes, I suspect that PUT would be better in some cases even when the PATCH
body is smaller than the resource, but still large enough that the
overhead involved in processing the instructions makes it less efficient.
I am planning to do some performance testing to explore this.

As I mentioned, I developed a proof-of-concept implementation for these
methods for the mod_dav server and will be doing some performance testing
to see how they compare to the PUT method.

Thanks for pointing these issues. I realize now that they need to be
clarified in the I-D.

-Suma

---

From: Jim Whitehead <[email-masked]>
Date: Mon, 7 Aug 2006 16:22:53 -0700

> You simply have to do that. RFC2616 *mandates* that when you send  
> an HTTP message, the Content-Type header defines the type of the  
> content. That's what it's for.
>

There can potentially be multiple diff formats that use the same MIME  
type for transport. For example, I can imagine multiple diff formats  
that would use text/plain (i.e., most of the human-readable diffs in  
use by version control systems today).

> The single issue with this spec (that eventually brought the  
> previous attempt to a halt) is that there seems to be agreement  
> that the spec should require at least one simple (!) patch format,  
> and that needs to have a MIME type (either already defined or  
> defined by this spec), and that needs to satisfy the IPR  
> requirements of the IETF.
>

Can you summarize the IPR issues that affected the previous  
specification?

> And then let the spec define the MIME types for "text/entity- 
> append" (appending to a resource with text/* MIME type) and  
> possibly "application/entity-append" (binary).
>

Hmm, I would think we'd just want an octet stream, and not  
differentiate between text and binary append -- just send octets.

- Jim

---

From: Julian Reschke <[email-masked]>
Date: Tue, 08 Aug 2006 10:36:01 +0200

Jim Whitehead schrieb:

> that would use text/plain (i.e., most of the human-readable diffs in use 
> by version control systems today).

OK, there are two ways to approach that:

1) either make sure that each of the patch formats indeed has it's own 
MIME type,

2) use generic types, and have an additional request header specify the 
type of operation.

 From a spec writing point of view, 1) seems to be preferable to me. If 
you choose 2), you will essentially have to invent a new registry for 
types (IANA??), and have to maintain specifications for these types 
(which can be just pointers to existing specs). It's not entirely clear 
to me how this is better than registering MIME types for these formats.

> Can you summarize the IPR issues that affected the previous specification?

As far as I can recall, there was no single patch algorithm where the 
format description could be easily be used in a standards-track 
document. But maybe back then we haven't looked hard enough.

Of course the alternative is to define the format in the spec itself, 
but if that format is identical/similar to another one, you still risk 
IPR problems, and you have the additional problem that your own 
description may be incomplete or buggy (thus duplicating the 
specification of the patch format maybe a bad idea).

> Hmm, I would think we'd just want an octet stream, and not differentiate 
> between text and binary append -- just send octets.

Binary append would be the standard use case. Text append could take 
advantage of re-encoding. Thus, I could append to a text/* resource 
without having to know it's encoding.

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Mon, 14 Aug 2006 17:35:59 -0700

> so that we could get something useful out of this.

That would be great.  I've been letting the  PATCH document languish  
waiting for some broader consensus for a resolution to the question  
of what Content-Type means. That's the essential difference between  
patch draft -05 and draft -06.  The changes made to 06 were inspired  
by Jeff Mogul; I find his entity/instance model elegant in its  
resolution of a few modeling problems, even if it wasn't how HTTP was  
initially intended to be modeled.  However, Roy Fielding has objected  
to draft -06 and to Mogul's model and I didn't see a clear consensus  
which way to go.

Lisa

>> specific to